Blender 2.8 facial mocap using OpenCV and webcam

webcam facial This is a topic that many people are looking for. is a channel providing useful information about learning, life, digital marketing and online courses …. it will help you have an overview and solid multi-faceted knowledge . Today, would like to introduce to you Blender 2.8 facial mocap using OpenCV and webcam. Following along are instructions in the video below:

“This video. I ll share how to set up facial motion capture in blender using using opencv in a web camera let s get started this is a character called available for free from blender. Cloud has these little nodes that allow you to control the bones. He is one that controls the jaw and lips and up and down moves a jaw up and down and side to side moves mouth wider and closed.

And i noticed that when you look at all the properties. It shows these python objects. Showing what the property name is and that got me thinking what if i can script this character to move with opencv now blender has his scripting tab. Which allows you to script objects within blender using python so i ve created two python scripts.

One adds add on to the user interface so when we re in object mode. It adds. Opencv animation and a little capture button that when we press. It opens.

The capture window and starts doing an open cv capture of control points on your face. These control points. Are then mapped to the character and we show how open or close the mouth is aisle raised or lowered. The eyebrows are and the position of the head.

Which direction..

It s facing and then we can right click or press escape. And it closes the capture window. While we re capturing points on the face. We re also recording the animation keys for each one of the objects that are moving so if we start recording within blender.

If we hit space. We can then record all these capture points and then play them back as an animation later you go back and then press play and so we have that recorded animation within blender. So now i ll show you how to set this up in blender. The steps that i ll be going through are installing opencv downloading the vincent 3d model creating python scripts in blender and downloading the facial landmark database to start off with we ll have to find out where blenders python is so that we can install the opencv libraries.

Now can be in one of two places blender. Could use the operating systems. Python or there could be a standalone version of python packaged with blender. There s an easy way you can find out where your version of python is open up the scripting tab within blender.

And then in the python console type in imports s. And then c star. Path. Now here my python.

Libraries folder is in our system folder meaning..

That i m using the operating systems version of python. So the terminal. I can just type in python and around this same version and instance of python 373. Now the other option is if your version of blender.

Uses its own self contained python in this case. The libraries files would be in the installation folder of blender. So to run. This instance of.

Python you would go to the installation folder go to. The. 280. Path.

Python. Bin and then. Type in. Python 37.


Now. That we know. Which command runs the same instance of python from the terminal. We can install the opencv libraries.

First will ensure that pipe is installed the python package. Installer. Now. This command to update.

Pip and then finally we ll run this command to install all the packages. We need all these commands are listed in their description below now that we have the opencv library is installed we can test that it works by going back to blender opening up the python terminal and typing. In import c. V2.

Then c. V2. Dot underscore. Underscore version.


Underscore. And i should tell us that we have the right version installed to download the vincent 3d model go to cloud blenderorg scroll down to the characters section and then we can download the free model. Vincent and now we ll set up the python scripts to animate the character open up the vincent model. That you just downloaded and go to the scripting tab now create a new script and copy and paste the contents of the opencv anam operator python file linked in the description now we can give the file a name and then we can repeat the same process with the opencv and m.

Dot. Py file. Now you ll need to open. Both python scripts and click the run script button and while you re there click the register checkbox this will run the script.

When you open the file the next time now the last topic is downloading the facial landmarks database this database is based off of our bug database which is for research purposes. Only so that means no commercial use the link for this database is in the description. You ll need to download this and update the past which you saved it to in the opencv am operator python file now you should have a working blender capsure you ll need to do a little bit of calibration by moving your face around at the start it determines the extent to which everything moves now if you get this working let me know down in the description. Otherwise if you d like to see more content like this don t forget to ” .


Thank you for watching all the articles on the topic Blender 2.8 facial mocap using OpenCV and webcam. All shares of are very good. We hope you are satisfied with the article. For any questions, please leave a comment below. Hopefully you guys support our website even more.


Leave a Comment