The Very Simple Reason that Facial Motion Capture is Terrible

I have seen a lot of different research videos and papers on the topic of facial performance capture.  They put any number of markers on a face, video the markers with cameras  and they create a 3d mesh out of all of this data.

And than usually the results look smooth but ugly.  And not really acting that you would want to spend much time watching.

And often, going through this research, you will find images like this.


I've been drawing faces a lot lately, really simple faces.  Really expressive faces.  And you know what I always make sure to get right, even if everything else isn't great?  I always go for the eyes.  And than the mouth, and then the eye brows and maybe the nose.  And usually after that, the point and emotion of the drawing is pretty obvious.

So I look at these performance capture technologies and the first and most obvious thing that they don't capture is the eye.  And it isn't just the rotational position of the eye that is important to capture, it is the white space around the eye.

It's about the amount of the iris and pupil that is visible, the shape that cuts it off and the white space that surrounds it.

If you are to translate a performance capture onto a digital character, it is not the amount of rotation of the eye in the socket that must be transferred, it is the end result of that rotation that makes an image that we as humans with advanced facial recognition abilities can interpret that must be captured and transferred.

Human facial recognition is good, but it has been tricked by drawings for a long time.  It is just a matter on focusing on the important parts and letting everything else fall to the side.

Kyler