As you have seen in the last section, we created a character lip sync demo driven by real-time microphone input. While the demo is fun to play, it starts to show what is possible, but it could hardly be mistaken for real speech. After all, we don't just flap our jaws when we speak. Speech and vocalization are a very complex process requiring a synchronization of your larynx, tongue, lips, and even teeth. So as much as we understand how to visualize sound, we will need to go a little further into understanding the key elements in our speech patterns in order to build a more authentic lip syncing simulation.
Animators have been performing realistic lip syncing for many years now, and there are plenty of well established practices for creating speech animation, which means we will be able to use some of those learning and best practices. However, real-time...
