Mocap with Optitrack and AR Kit
Fall 2021 - While preparing to move into a new studio space, we chose to work on a virtual-only motion capture project. We used Metahumans with Optitrack Motive and AR Kit on iPhone.
The Cold Within
Based on the 1960s poem by James Patrick Kinney, this short-film was our vision of wisdom being passed down to a new generation. Assets were assembled in UE4 using Metahuman models, a Houdini landscape, and Niagara lighting effects. Camera moves and renders were done in UE4 Sequencer. Final edit with audio was assembled in Adobe Premiere.
Primary Contributions:
Motion Capture & Cinematography
Our project explored combining data from OptiTrack and ARKit’s Live Link Face to drive a virtual actor. The environment was built in Unreal Engine, and the character was produced in MetaHuman Creator.
Optitrack Motion Capture
Using v1.10, we captured performances using a simple 8 camera setup. We discovered that such few cameras resulted in tracking points being occluded very easily when trying to record the children’s sitting poses. This system also did not have hand or face tracking, so we had to find different solution for driving expressions for our three characters.
Maya Retargeting
Once data for each character was captured, we used Maya Human IK to retarget the older Optitrack skeleton with the Metahumans skeleton. Using a combination of Optitrack’s own software and Maya, we were able to clean up our animations to a reasonable degree.
ARKit Facial Capture
While recording our voice actor’s monologue, we also captured her expressions using LiveLink in Unreal Engine; ensuring speech and expressions were aligned. This system was incredibly convenient and fairly capable for a setup that simply used an iPhone or iPad connected to Unreal Engine over wifi.
Special thanks to Bailey’s grandmother for trying multiple takes with us!
