Unreal Engine’s 4.25 Release Pairs with iOS App to Capture Facial Movements
The 4.25 release of Unreal Engine can pair with a new iOS app, Live Link for Unreal Engine, to capture facial expressions of live actors. The new app piggybacks off of the sophisticated facial recognition and motion tracking software already included in iPhones. Thus, if your iPhone already has a depth camera and ARKit capabilities, you can access these new Unreal Engine features for free by downloading the app in the App Store.
Based on an online demo, the raw abilities of this new app is pretty impressive. Large facial motions such as mouth movements, eye movements, and other large facial movements are smoothly captured in real time. It works by mapping a designers pre-built animations to ARKit tracked facial expressions. For example, an iPhone with these abilities notes when you blink your right eye and has an event trigger for this action. With Live Link for Unreal Engine, you can map that event to an animation of your character blinking their right eye. Now whenever your live actor blinks their right eye, your game character’s right eye blink animation will be triggered.
I am excited to see how this new technology will be implemented. The app was just released so it is too early to see examples of it’s full capacity. Based on the demo’s I have watched and example photos released in the press release, a potential issue is capturing fine facial movements. For example, it is the fine muscle movements around the eyes that give us subtle cues about the true emotional state of those around us. With out this, characters can seem flat and lifeless pushing the character into the uncanny valley. This isn’t a large issue for game design because those details could be added in once the majority of the work is done with this capture tool. However, I do question whether Live Link for Unreal Engine will see wide spread appeal for live streams if this is indeed a limitation.
Are you a developer and trying out the Live Link for Unreal Engine app? Post a link to what you are working on below. Or if you have thought on how this new technology could be used, please share in the comments below.