No mop cap suite required, just an RGB web camera
Green Screen video source from storyblocks.com
3D Pose Unity Barracuda ($0 for indie or $500 for commercial)
Hardware minimum require:
GPU: GTX 1080 TI/ RTX 2060
CPU: I7 or equivalent
RAM: 4GB
Let's get started.
Download the code. Github link https://github.com/digital-standard/ThreeDPoseUnityBarracuda
2. Open Unity (Version 2019.3 and up is required), make a new 3d project, Download Barracuda in package manager
3. Import: extract the download then drag the whole folder into your unity project
4. Download onnx file : https://digital-standard.com/threedpose/models/Resnet34_3inputs_448x448_20200609.onnx
5.Open the Unity project with Unity Editor and put the onnx file in /Assets/Scripts/Model/
In this action, the onnx file is being converted into NNModel type of Barracuda automatically.
6. Open the sample scene file in scene folder
7.Set model
Select BarracudaRunner in Hierarchy, place Resnet34_inputs_448x448_20200609import setting Script in NNModel parameter under inspector tab. The script is under Assets/Scripts/Model folder.
8. Hit play to debug
9. Choose a different video
Select Video Player in Hierarchy, replace videos in video clip parameter under inspector tab.
Tips for video capture that use for motion capture
actor stay in center
higher resolution (4k), higher frame rate(60fps)
fit clothing
good lighting on actor
easy background, green screen if possible
remove shadow
10. Use web Camera for real time tracking
Select main texture in hierarchy, under Video capture Script component
check the "use web cam"
11.Record The Motion
Easy motion recorder
Download the code
Github Link: https://github.com/neon-izm/EasyMotionRecorder
Import to unity
Drag easy motion prefab to the scene
In Hierarchy, select easy motion recorder, under motion data recorder component under inspector, choose your character for the animator parameter
When play, hit R to start record, X to stop Record
The recorded animation has been saved to Asset/Resource folder
To export the animation, select the clip, go to inspector tab and right click, choose export as humanoid animation clips.
Then select the animation clip, set the root transform position (Y) base upon: original
12. Re-target the animations to a different character
Import a rigged model
Select the character, under inspector tab,
Rig tab, Animatiuon type: selectHumanoid,
Avatar definition: click on configure
Then create a new animation controller
Inside animation controller, add new state, drag the animation clip in it
Back to the scene, select the character in hierarchy, under inspector tab, animator component, add the new animation controller to the controller parameter.
Hooray!
The animation is not perfect. It will need some clean up. I'll have another post about motion capture data clean up in Maya coming soon.
Thank you for reading.