Low budget Full body motion capture with 3D pose Barracuda motion capture and Unity

No mop cap suite required, just an RGB web camera

Green Screen video source from storyblocks.com

3D Pose Unity Barracuda ($0 for indie or $500 for commercial)

Hardware minimum require:

GPU: GTX 1080 TI/ RTX 2060

CPU: I7 or equivalent

RAM: 4GB

Let's get started.

1. Download the code. Github link https://github.com/digital-standard/ThreeDPoseUnityBarracuda

2. Open Unity (Version 2019.3 and up is required), make a new 3d project, Download Barracuda in package manager

3. Import: extract the download then drag the whole folder into your unity project

4. Download onnx file : https://digital-standard.com/threedpose/models/Resnet34_3inputs_448x448_20200609.onnx

5.Open the Unity project with Unity Editor and put the onnx file in /Assets/Scripts/Model/

In this action, the onnx file is being converted into NNModel type of Barracuda automatically.

6. Open the sample scene file in scene folder

7.Set model

Select BarracudaRunner in Hierarchy, place Resnet34_inputs_448x448_20200609import setting Script in NNModel parameter under inspector tab. The script is under Assets/Scripts/Model folder.

8. Hit play to debug

9. Choose a different video

Select Video Player in Hierarchy, replace videos in video clip parameter under inspector tab.

Tips for video capture that use for motion capture

1. actor stay in center

2. higher resolution (4k), higher frame rate(60fps)

3. fit clothing

4. good lighting on actor

5. easy background, green screen if possible

6. remove shadow

10. Use web Camera for real time tracking

Select main texture in hierarchy, under Video capture Script component

check the "use web cam"

11.Record The Motion

Easy motion recorder

Download the code

Github Link: https://github.com/neon-izm/EasyMotionRecorder

Import to unity

Drag easy motion prefab to the scene

In Hierarchy, select easy motion recorder, under motion data recorder component under inspector, choose your character for the animator parameter

When play, hit R to start record, X to stop Record

The recorded animation has been saved to Asset/Resource folder

To export the animation, select the clip, go to inspector tab and right click, choose export as humanoid animation clips.

Then select the animation clip, set the root transform position (Y) base upon: original

12. Re-target the animations to a different character

Import a rigged model

Select the character, under inspector tab,

Rig tab, Animatiuon type: selectHumanoid,

Avatar definition: click on configure

Then create a new animation controller

Inside animation controller, add new state, drag the animation clip in it

Back to the scene, select the character in hierarchy, under inspector tab, animator component, add the new animation controller to the controller parameter.

Hooray!

The animation is not perfect. It will need some clean up. I'll have another post about motion capture data clean up in Maya coming soon.

Thank you for reading.