Unlock Multi-Person 3D Motion Capture with CoMotion: A Comprehensive Guide

Tired of single-person pose estimation? CoMotion revolutionizes 3D motion capture by detecting and tracking multiple people simultaneously from a single camera. This guide provides everything you need to get started with this groundbreaking technology, perfect for researchers and developers alike.

What is CoMotion and Why Should You Care?

CoMotion is an innovative approach to multi-person 3D motion capture. Unlike traditional methods, it excels in crowded scenes and handles complex poses with ease; plus, it provides significant upgrades:

• Real-Time Tracking: CoMotion updates poses directly from new images, enabling robust online tracking, even through occlusions.
• Single Camera Simplicity: Forget complex multi-camera setups. CoMotion works with a single, monocular camera.
• Detailed 3D Poses: Capture accurate and detailed 3D poses of multiple individuals, opening doors to advanced motion analysis.

Quick Start: Installing and Running CoMotion

Ready to jump in? Follow these simple steps to install and run CoMotion on your system using the detailed CoMotion installation guide:

1. Create a Conda Environment:

   conda create -n comotion -y python=3.10
   conda activate comotion
   pip install -e '.[all]'
   

   Copy
2. Download Pre-trained Models:

   bash get_pretrained_models.sh
   

   Copy
   This downloads the necessary checkpoints for both the detection and refinement stages.
3. Get the SMPL Body Model:
   • Download the neutral SMPL body model (version 1.1.0) from the official SMPL website.
   • Copy basicmodel_neutral_lbs_10_207_0_v1.1.0.pkl to src/comotion_demo/data/smpl/SMPL_NEUTRAL.pkl (renaming the file).

Running the CoMotion Demo: Video and Image Examples

CoMotion offers flexible demo options for both video and image inputs. Here's how to run them:

• Video Demo:

  python demo.py -i path/to/video.mp4 -o results/
  

  Copy

  Customize your runs by using arguments like --start-frame and --num-frames. Disable visualization with --skip-visualization.

• Single Image Demo:

  python demo.py -i path/to/image.jpg -o results/
  

  Copy

  CoMotion automatically detects image inputs based on the file suffix (.png, .jpeg, .jpg). The 3D pose estimation will be processed.

Troubleshooting Common Issues

Encountering problems? Here are quick solutions to common issues:

• "libc++.1.dylib not found" Error: Resolve this by running conda install libcxx.
• "XOpenDisplay: cannot open display" Error (Headless Rendering):
  • Start a virtual display using Xvfb :0 -screen 0 640x480x24 & export DISPLAY=:0.0.
  • Install Xvfb if needed: apt install xvfb.

Standalone Single-Image Multi-Person Pose Estimation

Utilize the detection stage independently from CoMotion for single-image multi-person pose estimation tasks. No tracking required! The comotion_detection_checkpoint.pt is perfect for leveraging the existing multi-person 3D motion capture base.

Citing CoMotion

If you find CoMotion useful in your research, please cite the following paper:

@inproceedings{newell2025comotion,
 title = {CoMotion: Concurrent Multi-person 3D Motion},
 author = {Alejandro Newell and Peiyun Hu and Lahav Lipson and Stephan R. Richter and Vladlen Koltun},
 booktitle = {International Conference on Learning Representations},
 year = {2025},
 url = {https://openreview.net/forum?id=qKu6KWPgxt},
}

License Information

The sample code is released under the LICENSE terms, and the model weights are released under the MODEL LICENSE terms. Please refer to these licenses for usage details.