Monocular, One-stage, Regression of Multiple 3D People

ROMP is a concise one-stage network for multi-person 3D mesh recovery from a single image. It can achieve real-time inference speed on a 1070Ti GPU.

BEV is built on ROMP to further explore multi-person depth relationships and support all ages. To be released on this repo. Stay tuned.

We provide user cross-platform API to run on Linux / Windows / Mac.

Table of contents

• Table of contents
• Features
• News
• Getting started
  • Installation
  • Try on Google Colab
• How to use it
  • Inference
  • Export
  • Train
  • Evaluation
  • Bugs report
• Citation
• Acknowledgement

Features

features

News

2022/03/27:Relative Human dataset has been released.
2022/03/18: Simple version of ROMP for all platform. Let's pip install simple-romp. See the guidance for details
Old logs

Getting started

Installation

pip install simple-romp

To run in real time, please refer to install.md for installation.

Try on Google Colab

It allows you to run the project in the cloud, free of charge. Google Colab demo.

How to use it

Inference

Please refer to the guidance.

Export

Please refer to expert.md to export the results to fbx files for Blender usage.

Train

For training, please refer to installation.md for full installation. Please prepare the training datasets following dataset.md, and then refer to train.md for training.

Evaluation

Please refer to evaluation.md for evaluation on benchmarks.

Bugs report

Please refer to bug.md for solutions. Welcome to submit the issues for related bugs. I will solve them as soon as possible.

Citation

@InProceedings{BEV,
author = {Sun, Yu and Liu, Wu and Bao, Qian and Fu, Yili and Mei, Tao and Black, Michael J},
title = {Putting People in their Place: Monocular Regression of 3D People in Depth},
booktitle = {CVPR},
year = {2022}
}

@InProceedings{ROMP,
author = {Sun, Yu and Bao, Qian and Liu, Wu and Fu, Yili and Michael J., Black and Mei, Tao},
title = {Monocular, One-stage, Regression of Multiple 3D People},
booktitle = {ICCV},
year = {2021}
}

Acknowledgement

We thank all contributors for their help!

We thank Peng Cheng for his constructive comments on Center map training.

Here are some great resources we benefit:

• SMPL models and layer is borrowed from MPII SMPL-X model.
• Some functions are borrowed from HMR-pytorch and SPIN.
• The evaluation code and GT annotations of 3DPW dataset is brought from 3dpw-eval and VIBE.
• 3D mesh visualization is supported by vedo, EasyMocap, minimal-hand, Open3D, and Pyrender.

Please consider citing their papers.