Code for the paper "Kinematic Motion Retargeting via Neural Latent Optimization for Learning Sign Language"
- PyTorch Tensors and Dynamic neural networks in Python with strong GPU acceleration
- pytorch_geometric Geometric Deep Learning Extension Library for PyTorch
- Kornia a differentiable computer vision library for PyTorch.
- HDF5 for Python The h5py package is a Pythonic interface to the HDF5 binary data format.
The Chinese sign language dataset can be downloaded here.
The pretrained model can be downloaded here.
Training
CUDA_VISIBLE_DEVICES=0 python main.py --cfg './configs/train/yumi.yaml'Inference
CUDA_VISIBLE_DEVICES=0 python inference.py --cfg './configs/inference/yumi.yaml'
We build the simulation environment using pybullet, and the code is in this repository.
After inference is done, the motion retargeting results are stored in a h5 file. Then run the sample code here.
Real-world experiments could be conducted on ABB's YuMi dual-arm collaborative robot equipped with Inspire-Robotics' dexterous hands.
We release the code in this repository, please follow the instructions.
If you find this project useful in your research, please cite this paper.
@article{zhang2022kinematic,
title={Kinematic Motion Retargeting via Neural Latent Optimization for Learning Sign Language},
author={Zhang, Haodong and Li, Weijie and Liu, Jiangpin and Chen, Zexi and Cui, Yuxiang and Wang, Yue and Xiong, Rong},
journal={IEEE Robotics and Automation Letters},
year={2022},
publisher={IEEE}
}