Skip to content

Latest commit

 

History

History
53 lines (32 loc) · 2.25 KB

README.md

File metadata and controls

53 lines (32 loc) · 2.25 KB
Raw

Text and Image Guided 3D Avatar Generation and Manipulation

Installation and Usage

  1. Install the dependencies in env.yml

    conda env create -f env.yml

    conda activate latent3d-env

  2. Install pytorch3d by running the following:

    import sys
import torch
import subprocess
version_str="".join([
    f"py3{sys.version_info.minor}_cu",
    torch.version.cuda.replace(".",""),
    f"_pyt{torch.__version__.split("+")[0].replace(".", "")}"
])
subprocess.run(f"conda install https://anaconda.org/pytorch3d/pytorch3d/0.6.2/download/linux-64/pytorch3d-0.6.2-{version_str}.tar.bz2", shell=True)

  3. Download the pre-trained TBGAN model from this link: https://ibug.doc.ic.ac.uk/resources/tbgan/.

  • Note: You will be required to fill the form for 'End User Licence Agreement'.

  1. Place the pre-trained TBGAN model snapshot in the '/models/snapshots' directory.

  2. Download the pre-trained ArcFace model

    wget https://www.dropbox.com/s/kzo52d9neybjxsb/model_ir_se50.pth?dl=0 -O model_ir_se50.pth

  3. Place the pre-trained ArcFace model in the '/models/snapshots' directory.

Text-Based Manipulations

Running the 'optimize.py' with the required parameters and list of text prompts for manipulation, the rendered manipulated 3D faces and their originals will be saved under the direcory './results'.

python3 optimize.py --seed [SEED] --mode [MODE] --text_list [TEXT_LIST] --num_epochs [NUM_EPOCHS] --lambda_id [ID_COEFF] --lambda_l2 [L2_COEFF] --learning_rate [LEARNING_RATE]

  • Example: python3 optimize.py --seed 1000 --mode text-based --text_list "happy human" "happy person" --num_epochs 100 --lambda_id 0.01 --lambda_l2 0.0 --learning_rate 0.01 --folder_title happy

Image-based Manipulations

Running the 'optimize.py' with given parameters and an image for manipulation, the rendered manipulated 3D faces and their originals will be saved under the direcory './results'.

python3 optimize.py --seed [SEED] --mode [MODE] --image_path [IMAGE_PATH] --num_epochs [NUM_EPOCHS] --lambda_id [ID_COEFF] --lambda_l2 [L2_COEFF] --learning_rate [LEARNING_RATE]

Generate Mesh from Intermediate Latent Vector

python3 sample.py --functionality latent_vector --result_subdir ./results/003-optimize