Gallery

[guochengqian]

Demo

A demo example using an ironman image from internet. We run Magic123 without textural inversion, taking 1 hour on a 32G V100 to finish.

Renderings:

magic123-results.mp4

Meshes:
ironman.zip

Command:

• Run Magic123 coarse stage without textural inversion, takes ~40 mins

  export RUN_ID='default-a-full-body-ironman'
  export DATA_DIR='data/demo/ironman'
  export IMAGE_NAME='rgba.png'
  export FILENAME=$(basename $DATA_DIR)
  export dataset=$(basename $(dirname $DATA_DIR))
  CUDA_VISIBLE_DEVICES=0 python main.py -O \
  --text "A high-resolution DSLR image of a full body ironman" \
  --sd_version 1.5 \
  --image ${DATA_DIR}/${IMAGE_NAME} \
  --workspace out/magic123-${RUN_ID}-coarse/$dataset/magic123_${FILENAME}_${RUN_ID}_coarse \
  --optim adam \
  --iters 5000 \
  --guidance SD zero123 \
  --lambda_guidance 1.0 40 \
  --guidance_scale 100 5 \
  --latent_iter_ratio 0 \
  --normal_iter_ratio 0.2 \
  --t_range 0.2 0.6 \
  --bg_radius -1 \
  --save_mesh
  

• Run Magic123 fine stage without textural inversion, takes around ~20 mins

  export RUN_ID='default-a-full-body-ironman'
  export RUN_ID2='dmtet'
  export DATA_DIR='data/demo/ironman'
  export IMAGE_NAME='rgba.png'
  export FILENAME=$(basename $DATA_DIR)
  export dataset=$(basename $(dirname $DATA_DIR))
  CUDA_VISIBLE_DEVICES=0 python main.py -O \
  --text "A high-resolution DSLR image of a full body ironman" \
  --sd_version 1.5 \
  --image ${DATA_DIR}/${IMAGE_NAME} \
  --workspace out/magic123-${RUN_ID}-${RUN_ID2}/$dataset/magic123_${FILENAME}_${RUN_ID}_${RUN_ID2} \
  --dmtet --init_ckpt out/magic123-${RUN_ID}-coarse/$dataset/magic123_${FILENAME}_${RUN_ID}_coarse/checkpoints/magic123_${FILENAME}_${RUN_ID}_coarse.pth \
  --iters 5000 \
  --optim adam \
  --known_view_interval 4 \
  --latent_iter_ratio 0 \
  --guidance SD zero123 \
  --lambda_guidance 1e-3 0.01 \
  --guidance_scale 100 5 \
  --rm_edge \
  --bg_radius -1 \
  --save_mesh 
  

[guochengqian]

Failure Case

Not optimal for human reconstruction. Suggest to use SMPL model as an additional prior, which I believe will directly boost the performance to realism.

human-examples.mp4