It's for waymo open dataset and real-time rendering (low FPS).
CLI
conda create -n $env_name python=$version
# install torch with correct version
pip install -r requirements.txt
pip install ./diff-gaussian-rasterization
pip install ./pointops2
pip install ./simple-knn
-
Download individual scene in waymo open dataset scene flow labels
-
Use EmerNeRF to preprocess the individual scene, the preprocessed scene data folder should be named as xxx (e.g. 000, 001, 002)
-
Modify the
./config/waymo/xxx.yaml
, especiallysource_path
(input) andmodel_path
(output)
Train (no implementation for load checkpoint and continue train):
python train.py --config configs/waymo/xxx.yaml
Render:
python eval.py --config configs/waymo/xxx.yaml --pth output_path/xxx/chkpntxxx.pth --mode 0 --viewDir 0
- render point cloud with random color in real time
- render depth map in real time
- render training camera trace in real time
- render free camera in real time
- save training camera trace as video
- save depth map as video
- evaluate metrics (PSNR, SSIM, LPIPS)
Mode 1, 4 support interacitve real time rendering
W S A D : move forward/backward/left/right
Q E: move upwards/downwards
Hold left or right mouse: rotation
Roll: change focal
Resize window: the render resolution also resize
- Use interacitve window and select a new view in your local computer.
- Close the window, and a
view.obj
file would be saved in working directory. - Move the file to server working directory
- render with mode 5/6