This repository presents GPSDiffusion-SDXL, an upgraded version of our GPSDiffusion:
Shadow Generation Using Diffusion Model with Geometry Prior [pdf] [supp]
Haonan Zhao, Qingyang Liu, Xinhao Tao, Li Niu, Guangtao Zhai
Accepted by CVPR 2025.
We replace the original Stable Diffusion 1.5 (SD 1.5) with the more advanced Stable Diffusion XL (SDXL) for enhanced generation performance. The following visual comparison demonstrates the quality improvements of SDXL over its predecessor SD 1.5. From left to right, we show the composite image, foreground mask, the result based on SD 1.5, the result based on SD XL, and ground-truth.
We also present a visual comparison between SDXL and SD 1.5 using different random seeds. From left to right, we show the composite image, foreground mask, four SD1.5 outputs with varying seeds, four SDXL outputs with varying seeds, and the ground-truth. Notably, the SDXL version demonstrates significantly improved output stability.
- Clone this repo: git clone https://github.com/bcmi/GPSDiffusion-Object-Shadow-Generation-SDXL.git
- Download the DESOBAv2 dataset from [Baidu Cloud] (access code: bcmi) or [Dropbox]. Unzip
desobav2-256x256.rarto./data/, and rename it todesobav2. - Download the checkpoints from [Baidu Cloud] (access code: bcmi). Unzip
pretrained_models.zip.
conda create -n GPSDiffusion python=3.8
conda activate GPSDiffusion
pip install -r requirements.txt
git clone https://github.com/huggingface/diffusers
cd diffusers
pip install -e .
pip install -r requirements.txt
And initialize an 🤗Accelerate environment with:
accelerate config
Or for a default accelerate configuration without answering questions about your environment:
accelerate config default
accelerate launch train_GPSDiffusion_sdxl.py
accelerate launch test_GPSDiffusion_sdxl.py
python post_processing.py
Parts of this code were derived from:
https://github.com/huggingface/diffusers