Part-aware-Prototype-Network

Official Implementation of Part-aware Prototype Network for Few-shot Semantic Segmentation (ECCV 2020)

Installation

Dependencies

• Python 3.7.3
• PyTorch 1.3.1
• torchvision 0.4.2
• Cuda version 10.0
• pykeops
• fast_slic(Option)

Get Started

git clone 
cd PPNet-PyTorch
mkdir logs
mkdir outputs

Data Preparation for VOC Dataset

1. Download Pascal VOC dataset

Please go to PANet and download VOC dataset and put them under FewShotSeg-dataset/Pascal folder.

# symlink the pascal dataset
mkdir -p FewShotSeg-dataset/Pascal
ln -s /path_to_pascal_dataset/ FewShotSeg-dataset/Pascal/

2. Download pretrained model

Download the ResNet50 and Resnet101 weights and put them under FewShotSeg-dataset/cache/ folder.

3. Download the unlabel superpixel data

Download the unlabel superpixel from here and put it under FewShotSeg-dataset/Pascal/superpixel folder. If you want to generate your own superpixel data, please follow the fast_slic.

Training & Evaluation in Command Line for Pascal VOC

# Train baseline model 
sh script/train_fewshot.sh

# Train part model
sh script/train_part.sh

# Train part+semantic branch model
sh script/train_part_sem.sh

# Train part + semantic branch + unlabel data model
sh script/train_graph.sh

Inference by pretrained model

Change the ckpt_dir in script to your pretrained model path.

# Test baseline model 
sh script/test_fewshot.sh

# Test part model
sh script/test_part.sh

# Test part+semantic branch model
sh script/test_part_sem.sh

# Test part + semantic branch + unlabel data model
sh script/test_graph.sh

1-way 1-shot

1-way 1-shot	Download link	meanIoU
PANet*	model	49.10
+ PAP	model	50.40
+ PAP + SEM	model	51.50
+ PAP + SEM + UD	model	52.84

log files are available here

N-way K-shot final model

N-way K-shot	Setting	Download link	meanIoU
1-way 5-shot	+ PAP + SEM + UD	model	62.97
2-way 1-shot	+ PAP + SEM + UD	model	51.65
2-way 5-shot	+ PAP + SEM + UD	model	61.30

Visualization for Pascal VOC

Citation

Please consider citing our paper if the project helps your research. BibTeX reference is as follows.

@article{liu2020part,
  title={Part-aware Prototype Network for Few-shot Semantic Segmentation},
  author={Liu, Yongfei and Zhang, Xiangyi and Zhang, Songyang and He, Xuming},
  journal={arXiv preprint arXiv:2007.06309},
  year={2020}
}

References

Part of our code are based on PANet