Learning to Compose Hypercolumns for Visual Correspondence

This is the implementation of the paper "Learning to Compose Hypercolumns for Visual Correspondence" by J. Min, J. Lee, J. Ponce and M. Cho. Implemented on Python 3.7 and PyTorch 1.0.1.

For more information, check out project [website] and the paper on [arXiv].

Requirements

• Python 3.7
• PyTorch 1.0.1
• tensorboard
• scipy
• pandas
• requests
• scikit-image

Conda environment settings:

conda create -n dhpf python=3.7
conda activate dhpf

conda install pytorch=1.0.1 torchvision cudatoolkit=10.0 -c pytorch
pip install tensorboardX
conda install -c anaconda scipy
conda install -c anaconda pandas
conda install -c anaconda requests
conda install -c anaconda scikit-image
conda install -c anaconda "pillow<7"

Training

Training DHPF with strong supervision (keypoint annotations) on PF-PASCAL and SPair-71k
(reproducing strongly-supervised results in Tab. 1 and 2):

python train.py --supervision strong \
                --lr 0.03 \
                --bsz 8 \
                --niter 100 \
                --selection 0.5 \ 
                --benchmark pfpascal \ 
                --backbone {resnet50, resnet101}

python train.py --supervision strong \
                --lr 0.03 \
                --bsz 8 \
                --niter 5 \
                --selection 0.5 \ 
                --benchmark spair \ 
                --backbone {resnet50, resnet101}

Training DHPF with weak supervision (image-level labels) on PF-PASCAL
(reproducing weak-supervised results in Tab. 1):

python train.py --supervision weak \
                --lr 0.1 \
                --bsz 4 \
                --niter 30 \
                --selection 0.5 \  
                --benchmark pfpascal \
                --backbone {resnet50, resnet101}

Testing

We provide trained models available on [Google drive].

PCK @ α_img=0.1 on PF-PASCAL at different μ:

Trained models at differnt μ	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1
weak (res50)	77.3	79	79	79.3	79.6	80.7	81.1	80.7
weak (res101)	80.3	81.2	82.1	80.1	81.7	80.9	81.3	81.3
strong (res50)	87.7	89.1	88.9	88.5	89.4	89.1	89	89.5
strong (res101)	88.7	90	90.7	90.2	90.1	90.6	90.6	90.4

PCK @ α_img=0.1 on SPair-71k at μ=0.5:

Trained models at μ=0.5	PCK
weak (res101)	27.7
strong (res101)	37.3

Reproducing results in Tab. 1, 2 and 3:

python test.py --backbone {resnet50, resnet101} \
               --benchmark {pfpascal, pfwillow, caltech, spair} \
               --load "path_to_trained_model"

BibTeX

If you use this code for your research, please consider citing:

@InProceedings{min2020dhpf, 
   title={Learning to Compose Hypercolumns for Visual Correspondence},
   author={Juhong Min and Jongmin Lee and Jean Ponce and Minsu Cho},
   booktitle={ECCV},
   year={2020}
}