Crowd Density & Segmentation Prediction

This project is the implementation and extension of the previous work, which is based on patches. For more details, please refer to the original paper.

Citation

Please cite the original paper in your publications if it helps your research:

@inproceedings{zhang2015cross,
  title={Cross-scene crowd counting via deep convolutional neural networks},
  author={Zhang, Cong and Li, Hongsheng and Wang, Xiaogang and Yang, Xiaokang},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={833--841},
  year={2015}
}

Requirements: software

Caffe and MATLAB 2014a or above are required. Notice: A modified version of Caffe is requried.

Installation

1. Clone the repository

# Make sure to clone with --recursive
git clone https://github.com/wk910930/crowd_density_segmentation.git

2. Build Caffe (Code is private for now)

# Currently the code is not allowed to be public.

Prepare training data

We will call the directory that you cloned $CROWD_ROOT. We have prepared a toy dataset called dataset_01, which is used to demonstrate how to generate training data. Place your own dataset(s) under the data for further research.

1. Generate density map
   • Go to the scripts folder.

   cd $CROWD_ROOT/scripts

   • Run the gen_density_map.m. It will generate density map for each frame.
   • Run the gen_patch.m. It will crop the frames and density maps.
2. Convert to Leveldb
   • Training the model needs three leveldb datasets, i.e. data_db, regression_db and segment_db, which stores the frames, segmentation masks and density maps, respectively.
   • Converting to Leveldb needs the Caffe, which is private for now.

   # Notice:
   #   jpg_id is the file list contains all frame crops, which can be generated by the command `ls sub_frames/*.jpg`. It looks like:
   #       xxxxxxx01.jpg
   #       xxxxxxx02.jpg
   #       xxxxxxx03.jpg
   #       xxxxxxx04.jpg
   #       ...
   #   Similar to png_id and mat id.
   
   # Example:
   #
   # Convert frame crops to leveldb
   ./caffe/build/tools/convert_imageset.bin -backend leveldb /path/to/sub_frames/ jpg_id data_db
   # Convert segment crops to leveldb
   ./caffe/build/tools/convert_imageset.bin -backend leveldb -gray /path/to/sub_segs/ png_id segment_db
   # Convert density crops to leveldb
   ./caffe/build/tools/convert_matset.bin -backend leveldb -height 256 -width 256 -varname crop_dens /path/to/sub_density/ mat_id regression_db