We propose an algorithm for taming Normalizing Flow models - changing the probability that the model will produce a specific image or image category. We focus on Normalizing Flows because they can calculate the exact generation probability likelihood for a given image. We demonstrate taming using models that generate human faces, a subdomain with many interesting privacy and bias considerations. Our method can be used in the context of privacy, e.g., removing a specific person from the output of a model, and also in the context of debiasing by forcing a model to output specific image categories according to a given distribution. Taming is achieved with a fast fine-tuning process without retraining from scratch, achieving the goal in a matter of minutes. We evaluate our method qualitatively and quantitatively, showing that the generation quality remains intact, while the desired changes are applied.
- CelebA dataset.
- Base Glow model.
- Pretrained ArcFace face vlassifier.
- Pretrained celebA attribute classifier.
conda create -n taming python=3.11
conda activate taming
cd taming_norm_flows
pip install -r requirements.txtCELEBA_ROOT="/path/to/celeba/dataset"
ARCFACE_CKPT="/path/to/arcface/model"
BASE_MODEL_PATH="/path/to/base/pretrained/model"the forget_identity attribute in the configs/forget_identity.json determines the id of the identity in celeba to forget. See all parameters by running python taming_an_identity -h
python taming_an_identity.py --config configs/forget_identity.jsonWill be uploaded soon
This repository builds on code from the on the glow-pytorch repository. In addition, we also use code from ArcFace for a face classifer.
@inproceedings{malnick2024taming,
title={Taming Normalizing Flows},
author={Malnick, Shimon and Avidan, Shai and Fried, Ohad},
booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
pages={4644--4654},
year={2024}
}