Releases: duskvirkus/alias-free-gan
Releases · duskvirkus/alias-free-gan
Version 1.1.0
Bug fixes:
- Fixed kimg counting when using --accumulate_grad_batches. kimgs was under counting by factor of accumulate_grad_batches. If using v1.0.0 training notebook multiply the kimg count by 4 to get the correct number and use --start_kimg_count when resuming training to override the file name parsing.
Non breaking updates:
- Refactored download and loading pretrained models. Added code to load pretrained models in all generate scripts.
- Updated interpolation notebook to include commands to convert frames to .mp4 and .gif
- Added more examples to readme and fixed a few typos.
Version 1.0.0
Release Overview
In theory this version is a stable working version of Alias-Free GAN supporting rosinality's unofficial implementation.
Changes from the Rosinality Repository
- Converted to run using pytorch lighting - Supports GPU and TPU training with many built in options. For more information see: https://pytorch-lightning.readthedocs.io/en/1.4.2/
- Added CPU op library for TPU support
- prepare_data.py renamed to convert_dataset.py and moved to
scripts/convert_dataset.py - generate.py adapted and put in a script under
scripts/rosinality_generate.py
Scripts Added
Trainer
Creates an Alias-Free GAN instance and trains the model saving checkpoints based on kimgs (thousands of image).
Using Alias-Free GAN version: 1.0.0
usage: trainer.py [-h] --dataset_path DATASET_PATH [--resume_from RESUME_FROM]
--size SIZE [--batch BATCH] [--lr_g LR_G] [--lr_d LR_D]
[--r1 R1] [--augment AUGMENT] [--augment_p AUGMENT_P]
[--ada_target ADA_TARGET] [--ada_length ADA_LENGTH]
[--ada_every ADA_EVERY]
[--stylegan2_discriminator STYLEGAN2_DISCRIMINATOR]
[--save_sample_every_kimgs SAVE_SAMPLE_EVERY_KIMGS]
[--save_checkpoint_every_kimgs SAVE_CHECKPOINT_EVERY_KIMGS]
[--start_kimg_count START_KIMG_COUNT]
[--stop_training_at_kimgs STOP_TRAINING_AT_KIMGS]
[--sample_grid SAMPLE_GRID] [--logger [LOGGER]]
[--checkpoint_callback [CHECKPOINT_CALLBACK]]
[--default_root_dir DEFAULT_ROOT_DIR]
[--gradient_clip_val GRADIENT_CLIP_VAL]
[--gradient_clip_algorithm GRADIENT_CLIP_ALGORITHM]
[--process_position PROCESS_POSITION]
[--num_nodes NUM_NODES] [--num_processes NUM_PROCESSES]
[--devices DEVICES] [--gpus GPUS]
[--auto_select_gpus [AUTO_SELECT_GPUS]]
[--tpu_cores TPU_CORES] [--ipus IPUS]
[--log_gpu_memory LOG_GPU_MEMORY]
[--progress_bar_refresh_rate PROGRESS_BAR_REFRESH_RATE]
[--overfit_batches OVERFIT_BATCHES]
[--track_grad_norm TRACK_GRAD_NORM]
[--check_val_every_n_epoch CHECK_VAL_EVERY_N_EPOCH]
[--fast_dev_run [FAST_DEV_RUN]]
[--accumulate_grad_batches ACCUMULATE_GRAD_BATCHES]
[--max_epochs MAX_EPOCHS] [--min_epochs MIN_EPOCHS]
[--max_steps MAX_STEPS] [--min_steps MIN_STEPS]
[--max_time MAX_TIME]
[--limit_train_batches LIMIT_TRAIN_BATCHES]
[--limit_val_batches LIMIT_VAL_BATCHES]
[--limit_test_batches LIMIT_TEST_BATCHES]
[--limit_predict_batches LIMIT_PREDICT_BATCHES]
[--val_check_interval VAL_CHECK_INTERVAL]
[--flush_logs_every_n_steps FLUSH_LOGS_EVERY_N_STEPS]
[--log_every_n_steps LOG_EVERY_N_STEPS]
[--accelerator ACCELERATOR]
[--sync_batchnorm [SYNC_BATCHNORM]] [--precision PRECISION]
[--weights_summary WEIGHTS_SUMMARY]
[--weights_save_path WEIGHTS_SAVE_PATH]
[--num_sanity_val_steps NUM_SANITY_VAL_STEPS]
[--truncated_bptt_steps TRUNCATED_BPTT_STEPS]
[--resume_from_checkpoint RESUME_FROM_CHECKPOINT]
[--profiler PROFILER] [--benchmark [BENCHMARK]]
[--deterministic [DETERMINISTIC]]
[--reload_dataloaders_every_n_epochs RELOAD_DATALOADERS_EVERY_N_EPOCHS]
[--reload_dataloaders_every_epoch [RELOAD_DATALOADERS_EVERY_EPOCH]]
[--auto_lr_find [AUTO_LR_FIND]]
[--replace_sampler_ddp [REPLACE_SAMPLER_DDP]]
[--terminate_on_nan [TERMINATE_ON_NAN]]
[--auto_scale_batch_size [AUTO_SCALE_BATCH_SIZE]]
[--prepare_data_per_node [PREPARE_DATA_PER_NODE]]
[--plugins PLUGINS] [--amp_backend AMP_BACKEND]
[--amp_level AMP_LEVEL]
[--distributed_backend DISTRIBUTED_BACKEND]
[--move_metrics_to_cpu [MOVE_METRICS_TO_CPU]]
[--multiple_trainloader_mode MULTIPLE_TRAINLOADER_MODE]
[--stochastic_weight_avg [STOCHASTIC_WEIGHT_AVG]]
optional arguments:
-h, --help show this help message and exit
Trainer Script:
--dataset_path DATASET_PATH
Path to dataset. Required!
--resume_from RESUME_FROM
Resume from checkpoint or transfer learn off
pretrained model. Leave blank to train from scratch.
AliasFreeGAN Model:
--size SIZE Pixel dimension of model. Must be 256, 512, or 1024.
Required!
--batch BATCH Batch size. Will be overridden if
--auto_scale_batch_size is used. (default: 16)
--lr_g LR_G Generator learning rate. (default: 0.002)
--lr_d LR_D Discriminator learning rate. (default: 0.002)
--r1 R1 R1 regularization weights. (default: 10.0)
--augment AUGMENT Use augmentations. (default: False)
--augment_p AUGMENT_P
Augment probability, the probability that augmentation
is applied. 0.0 is 0 percent and 1.0 is 100. If set to
0.0 and augment is enabled AdaptiveAugmentation will
be used. (default: 0.0)
--ada_target ADA_TARGET
Target for AdaptiveAugmentation. (default: 0.6)
--ada_length ADA_LENGTH
(default: 500000)
--ada_every ADA_EVERY
How often to update augmentation probabilities when
using AdaptiveAugmentation. (default: 8)
--stylegan2_discriminator STYLEGAN2_DISCRIMINATOR
Provide path to a rosinality stylegan2 checkpoint to
load the discriminator from it. Will load second so if
you load another model first it will override that
discriminator.
kimg Saver Callback:
--save_sample_every_kimgs SAVE_SAMPLE_EVERY_KIMGS
Sets the frequency of saving samples in kimgs
(thousands of image). (default: 1)
--save_checkpoint_every_kimgs SAVE_CHECKPOINT_EVERY_KIMGS
Sets the frequency of saving model checkpoints in
kimgs (thousands of image). (default: 4)
--start_kimg_count START_KIMG_COUNT
Manually override the start count for kimgs. If not
set the count will be inferred from checkpoint name.
If count can not be inferred it will default to 0.
--stop_training_at_kimgs STOP_TRAINING_AT_KIMGS
Automatically stop training at this number of kimgs.
(default: 12800)
--sample_grid SAMPLE_GRID
Sample grid to use for samples. Saved under
assets/sample_grids. (default:
default_5x3_sample_grid)
pl.Trainer:
--logger [LOGGER] Logger (or iterable collection of loggers) for
experiment tracking. A ``True`` value uses the default
``TensorBoardLogger``. ``False`` will disable logging.
If multiple loggers are provided and the `save_dir`
property of that logger is not set, local files
(checkpoints, profiler traces, etc.) are saved in
``default_root_dir`` rather than in the ``log_dir`` of
any of the individual loggers.
--checkpoint_callback [CHECKPOINT_CALLBACK]
If ``True``, enable checkpointing. It will configure a
default ModelCheckpoint callback if there is no user-
defined ModelCheckpoint in :paramref:`~pytorch_lightni
ng.trainer.trainer.Trainer.callbacks`.
--default_root_dir DEFAULT_ROOT_DIR
Default path for logs and weights when no
logger/ckpt_callback passed. Default: ``os.getcwd()``.
Can be remote file paths such as `s3://mybucket/path`
or 'hdfs://path/'
--gradient_clip_val GRADIENT_CLIP_VAL
0 means don't clip.
--gradient_clip_algorithm GRADIENT_CLIP_ALGORITHM
'value' means clip_by_value, 'norm' means
clip_by_norm. Default: 'norm'
--process_position PROCESS_POSITION
orders the progress bar when running multiple models
on same machine.
--num_nodes NUM_NODES
number of GPU nodes for distributed training.
--num_processes NUM_PROCESSES
number of processes for distributed training with
distributed_backend=\"ddp_cpu\"
--devices DEVICES Will be mapped to either `gpus`, `tpu_cores`,
`num_processes` or `ipus`, based on the accelerator
type.
--gpus GPUS number of gpus to train on (int) or which GPUs to
train on (list or str) applied per node
--auto_select_gpus [AUTO_SELECT_GPUS]
If enabled and `gpus` is an integer, pick available
gpus automatically. This is especially useful when
GPUs are configu...