Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Updated synthetic image mirror generation script, created helper function for generating images iin SyntheticImageGenerator class, moved notebooks to new notebooks dir. * Restored ensure_save_path func back to annotation_utils.py * Added latency tracking, max images generated field for synthetic image mirror pipeline. * Added mean synthetic image gen latency print statement * Update example arg inputs * Fix imports * Fixed and reformatted args * Suppressed TensorFlow warnings, fixed image gen from annotation * Index and loop bugfixes * Index, looping, args logic fixes. * Add load diffuser function call * Clear gpu after using synthetic image generator * Always load from Hugging Face. * Batch processing for memory optimization. Added optional name field to generate_image in SyntheticImageGenerato for future customization. * Memory optimizations (saving annotation .jsons to disk), added args for chunking, pm2 examples * Fix save as json on disk, ensure no hanging reference when gpu is cleared in SyntheticImageGenerator * Replaced generic DiffusionPipeline with StableDiffusionPipeline that inherits from it. Specified generated image dimensions in diffuser call params. * convert diffuser to float32 before moving onto cpu, fixed duplicate image count logs * Added a testing function to save images from real image dataset, changed annotations 'index' field to 'id' for consistency, various data loading and parameter fixes * Added pipeline for diffusion models to constants.py, and dynamic pipeline loading and image size customization to generation. * Fixed Hugging Face authentication errors. Added instruction to authenticate with huggingface-cli login * Fixed all annotations being used to generate mirrors regardless of start and end indices * Added a new load_and_sort_dataset function to handle Hugging Face dataset rows being ordered by filename string-wise instead of numerically. Added generate_synthetic_images arg and updated dataset naming conventions for parallelization-friendliness. Disabled diffusion pipeline progress bars. Added const for progress updates in terminal. * Removed extra disable progress bar call. Added ceil import for progress calculation. * Adjust Hugging Face annotations dataset name * Reverted annotations dataset name to have data range, now requiring start_index and end_index args. * Re-removed data range from annotations * Update 'index' to 'id' * Fixed loading annotations from Hugging Face and savng specified indices to disk. * Utils refactored, smaller functions. Added resize arg. Added combine_datasets script to put together all generated splits into one Hugging Face dataset. * Replace hardcoded name * Fix fstring * Fix args * Fixed typos * Updated combine_datasets.py to match Hugging Face dataset nomenclature. * removing unused files * initial validator forward pytest * initial ci.yml * new mock classes for ci workflow * temporarily removing old version of generate_synthetic_data.py * rename get_mock_image() -> create_random_image() * adding test_mock.py * renaming build -> test step in ci.yml * test_rewards.py * parameterizing fake_prob to allow intentionally testing real/synth image flows in vali fwd * forcing vali fwd through real and synth image flows * fake_prob -> _fake_prob * using dot operator to read config in mock vali until I replace namespace cfg with bt.config * allowing mock code to skip force_register_neuron in the case that the neuron was already registered in previous test instance * removing unused circleci dir from template repo * image transforms tests * fixing setting of mock dentrite process_time * adding test_mock.py * reset mock chain state in between test cases * cleaning up state management for MockSubtensor * __init__.py * replacing hardcoded string with random image b64 * Fixed saving synthetic images after resizing. * new auto update implementation from sn19 * inital self heal script from sn19 * Flag for downloading annotations from HuggingFace * fixing reference to self.config * Enforcing no watermarking in all cases * self heal in autoupdate script * making autoupdate scripts executable * self heal restart 6 -> 6.5 * typo * allowing --no-auto-update and --no-self-heal for validators * combining run scripts into run_neuron.py * replacing neuron type with --validator and --miner * documentation updates for new run script * docs update * adding wandb to docs * Arg for skipping annotation generation * Prompt truncation for annotations longer than max token length * Suppress token max exceeded warning, cleaned up error logging * Removed all tqdm loading bars, cleaned imports, updated fake dataset paths to parquet versions. * Improved annotation cleanliness with inter-prompt spacing and stripped endings. * removing fixtures reference from mock.py * read btcli args from .env * docs update * Formatting * fixing fixtures import * adding .env file creation to install script * moving network (test/finney) into .env, reducing script redundancy * missing netuid arg for MockSubtensor/MockMetagraph inits in test * adding .env to .gitignore * AXON_PORT -> MINER_AXON_PORT env var rename * docs updates to reflect latest run_neuron.py updates * updating .env paths * small docs update * Fixed annotation json filenames not starting with start_idx arg * locking down version numbers * Added docstrings and comments * fixing image_index field for wanbd logging * try except for wandb init * adding retries for nan images * fixing image isnan check by adding np.any * rename wandb fields *image_id -> *image_name * Updated failure case for generating annotations. * Adjusted TF logging level to include error messages. Cleaned up unnecessary imports. Simplified clear_gpu to not moving tensor to CPU. * Reverted deletion of necessary diffusion pipeline imports. Adjusted TF logging level in dataset generation script to be consistent with synthetic generation classes. * adding a sleep to reduce metagraph resync freq * fixing edge case that occurs when only 1 miner has nonzero weight * bump version to 1.0.2 * fixing download_data extension * Update fake dataset paths * replacing conda activate with /home/user/mambaforge/envs/tensorml * Base miner training improvements and Content-Aware Model Orchestration (CAMO) Framework (#55) * Added DeepfakeBench submodule to base_miner dir * Added initial adaptation of pretrained UCF inference. Refactored NPR files into new dir. * Added setup readme and a sample image for inference. * Added loss functions and backbone network. Updated readme. * Enable loading model checkpoints from Hugging Face. * migrated training scripts from DeepfakeBench * Added package initialization, renaming configs to config. * Added fix to missing weights directory * Finished ucf_test on sample images. * Added dlib shape detector for face detection and alignment * Added face_recognition implementation of face alignment * Fixed variable names * Update dlib requirements * Implemented ucf_miner and created a class for the pretrained UCF model * Renamed files for clarity. Added unit test for pretrained UCF. * Migrated train utils from NPR base miner, modified train_ucf.py to use BitMind datasets * Fix image input type errors * Added xception training backbone, logging files * Detectors module path fix * bug fixes for live miner * BitMind data load and restructure for integrated DeepfakeBench train loop * Added DeepfakeBench training logs to .gitignore * Removed unused import, local data saving * Fixed prediction_class referenced before assignment * Fixed test metric using logits and not class labels * Train source labels for learning specific forgery, added separate test and validation loops * Corrected variable name typo * Refactored eval in training loop, renamed test stage to validation * Implemented source label mapping in UCF training splits * Added test stage, source labels to training data dict for learning dataset specific forgery features * Added gpu cache cleanup, now using configs for batch size and data loader workers. * Batch to cpu after train loader iteration, logging cleanup * Fixed test metrics not logging * Added logging for train and test time * Added image normalization for training data * Re-added check for data label during inference. * Adjusted UCF image normalization to be in line with config. Fixed processing of local images for UCF testing. * Adjusted image preprocessing for experiments. * Added face cropping and alignment to preprocess images for UCF detection. * Typo fixes, added readme to credit face shape predictor file. * Made face crop and align False by default. * New miner script for running UCF-BitMind * Added handling for the case when face_detector does not find any images. Reduced warning messages. * Removed duplicate function. * Adapted face detection and extraction functions to UCF class for modularity. Updated and refactored test, miner scripts. * First iteration of context_aware_miner.py * Fixed ucf_miner import error by simplifying path and import statement for UCF module. * Fixed dlib predictor path, explicitly define map_location for torch.load * Fixed imports in ucf_bitmind_miner and removed rounding or predictions. * Fixed imports for context aware miner. * Fix NPR model weight variable name. * Typo * Added Context-Aware v2 with UCF-BitMind for general images * Remove unused import * Moved UCF model loading to init function of Context Aware Miner v2 * Added free memory function to manage resources for multi-model miners. * Added script for miners to test their model loading and inference latency. * Moved model loading to init functions to avoid reloading. * Updated minimum miner requirements to require GPU * Fixed indents, load UCF-DFB model in init func * Update check for faces to be consistent with DFB preprocessing * Release 1.0.2 (#50) * Updated synthetic image mirror generation script, created helper function for generating images iin SyntheticImageGenerator class, moved notebooks to new notebooks dir. * Restored ensure_save_path func back to annotation_utils.py * Added latency tracking, max images generated field for synthetic image mirror pipeline. * Added mean synthetic image gen latency print statement * Update example arg inputs * Fix imports * Fixed and reformatted args * Suppressed TensorFlow warnings, fixed image gen from annotation * Index and loop bugfixes * Index, looping, args logic fixes. * Add load diffuser function call * Clear gpu after using synthetic image generator * Always load from Hugging Face. * Batch processing for memory optimization. Added optional name field to generate_image in SyntheticImageGenerato for future customization. * Memory optimizations (saving annotation .jsons to disk), added args for chunking, pm2 examples * Fix save as json on disk, ensure no hanging reference when gpu is cleared in SyntheticImageGenerator * Replaced generic DiffusionPipeline with StableDiffusionPipeline that inherits from it. Specified generated image dimensions in diffuser call params. * convert diffuser to float32 before moving onto cpu, fixed duplicate image count logs * Added a testing function to save images from real image dataset, changed annotations 'index' field to 'id' for consistency, various data loading and parameter fixes * Added pipeline for diffusion models to constants.py, and dynamic pipeline loading and image size customization to generation. * Fixed Hugging Face authentication errors. Added instruction to authenticate with huggingface-cli login * Fixed all annotations being used to generate mirrors regardless of start and end indices * Added a new load_and_sort_dataset function to handle Hugging Face dataset rows being ordered by filename string-wise instead of numerically. Added generate_synthetic_images arg and updated dataset naming conventions for parallelization-friendliness. Disabled diffusion pipeline progress bars. Added const for progress updates in terminal. * Removed extra disable progress bar call. Added ceil import for progress calculation. * Adjust Hugging Face annotations dataset name * Reverted annotations dataset name to have data range, now requiring start_index and end_index args. * Re-removed data range from annotations * Update 'index' to 'id' * Fixed loading annotations from Hugging Face and savng specified indices to disk. * Utils refactored, smaller functions. Added resize arg. Added combine_datasets script to put together all generated splits into one Hugging Face dataset. * Replace hardcoded name * Fix fstring * Fix args * Fixed typos * Updated combine_datasets.py to match Hugging Face dataset nomenclature. * removing unused files * initial validator forward pytest * initial ci.yml * new mock classes for ci workflow * temporarily removing old version of generate_synthetic_data.py * rename get_mock_image() -> create_random_image() * adding test_mock.py * renaming build -> test step in ci.yml * test_rewards.py * parameterizing fake_prob to allow intentionally testing real/synth image flows in vali fwd * forcing vali fwd through real and synth image flows * fake_prob -> _fake_prob * using dot operator to read config in mock vali until I replace namespace cfg with bt.config * allowing mock code to skip force_register_neuron in the case that the neuron was already registered in previous test instance * removing unused circleci dir from template repo * image transforms tests * fixing setting of mock dentrite process_time * adding test_mock.py * reset mock chain state in between test cases * cleaning up state management for MockSubtensor * __init__.py * replacing hardcoded string with random image b64 * Fixed saving synthetic images after resizing. * new auto update implementation from sn19 * inital self heal script from sn19 * Flag for downloading annotations from HuggingFace * fixing reference to self.config * Enforcing no watermarking in all cases * self heal in autoupdate script * making autoupdate scripts executable * self heal restart 6 -> 6.5 * typo * allowing --no-auto-update and --no-self-heal for validators * combining run scripts into run_neuron.py * replacing neuron type with --validator and --miner * documentation updates for new run script * docs update * adding wandb to docs * Arg for skipping annotation generation * Prompt truncation for annotations longer than max token length * Suppress token max exceeded warning, cleaned up error logging * Removed all tqdm loading bars, cleaned imports, updated fake dataset paths to parquet versions. * Improved annotation cleanliness with inter-prompt spacing and stripped endings. * removing fixtures reference from mock.py * read btcli args from .env * docs update * Formatting * fixing fixtures import * adding .env file creation to install script * moving network (test/finney) into .env, reducing script redundancy * missing netuid arg for MockSubtensor/MockMetagraph inits in test * adding .env to .gitignore * AXON_PORT -> MINER_AXON_PORT env var rename * docs updates to reflect latest run_neuron.py updates * updating .env paths * small docs update * Fixed annotation json filenames not starting with start_idx arg * locking down version numbers * Added docstrings and comments * fixing image_index field for wanbd logging * try except for wandb init * adding retries for nan images * fixing image isnan check by adding np.any * rename wandb fields *image_id -> *image_name * Updated failure case for generating annotations. * Adjusted TF logging level to include error messages. Cleaned up unnecessary imports. Simplified clear_gpu to not moving tensor to CPU. * Reverted deletion of necessary diffusion pipeline imports. Adjusted TF logging level in dataset generation script to be consistent with synthetic generation classes. * adding a sleep to reduce metagraph resync freq * fixing edge case that occurs when only 1 miner has nonzero weight * bump version to 1.0.2 * fixing download_data extension * Update fake dataset paths * replacing conda activate with /home/user/mambaforge/envs/tensorml --------- Co-authored-by: Benjamin <[email protected]> Co-authored-by: aliang322 <[email protected]> * TrainingDatasetProcessor class for loading, generating, and uploading preprocessing face only images into training datasets * Fixed transform dict var name * Removed normalize function in training dataset creation * Changed config dict to faces_only bool for clarity, changed hf repo type to dataset * Added splits instance variable, generalized function names, non face only processing * Script for interfacing with TrainingDatasetProcessor to create and upload preprocessed training datasets. * Added usage examples and explanation * Removed unused import * Simplified repo upload naming convention. * Added original image index column to training datasets. Consolidated transform datasets into HF subsets. * Added local save/load, upload repo destination options; Fixed dataset preprocessing to be performed in-place. * Created create_splits() helper function * Fixed not clearing dataset memory when loading pickle * Created Context-Aware Hierarchical Mixture-of-Agents (CAMO) miner. * Created modular helper function for loading detectors. * Rewording comments * Added YOLOv8 object detection for image classification * Renamed create_splits() to split_dataset() and added support for subset loading * Added HuggingFace subset download option * Restructured data loading for training. Support for face only subset loading with stratified splits. * Added YOLOv8 object detection experiments, renamed CAMO miner. * Updated paths for new CAMO model weights * Added object detection error checks * Switch debug to bt.logging, formatting * Set use_object_detection to False for current iteration of CAMO * Fixed assertion error on last batch of train epoch when incomplete batches present by dropping last in dataloaders. * Added data shuffling prior to splitting, check for disjoint stratified split indices. * Fixed faces not being used by face expert. * Improved clarity with face processing helper functions * Generalized to optionally include source label mapping * Consolidated real_fake_dataset.py into bitmind directory, updated references * Fixed docstring typo * Parameterized shuffling before splits, generalized params to adopt expert model terminology, made source label mapping optional * Renamed params appropriately, fixed generalist UCF not using source labels * Reformatted long lines, fixed split size printing num batches * Removed UCF-specific data utils, replaced with generalized utils at bitmind level * Standardized usage of bitmind.utils.train_data for data load/split across NPR and UCF base models * adding versions for new package * Update Mining.md * moving miner/validator specific dependencies to new requirements filse * setup scripts specific to miner/validator reqs * Relocated train/predict data processing scripts, updated imports and paths * Removed redundant face detection utils from UCF dir * Comment crediting original source of UCF training scripts * Removed DeepfakeBench submodule * Added auto download for backbone weights if not locally present * Removed redundant video metrics from validation logs * Updated README with script usage, removed deprecated manual weight download instructions * Cleaned up leftover DFB train configs, fixed training error when using only 1 real/fake train dataset * Deleted whitespace * Cleaned unused DFB config labels * adding subtensor.chain_endpoint to startup scripts * Added default value for specific task number for training * Standardized UCF paths with consts across UCF neurons and training files * Cleaned up experimental files. * Update Mining.md * Update Validating.md * Standardized neuron naming * Updated default miner to camo_miner.py * Fixed forgery dataset/method disentangling by setting specific_task_number value to num of fake datasets + 1 for real label * Added readme file for camo base miner * Fixed UCF constant weight path names * Removed non-validator dataset generation scripts, fixed camo readme. * Weights constant name typo * Fixed UCF miner imports * Added missing UCF weights import * Cleaned up utils and unnecessary files * adding testnet chain endpoint to docs * adding miner/vali dep installs to ci.yml --------- Co-authored-by: Benjamin <[email protected]> Co-authored-by: default <[email protected]> Co-authored-by: aliang322 <[email protected]> Co-authored-by: Ken Miyachi <[email protected]> Co-authored-by: Dylan Uys <[email protected]> * bump version 1.0.2 -> 1.1.0 * Removed sample images in UCF directory * parameterizing neuron filepath * Added base miner dir readme with CAMO information --------- Co-authored-by: Benjamin <[email protected]> Co-authored-by: aliang322 <[email protected]> Co-authored-by: default <[email protected]> Co-authored-by: aliang322 <[email protected]> Co-authored-by: Ken Miyachi <[email protected]>
- Loading branch information
6 people
committed
Sep 5, 2024
1 parent
6cc7811
commit 21d65f3
