Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 1.0.2 #50

Merged
merged 124 commits into from
Aug 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
124 commits
Select commit Hold shift + click to select a range
8e3169d
Updated synthetic image mirror generation script, created helper func…
benliang99 Jul 31, 2024
f939cbb
Restored ensure_save_path func back to annotation_utils.py
benliang99 Jul 31, 2024
26f695e
Added latency tracking, max images generated field for synthetic imag…
benliang99 Jul 31, 2024
810211a
Added mean synthetic image gen latency print statement
benliang99 Jul 31, 2024
8aa47c8
Update example arg inputs
benliang99 Jul 31, 2024
37b1bdd
Fix imports
benliang99 Jul 31, 2024
1b9f657
Fixed and reformatted args
benliang99 Jul 31, 2024
8927abf
Suppressed TensorFlow warnings, fixed image gen from annotation
benliang99 Jul 31, 2024
48456ce
Index and loop bugfixes
benliang99 Jul 31, 2024
6fc7bba
Index, looping, args logic fixes.
benliang99 Jul 31, 2024
c7e4045
Add load diffuser function call
benliang99 Jul 31, 2024
23d467d
Clear gpu after using synthetic image generator
benliang99 Jul 31, 2024
249cab8
Always load from Hugging Face.
benliang99 Jul 31, 2024
1d0565e
Batch processing for memory optimization. Added optional name field t…
benliang99 Jul 31, 2024
39bc7d4
Memory optimizations (saving annotation .jsons to disk), added args f…
benliang99 Jul 31, 2024
8f25209
Fix save as json on disk, ensure no hanging reference when gpu is cle…
benliang99 Jul 31, 2024
8b09c07
Replaced generic DiffusionPipeline with StableDiffusionPipeline that …
benliang99 Jul 31, 2024
420513f
convert diffuser to float32 before moving onto cpu, fixed duplicate i…
aliang322 Aug 1, 2024
4f2d671
Merge branch 'synthetic_image_gen_updates' of https://github.com/bitm…
aliang322 Aug 1, 2024
9e7e829
Added a testing function to save images from real image dataset, chan…
benliang99 Aug 1, 2024
9255df4
Added pipeline for diffusion models to constants.py, and dynamic pipe…
benliang99 Aug 1, 2024
ebd3171
Fixed Hugging Face authentication errors. Added instruction to authen…
benliang99 Aug 1, 2024
3f29b83
Fixed all annotations being used to generate mirrors regardless of st…
benliang99 Aug 1, 2024
65d04f7
Added a new load_and_sort_dataset function to handle Hugging Face dat…
benliang99 Aug 2, 2024
1a435ef
Removed extra disable progress bar call. Added ceil import for progre…
benliang99 Aug 2, 2024
d986842
Adjust Hugging Face annotations dataset name
benliang99 Aug 3, 2024
b59e513
Reverted annotations dataset name to have data range, now requiring s…
benliang99 Aug 3, 2024
2d0851c
Re-removed data range from annotations
benliang99 Aug 3, 2024
6af5b61
Update 'index' to 'id'
benliang99 Aug 3, 2024
98da219
Fixed loading annotations from Hugging Face and savng specified indic…
benliang99 Aug 3, 2024
70ae0ed
Utils refactored, smaller functions. Added resize arg. Added combine_…
benliang99 Aug 4, 2024
e14fb04
Replace hardcoded name
benliang99 Aug 4, 2024
69440d6
Fix fstring
benliang99 Aug 4, 2024
8df88a0
Fix args
benliang99 Aug 4, 2024
cf530ef
Fixed typos
benliang99 Aug 4, 2024
6fe022a
Updated combine_datasets.py to match Hugging Face dataset nomenclature.
benliang99 Aug 5, 2024
a813a55
removing unused files
dylanuys Aug 8, 2024
8b01977
initial validator forward pytest
dylanuys Aug 8, 2024
7aacd76
initial ci.yml
dylanuys Aug 8, 2024
30cf41b
new mock classes for ci workflow
dylanuys Aug 8, 2024
7ffa028
temporarily removing old version of generate_synthetic_data.py
dylanuys Aug 8, 2024
176fe3c
rename get_mock_image() -> create_random_image()
dylanuys Aug 8, 2024
5736d47
adding test_mock.py
dylanuys Aug 8, 2024
12b75cf
renaming build -> test step in ci.yml
dylanuys Aug 8, 2024
4bc5b2a
test_rewards.py
dylanuys Aug 8, 2024
92a9ae8
parameterizing fake_prob to allow intentionally testing real/synth im…
dylanuys Aug 9, 2024
3a0d940
forcing vali fwd through real and synth image flows
dylanuys Aug 9, 2024
cd2763d
fake_prob -> _fake_prob
dylanuys Aug 9, 2024
4d2964f
using dot operator to read config in mock vali until I replace namesp…
dylanuys Aug 9, 2024
6e27356
allowing mock code to skip force_register_neuron in the case that the…
dylanuys Aug 9, 2024
3ce4ccb
removing unused circleci dir from template repo
dylanuys Aug 9, 2024
8e38e45
image transforms tests
dylanuys Aug 10, 2024
29fe632
fixing setting of mock dentrite process_time
dylanuys Aug 10, 2024
9b3aacf
adding test_mock.py
dylanuys Aug 10, 2024
22aec1d
reset mock chain state in between test cases
dylanuys Aug 10, 2024
9e50a92
cleaning up state management for MockSubtensor
dylanuys Aug 10, 2024
22ecfc9
__init__.py
dylanuys Aug 10, 2024
c40aad9
replacing hardcoded string with random image b64
dylanuys Aug 10, 2024
04d96a8
Fixed saving synthetic images after resizing.
benliang99 Aug 11, 2024
2a96f27
Merge pull request #43 from BitMind-AI/initial-pytest-test
dylanuys Aug 12, 2024
49aaa89
new auto update implementation from sn19
dylanuys Aug 15, 2024
9928fea
inital self heal script from sn19
dylanuys Aug 15, 2024
1244f3d
Flag for downloading annotations from HuggingFace
aliang322 Aug 15, 2024
07f31e5
fixing reference to self.config
dylanuys Aug 15, 2024
0ab534b
Enforcing no watermarking in all cases
aliang322 Aug 15, 2024
4b32e19
self heal in autoupdate script
dylanuys Aug 15, 2024
913a6d7
making autoupdate scripts executable
dylanuys Aug 15, 2024
2925e1f
self heal restart 6 -> 6.5
dylanuys Aug 15, 2024
b02a60f
Merge branch 'qol-improvements' of github.com:BitMind-AI/bitmind-subn…
dylanuys Aug 15, 2024
2e0cf8a
typo
dylanuys Aug 15, 2024
3d75478
Merge branch 'qol-improvements' of github.com:BitMind-AI/bitmind-subn…
dylanuys Aug 15, 2024
a98abec
allowing --no-auto-update and --no-self-heal for validators
dylanuys Aug 16, 2024
14dd4d8
combining run scripts into run_neuron.py
dylanuys Aug 16, 2024
6a373db
replacing neuron type with --validator and --miner
dylanuys Aug 16, 2024
3e2913a
documentation updates for new run script
dylanuys Aug 16, 2024
2616c59
docs update
dylanuys Aug 16, 2024
e721e15
adding wandb to docs
dylanuys Aug 16, 2024
2cd32bd
Arg for skipping annotation generation
aliang322 Aug 16, 2024
041d139
Prompt truncation for annotations longer than max token length
aliang322 Aug 16, 2024
ad78c36
Suppress token max exceeded warning, cleaned up error logging
aliang322 Aug 16, 2024
adc77af
Removed all tqdm loading bars, cleaned imports, updated fake dataset …
benliang99 Aug 16, 2024
cd343b8
Merge
benliang99 Aug 16, 2024
0f298cd
Improved annotation cleanliness with inter-prompt spacing and strippe…
aliang322 Aug 16, 2024
f813e15
Merge branch 'synthetic_image_gen_updates' of https://github.com/BitM…
aliang322 Aug 16, 2024
ef7df05
removing fixtures reference from mock.py
dylanuys Aug 16, 2024
56d37a8
read btcli args from .env
dylanuys Aug 16, 2024
4c914c3
docs update
dylanuys Aug 16, 2024
5a4e9e8
Formatting
dylanuys Aug 16, 2024
9f2e02f
fixing fixtures import
dylanuys Aug 16, 2024
4320780
adding .env file creation to install script
dylanuys Aug 16, 2024
2e4fed1
moving network (test/finney) into .env, reducing script redundancy
dylanuys Aug 16, 2024
7ebcf77
missing netuid arg for MockSubtensor/MockMetagraph inits in test
dylanuys Aug 16, 2024
2843489
adding .env to .gitignore
dylanuys Aug 16, 2024
dd918d8
AXON_PORT -> MINER_AXON_PORT env var rename
dylanuys Aug 16, 2024
c02b562
docs updates to reflect latest run_neuron.py updates
dylanuys Aug 16, 2024
5282cdd
updating .env paths
dylanuys Aug 16, 2024
05a7ae0
small docs update
dylanuys Aug 16, 2024
628f040
Fixed annotation json filenames not starting with start_idx arg
aliang322 Aug 16, 2024
0bc4e01
Merge pull request #44 from BitMind-AI/qol-improvements
dylanuys Aug 16, 2024
9d93ffa
locking down version numbers
dylanuys Aug 16, 2024
3c54e64
Added docstrings and comments
aliang322 Aug 16, 2024
7ca62eb
fixing image_index field for wanbd logging
dylanuys Aug 17, 2024
3161a97
try except for wandb init
dylanuys Aug 17, 2024
46c572e
adding retries for nan images
dylanuys Aug 17, 2024
3eb7ad6
fixing image isnan check by adding np.any
dylanuys Aug 18, 2024
6429e39
Merged synthetic image gen updates with testnet
benliang99 Aug 19, 2024
981bb16
rename wandb fields *image_id -> *image_name
dylanuys Aug 19, 2024
3dcf25d
Merge pull request #46 from BitMind-AI/requirements-updates
dylanuys Aug 19, 2024
edf05fa
Merge pull request #47 from BitMind-AI/vali-fwd-resampling
dylanuys Aug 19, 2024
3a0fedf
Updated failure case for generating annotations.
benliang99 Aug 19, 2024
55a659f
Adjusted TF logging level to include error messages. Cleaned up unnec…
benliang99 Aug 19, 2024
80c3f83
Reverted deletion of necessary diffusion pipeline imports. Adjusted T…
benliang99 Aug 19, 2024
c5ee3aa
Merge pull request #45 from BitMind-AI/synthetic_image_gen_updates
benliang99 Aug 19, 2024
e75c0b5
adding a sleep to reduce metagraph resync freq
dylanuys Aug 19, 2024
445e822
fixing edge case that occurs when only 1 miner has nonzero weight
dylanuys Aug 19, 2024
dde6556
Merge pull request #49 from BitMind-AI/fix-weight-setting-edgecase
dylanuys Aug 19, 2024
079e461
bump version to 1.0.2
dylanuys Aug 19, 2024
deca46a
fixing download_data extension
dylanuys Aug 19, 2024
ae68510
Merge pull request #51 from BitMind-AI/script-extension-fix
dylanuys Aug 19, 2024
27bf00b
Update fake dataset paths
benliang99 Aug 19, 2024
5d7669c
Merge pull request #52 from BitMind-AI/update-dataset-paths
dylanuys Aug 19, 2024
bb60523
replacing conda activate with /home/user/mambaforge/envs/tensorml
dylanuys Aug 19, 2024
e4b2f5b
Merge pull request #53 from BitMind-AI/autoupdate-conda-prefix
benliang99 Aug 19, 2024
828e59f
resolving merge conflicts
dylanuys Aug 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
168 changes: 0 additions & 168 deletions .circleci/config.yml

This file was deleted.

40 changes: 40 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: Continuous Integration

on:
push:
branches: [ "main", "testnet" ]
pull_request:
branches: [ "main", "testnet" ]

permissions:
contents: read

jobs:
test:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- name: Set up Python 3.10
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest pytest-asyncio
pip install -r requirements.txt
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
# run tests in tests/ dir and only fail if there are failures or errors
pytest tests/ --verbose --failed-first --exitfirst --disable-warnings
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -162,4 +162,5 @@ cython_debug/
testing/
data/
checkpoints/
.requirements_installed
.requirements_installed
*.env
9 changes: 9 additions & 0 deletions autoupdate_miner_steps.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/bash

# Thank you to Namoray of SN19 for their autoupdate implementation!
# THIS FILE CONTAINS THE STEPS NEEDED TO AUTOMATICALLY UPDATE THE REPO
# THIS FILE ITSELF MAY CHANGE FROM UPDATE TO UPDATE, SO WE CAN DYNAMICALLY FIX ANY ISSUES

echo $CONDA_PREFIX
$CONDA_PREFIX/bin/pip install -e .
echo "Autoupdate steps complete :)"
10 changes: 10 additions & 0 deletions autoupdate_validator_steps.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/bin/bash

# Thank you to Namoray of SN19 for their autoupdate implementation!
# THIS FILE CONTAINS THE STEPS NEEDED TO AUTOMATICALLY UPDATE THE REPO
# THIS FILE ITSELF MAY CHANGE FROM UPDATE TO UPDATE, SO WE CAN DYNAMICALLY FIX ANY ISSUES

echo $CONDA_PREFIX
$CONDA_PREFIX/bin/pip install -e .
$CONDA_PREFIX/bin/python bitmind/download_data.py
echo "Autoupdate steps complete :)"
2 changes: 1 addition & 1 deletion bitmind/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
# DEALINGS IN THE SOFTWARE.


__version__ = "1.0.1"
__version__ = "1.0.2"
version_split = __version__.split(".")
__spec_version__ = (
(1000 * int(version_split[0]))
Expand Down
Empty file removed bitmind/api/__init__.py
Empty file.
44 changes: 0 additions & 44 deletions bitmind/api/dummy.py

This file was deleted.

Loading
Loading