-
-
Notifications
You must be signed in to change notification settings - Fork 758
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add prepare_data
method to generate and save protocol data on disk
#1500
Add prepare_data
method to generate and save protocol data on disk
#1500
Commits on Oct 13, 2023
-
add
prepare_data
method inTask
classThe goal of this method is to generate the data needed by the task and save it on disk for future uses, for example by the `setup` method. The objective is to avoid systematically recreating data on each process at the beginning of a training
clement-pages committedOct 13, 2023 Configuration menu - View commit details
-
Copy full SHA for 933a660 - Browse repository at this point
Copy the full SHA 933a660View commit details
Commits on Oct 26, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 5257145 - Browse repository at this point
Copy the full SHA 5257145View commit details
Commits on Nov 2, 2023
-
modify organisation of
pyannote
segmentation tasksNow all the segmentations tasks in `pyannote` inherit the `SegmentationTask` (previously `SegmentationTaskMixin`), which inherits the `Task` class. This commit also adds a `prepared_data` attribute to the `Task` class. That attribute is a dict which contains all the prepared data by the `prepare_data` method.
clement-pages committedNov 2, 2023 Configuration menu - View commit details
-
Copy full SHA for 8829574 - Browse repository at this point
Copy the full SHA 8829574View commit details -
Merge branch 'feat/data_preparation' of github.com:clement-pages/pyan…
…note-audio into feat/data_preparation
clement-pages committedNov 2, 2023 Configuration menu - View commit details
-
Copy full SHA for fa63c8a - Browse repository at this point
Copy the full SHA fa63c8aView commit details
Commits on Nov 7, 2023
-
One for the test of the `MultiLabelSegmentation` task, and the other for the test of the `SupervisedRepresentationLearningWithArcFace` task.
clement-pages committedNov 7, 2023 Configuration menu - View commit details
-
Copy full SHA for be6f7ec - Browse repository at this point
Copy the full SHA be6f7ecView commit details -
assign data directly to task in main process, in
prepare_data
This eliminates the need to reload pickle data in setup when in the main process
clement-pages committedNov 7, 2023 Configuration menu - View commit details
-
Copy full SHA for f447bb6 - Browse repository at this point
Copy the full SHA f447bb6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 930deda - Browse repository at this point
Copy the full SHA 930dedaView commit details
Commits on Nov 8, 2023
-
handle call to
Task.prepare_data
andTask.setup
under different s……cenarios
clement-pages committedNov 8, 2023 Configuration menu - View commit details
-
Copy full SHA for 05ccc30 - Browse repository at this point
Copy the full SHA 05ccc30View commit details -
Merge branch 'feat/data_preparation' of github.com:clement-pages/pyan…
…note-audio into feat/data_preparation
clement-pages committedNov 8, 2023 Configuration menu - View commit details
-
Copy full SHA for 44a01fe - Browse repository at this point
Copy the full SHA 44a01feView commit details
Commits on Nov 9, 2023
-
add training tests using task caches
clement-pages committedNov 9, 2023 Configuration menu - View commit details
-
Copy full SHA for 4b8e8a2 - Browse repository at this point
Copy the full SHA 4b8e8a2View commit details -
update
cache_path
type and docstringsclement-pages committedNov 9, 2023 Configuration menu - View commit details
-
Copy full SHA for 45918bd - Browse repository at this point
Copy the full SHA 45918bdView commit details -
fix
classes
variable used before assigmentThis issue occured when a list of classes was specified during `MultiLabelSegmentation` instanciation.
clement-pages committedNov 9, 2023 Configuration menu - View commit details
-
Copy full SHA for 980414e - Browse repository at this point
Copy the full SHA 980414eView commit details
Commits on Nov 14, 2023
-
Configuration menu - View commit details
-
Copy full SHA for a9ea07f - Browse repository at this point
Copy the full SHA a9ea07fView commit details
Commits on Nov 20, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 797a8a4 - Browse repository at this point
Copy the full SHA 797a8a4View commit details
Commits on Nov 21, 2023
-
clement-pages committed
Nov 21, 2023 Configuration menu - View commit details
-
Copy full SHA for 987e702 - Browse repository at this point
Copy the full SHA 987e702View commit details
Commits on Nov 27, 2023
-
improve: use
numpy
method for w/r task cache insteadpickle
(#1)* use npz archive instead pickle to save task data * improve code readability * improve(task): update numpy array dtypes In order to use types whose size better machtes the contents of the arrays * remove `end` entry from `annotated_regions` numpy array This entry was redundant with the start and duration entries, since `end` = `start` + `duration`. * fix: allow data preparation to be finished when task has no validation * improve: clear data lists after assignation to `self.prepared_data` This is to avoid data redundancy in the `prepare_data` method --------- Co-authored-by: clement-pages <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 042dc43 - Browse repository at this point
Copy the full SHA 042dc43View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5358986 - Browse repository at this point
Copy the full SHA 5358986View commit details