Setup_audio #322

monke6942021 · 2023-11-08T17:28:40Z

Croissant can now recognize sc:AudioObject types, but can't pull any relevant data.

github-actions · 2023-11-08T17:28:58Z

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

marcenacp

Thanks! Do you have a dataset to test this?

marcenacp · 2023-11-09T13:38:51Z

requirements.txt

@@ -0,0 +1,27 @@
+absl-py==1.4.0


We don't use requirements.txt, but pyproject.toml to manage dependencies.

marcenacp · 2023-11-09T13:39:26Z

python/mlcroissant/mlcroissant/_src/core/data_types.py

@@ -28,6 +28,9 @@ def check_expected_type(issues: Issues, jsonld: Json, expected_type: str):
    constants.SCHEMA_ORG_DATA_TYPE_IMAGE_OBJECT: (
        constants.SCHEMA_ORG_DATA_TYPE_IMAGE_OBJECT
    ),
+    constants.SCHEMA_ORG_DATA_TYPE_AUDIO_OBJECT: (


I believe this has changed. Can you rebase to have a freshest version of the code?

git pull --rebase origin main

monke6942021 · 2023-11-16T11:18:24Z

Thanks! Do you have a dataset to test this?

I just added the one that I've been using for testing. It's a mini-version of what I finally want to do with croissant, but maybe the 0-hz-immersion-into-wallbient directory can be used as another test if we want it to operate without looking into multiple directories.

marcenacp

Thanks for the dataset. My remarks:

The dataset should probably be added to the non-hermetic unit tests. Like this, we'll make sure we won't have any regression on audio.
Is it possible to not commit the *.mp3 and rather host them somewhere? Otherwise, it could make the repo heavier.
Could you address the previous reviews? (E.g., rebase your code and do not commit the requirements.txt file).

Thanks!

monke6942021 · 2023-11-16T18:46:49Z

Hi! I added the dataset to the non-hermetic tests, and I'm also hosting the dataset from a Google Drive, which I shared with you and Tim. Also, sorry about the problem with the rebase. I thought I rebased it when I added the dataset, but I did not complete the process. I have now rebased the branch and synced the changes. I hope it works properly now. I also removed the requirements.txt file.

Also, would it be possible to get some Documentation on how different datatypes get pulled. I ask because one thing I've noticed is that ImageObjects get pulled in a different manner than something like a text file. I've been trial and erroring my way through the process, but I could use some clarification on how different datatypes get pulled.

Thanks!

…t has been commented out for now.

…is helps.

monke6942021 · 2024-01-11T06:35:24Z

I passed the tests. Is there anything else I should do?

marcenacp · 2024-01-15T08:30:59Z

python/mlcroissant/mlcroissant/_src/datasets_test.py

+            "audio_test/metadata.json",
+            "records",
+            -1,
+        ],  # Switch the number to 10 if nessacary


Remove the comment?

I just removed it.

marcenacp · 2024-01-15T08:31:43Z

python/mlcroissant/mlcroissant/_src/core/optional.py

@@ -86,5 +86,10 @@ def PIL_Image(cls) -> types.ModuleType:  # pylint: disable=invalid-name
        """Cached git module."""
        return _try_import("PIL.Image", package_name="Pillow")

+    @cached_class_property
+    def LIB_Audio(cls) -> types.ModuleType:  # pylint: disable=invalid-name
+        """Cached git module."""


Cached librosa module

marcenacp · 2024-01-15T08:39:17Z

python/mlcroissant/mlcroissant/_src/operation_graph/operations/field.py

@@ -31,6 +31,9 @@ def _cast_value(value: Any, data_type: type | term.URIRef | None):
            return deps.PIL_Image.open(io.BytesIO(value))
        else:
            raise ValueError(f"Type {type(value)} is not accepted for an image.")
+    elif data_type == DataType.AUDIO_OBJECT:
+        output = deps.LIB_Audio.load(io.BytesIO(value))
+        return str([output[0].tolist(), output[1]])


Why a str? How will you use this in an ML pipeline? What would be the most useful signal to return here?

I did it like that because the output needs to have double quotes. I'm about to push a version that just outputs the regular librosa output, but I don't know if it will pass the test.

marcenacp · 2024-01-15T08:42:09Z

python/mlcroissant/pyproject.toml

@@ -18,6 +19,7 @@ authors = [
 # pip dependencies of the project
 # Installed locally with `pip install -e .`
 dependencies = [
+  "black[jupyter]",


Remove jupyter?

marcenacp · 2024-01-25T08:37:32Z

python/mlcroissant/mlcroissant/_src/core/optional.py

@@ -86,5 +86,10 @@ def PIL_Image(cls) -> types.ModuleType:  # pylint: disable=invalid-name
        """Cached git module."""
        return _try_import("PIL.Image", package_name="Pillow")

+    @cached_class_property
+    def LIB_Audio(cls) -> types.ModuleType:  # pylint: disable=invalid-name


def librosa (PIL.Image above is a library, so the name here should probably be librosa)

I just changed the name, and the usage in field.py

…tup_audio

…tly on my end.

monke6942021 added 4 commits September 14, 2023 22:50

This should make the validation process easier.

ade26eb

Validation will be easier now.

5faaf02

Merge branch 'main' of https://github.com/mlcommons/croissant

b84be69

This will let croissant recognize AudioObjects

82b0217

monke6942021 requested a review from a team as a code owner November 8, 2023 17:28

marcenacp requested changes Nov 9, 2023

View reviewed changes

mini_CC_BY_SA dataset has been added

506bd0d

marcenacp requested changes Nov 16, 2023

View reviewed changes

monke6942021 added 4 commits November 16, 2023 17:45

This should make the validation process easier.

9861395

mini_CC_BY_SA dataset has been added

50d78ac

mini_CC_BY_SA is now part of the non-hermetic tests

369621d

Now it takes input from a Google Drive

5723b3f

monke6942021 added 15 commits November 20, 2023 05:17

Many of the tests were failing because of librosa, so anything with i…

4b7150e

…t has been commented out for now.

Hopefully this passes the pr test

fafe57d

The validation script CLAIMS that it works.

30d2aca

Hopefully black[jupyter] helps

5f23744

Now jupyter has been added before the original black. Let's see if th…

56a86e8

…is helps.

I can try resetting to see if the formatting test works.

babbbc2

Changed treatment of AudioObject in field.py

8cd0e89

This version can load audio in field.py

7d22310

Merge branch 'main' into setup_audio

363bac6

Audio files have some level of usability now.

4e5f54e

The exact dataset I worked with

081c129

I got the audio support to work.

5a88008

This should be a good touch up to help with jpeg files too.

154ef14

This is a cheap shot option, but it might pass the unit tests.

28116d8

Removed URL because it's not being used.

f42626f

monke6942021 added 3 commits January 10, 2024 23:44

Hope this works.

ab23572

updated dev dependencies.

cf043c8

I used black in a different way, so it might work this time.

fd5393a

monke6942021 self-assigned this Jan 11, 2024

monke6942021 requested a review from marcenacp January 11, 2024 06:40

marcenacp reviewed Jan 15, 2024

View reviewed changes

marcenacp mentioned this pull request Jan 15, 2024

Monjishb #222

Closed

monke6942021 added 2 commits January 16, 2024 08:03

addressing the issues that Pierre bought up.

4c40d91

Since I used load.py for this one, I'm confident that this should work.

230d1fd

monke6942021 requested a review from marcenacp January 16, 2024 08:22

marcenacp reviewed Jan 25, 2024

View reviewed changes

marcenacp approved these changes Jan 25, 2024

View reviewed changes

monke6942021 added 14 commits January 30, 2024 03:35

Changed the name of the library

ba72b11

removed the commment

0177150

dealt with conflicts.

9f2d083

Hopefully I don't need git LFS for this.

c5d38e2

black reformatted optional.py

b587614

For some reason, my unit test is supposed to be in 0.8

1834c49

Removed audio_test from 0.8.

7611a5a

Added fileObject and fileSet in context.

c66524b

I will remove this from 0.8 later, but I need to prove a point.

a777c7f

Adding audio_test to 0.8 did nothing of value.

8b9b511

Merge branch 'main' of https://github.com/mlcommons/croissant into se…

b34ede7

…tup_audio

Changed audio_test to be compatible with new changes.

c0ce931

This should pass the tests, but validation still works a bit differen…

63603e4

…tly on my end.

They needed a version of the audio dataset in 0.8

d212703

monke6942021 merged commit 4f2e86f into main Feb 8, 2024
14 checks passed

github-actions bot locked and limited conversation to collaborators Feb 8, 2024

monke6942021 deleted the setup_audio branch February 8, 2024 08:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Setup_audio #322

Setup_audio #322

monke6942021 commented Nov 8, 2023

github-actions bot commented Nov 8, 2023 •

edited

Loading

marcenacp left a comment

marcenacp Nov 9, 2023

marcenacp Nov 9, 2023

monke6942021 commented Nov 16, 2023

marcenacp left a comment •

edited

Loading

monke6942021 commented Nov 16, 2023 •

edited

Loading

monke6942021 commented Jan 11, 2024

marcenacp Jan 15, 2024

marcenacp Jan 25, 2024

monke6942021 Jan 30, 2024

marcenacp Jan 15, 2024

marcenacp Jan 15, 2024

monke6942021 Jan 16, 2024

marcenacp Jan 15, 2024

marcenacp Jan 25, 2024

monke6942021 Jan 30, 2024

Setup_audio #322

Setup_audio #322

Conversation

monke6942021 commented Nov 8, 2023

github-actions bot commented Nov 8, 2023 • edited Loading

marcenacp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

monke6942021 commented Nov 16, 2023

marcenacp left a comment • edited Loading

Choose a reason for hiding this comment

monke6942021 commented Nov 16, 2023 • edited Loading

monke6942021 commented Jan 11, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Nov 8, 2023 •

edited

Loading

marcenacp left a comment •

edited

Loading

monke6942021 commented Nov 16, 2023 •

edited

Loading