Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fiboa Cli Converter for Brazil #84

Merged
merged 6 commits into from
Jul 30, 2024
Merged

Fiboa Cli Converter for Brazil #84

merged 6 commits into from
Jul 30, 2024

Conversation

Barasakar
Copy link
Contributor

This is the pull request for the Brazil dataset (#49 and an issue in fiboa/data).

The name of the converter is fs_br.py. I was planning to name it br.py changed my mind after seeing one of the comments in #49 .

I added some comments about some of my decisions in the converter file. Let me know what you think. Thank you!

@Barasakar
Copy link
Contributor Author

Barasakar commented Jul 25, 2024

Sorry, I forgot to create a new branch off main. I will be mindful about that next time.
I commented out some of the code because I wasn't sure if I was doing it correctly or if that's what we want. They are mostly for the COLUMNS variable and MISSING_SCHEMAS variable

fiboa_cli/datasets/fs_br.py Outdated Show resolved Hide resolved
fiboa_cli/datasets/fs_br.py Outdated Show resolved Hide resolved
fiboa_cli/datasets/fs_br.py Outdated Show resolved Hide resolved
fiboa_cli/datasets/fs_br.py Outdated Show resolved Hide resolved
fiboa_cli/datasets/fs_br.py Outdated Show resolved Hide resolved
Copy link
Contributor

@m-mohr m-mohr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I made a couple of comments to potentially improve the converter. Let me know what you think :-)

Copy link
Contributor

@m-mohr m-mohr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The converter looks good now, I've added a changelog entry, too. It would be good to have tests, but that's optional.
Thank you!

@Barasakar
Copy link
Contributor Author

The converter looks good now, I've added a changelog entry, too. It would be good to have tests, but that's optional. Thank you!

Awesome. Thank you both for the help!

@Barasakar
Copy link
Contributor Author

The converter looks good now, I've added a changelog entry, too. It would be good to have tests, but that's optional. Thank you!

Could you elaborate what you meant by tests? Did you mean running the command fiboa validate xxxx.parquet --data? I am also happy to learn how to create tests if needed

@m-mohr
Copy link
Contributor

m-mohr commented Jul 26, 2024

Oh, yeah, sorry. I think we don't have the tests well documented.

There's a folder named tests, which has a file test_convert.py.
There you can implement tests for your new converter using pytest.
In your case you probably need to extend the second test, test_converter_with_input, as you have a zip input.
You also need to create a subset of the data and place it in tests/data-files/convert/br_ba_lem.
Afterwards the tests can be run through pytest and succeed.
But this is just a high-level overview. If you are not so used to pytest, it might be a little too complex to explain via a GitHub comment.
We can also merge it as is for now, or I could add the test... however you prefer.

@Barasakar
Copy link
Contributor Author

I see. I could look into the files you mentioned. But for now I guess we could just merge it! Thanks.

@m-mohr m-mohr merged commit e3b88b3 into fiboa:main Jul 30, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants