Misc improvements (huggingface#6004)

* Misc improvements * More fixes * Check utils and setup.py style * Update utils/release.py Co-authored-by: Quentin Lhoest <[email protected]> --------- Co-authored-by: Quentin Lhoest <[email protected]>
SeekPoint · Jul 6, 2023 · 3e34d06 · 3e34d06
1 parent 535e972
commit 3e34d06
Show file tree

Hide file tree

Showing 6 changed files with 84 additions and 284 deletions.
diff --git a/Makefile b/Makefile
@@ -1,18 +1,18 @@
 .PHONY: quality style test
 
-check_dirs := tests src benchmarks metrics
+check_dirs := tests src benchmarks metrics utils
 
 # Check that source code meets quality standards
 
 quality:
-	black --check $(check_dirs)
-	ruff $(check_dirs)
+	black --check $(check_dirs) setup.py
+	ruff $(check_dirs) setup.py
 
 # Format source code automatically
 
 style:
-	black tests src benchmarks metrics
-	ruff $(check_dirs) --fix
+	black tests src benchmarks metrics setup.py
+	ruff $(check_dirs) setup.py --fix
 
 # Run tests for the library
 

diff --git a/README.md b/README.md
@@ -37,7 +37,7 @@
 
 [🎓 **Documentation**](https://huggingface.co/docs/datasets/) [🕹 **Colab tutorial**](https://colab.research.google.com/github/huggingface/datasets/blob/main/notebooks/Overview.ipynb)
 
-[🔎 **Find a dataset in the Hub**](https://huggingface.co/datasets) [🌟 **Add a new dataset to the Hub**](https://huggingface.co/docs/datasets/share.html)
+[🔎 **Find a dataset in the Hub**](https://huggingface.co/datasets) [🌟 **Share a dataset on the Hub**](https://huggingface.co/docs/datasets/share)
 
 <h3 align="center">
     <a href="https://hf.co/course"><img src="https://raw.githubusercontent.com/huggingface/datasets/main/docs/source/imgs/course_banner.png"></a>
@@ -155,9 +155,7 @@ If you are familiar with the great TensorFlow Datasets, here are the main differ
 
 # Disclaimers
 
-Similar to TensorFlow Datasets, 🤗 Datasets is a utility library that downloads and prepares public datasets. We do not host or distribute most of these datasets, vouch for their quality or fairness, or claim that you have license to use them. It is your responsibility to determine whether you have permission to use the dataset under the dataset's license.
-
-Moreover 🤗 Datasets may run Python code defined by the dataset authors to parse certain data formats or structures. For security reasons, we ask users to:
+🤗 Datasets may run Python code defined by the dataset authors to parse certain data formats or structures. For security reasons, we ask users to:
 - check the dataset scripts they're going to run beforehand and
 - pin the `revision` of the repositories they use.
 

diff --git a/convert_dataset.sh b/convert_dataset.sh
diff --git a/setup.py b/setup.py
@@ -253,7 +253,7 @@
     package_dir={"": "src"},
     packages=find_packages("src"),
     package_data={
-        "datasets": ["py.typed", "scripts/templates/*"],
+        "datasets": ["py.typed"],
         "datasets.utils.resources": ["*.json", "*.yaml", "*.tsv"],
     },
     entry_points={"console_scripts": ["datasets-cli=datasets.commands.datasets_cli:main"]},

diff --git a/tests/test_dataset_scripts.py b/tests/test_dataset_scripts.py