Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data formats tutorial #3

Draft
wants to merge 31 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
277e228
doc: add readme
EugenePlanteur Nov 12, 2024
f0c4714
feat: add utils functions
EugenePlanteur Nov 12, 2024
c6f9c08
feat: add raster formats notebook
EugenePlanteur Nov 12, 2024
9fdb02a
feat: add vector formats notebook
EugenePlanteur Nov 12, 2024
2236b7b
feat: add vector formats notebook
EugenePlanteur Nov 12, 2024
1d886c3
install: add requirements.txt
EugenePlanteur Nov 12, 2024
73bdbf1
doc: update readme
EugenePlanteur Nov 12, 2024
2c5c740
feat: add datacube notebook
EugenePlanteur Nov 19, 2024
fc37617
style: fix typo
EugenePlanteur Nov 19, 2024
e68b422
fix: fix internal references for datacube tutorial
EugenePlanteur Nov 19, 2024
9356567
feat: add point cloud notebook
EugenePlanteur Nov 20, 2024
69e0c69
doc: update readme
EugenePlanteur Nov 20, 2024
b80fc2f
install: update requirements.txt
EugenePlanteur Nov 20, 2024
22f8885
doc: fix typos
EugenePlanteur Nov 25, 2024
f0feb90
doc: fix typos
EugenePlanteur Nov 25, 2024
7279581
chore: add .las example file
EugenePlanteur Nov 25, 2024
d0c084f
refactor: change dataset name
EugenePlanteur Nov 25, 2024
e13c369
install: update requirements.txt
EugenePlanteur Nov 25, 2024
4011300
fix: create parent dir when generating hdf5 file
EugenePlanteur Nov 25, 2024
0f63f90
feat: add data optimization notebook
EugenePlanteur Nov 25, 2024
fdc0f34
doc: add data optimization ref to readme
EugenePlanteur Nov 25, 2024
8262143
style: add types, docstrings to functions + small refactor
EugenePlanteur Nov 25, 2024
e7bb48a
style: fix typo
EugenePlanteur Nov 27, 2024
0f5508f
style: fix typos
EugenePlanteur Nov 28, 2024
2244348
doc: improve install instructions
EugenePlanteur Nov 28, 2024
e6ba6d1
style: improve wording in data optimization notebook
EugenePlanteur Dec 3, 2024
dd40e96
feat: add KML format to vector formats tutorials
EugenePlanteur Jan 29, 2025
694c504
doc: update readme with instructions on the need to have an internet …
EugenePlanteur Jan 29, 2025
6122375
doc: update doc to link to data providers
EugenePlanteur Feb 3, 2025
9e8292b
doc: fix install instructions
EugenePlanteur Feb 3, 2025
2613ba8
doc: add documentation about sample data used in the notebooks
EugenePlanteur Feb 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2,756 changes: 2,756 additions & 0 deletions data_type/data_optimization.ipynb

Large diffs are not rendered by default.

6,306 changes: 6,306 additions & 0 deletions data_type/datacube_formats.ipynb

Large diffs are not rendered by default.

1,450 changes: 1,450 additions & 0 deletions data_type/point_clouds.ipynb

Large diffs are not rendered by default.

341 changes: 341 additions & 0 deletions data_type/raster_formats.ipynb

Large diffs are not rendered by default.

26 changes: 26 additions & 0 deletions data_type/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Tutorial: Geospatial Data Formats

Geospatial data allows the communication of information in a wide range of fields. Over time,
many data formats have emerged to support this diversity. This tutorial introduces these various
formats, their evolution, their advantages and disadvantages, and provides guidance on how and in
which contexts to use them most effectively. It will be divided into four jupyter notebooks:
EugenePlanteurCS marked this conversation as resolved.
Show resolved Hide resolved

1) [Raster formats](./raster_formats.ipynb)
2) [Vector formats](./vector_data_formats.ipynb)
3) [Data cube formats](./datacube_formats.ipynb)
4) [Point clouds](./point_clouds.ipynb)
5) [Data optimization](./data_optimization.ipynb)

For each tutorial, sample data will be provided in the `sample_data` directory or automatically
downloaded (in the `sample_data` directory by default)


To run a notebook, first create a virtual environment using pip. Then, install the required
packages (if not already done), using the `requirements.txt` file.

```bash
python3.11 -m venv venv
EugenePlanteurCS marked this conversation as resolved.
Show resolved Hide resolved
pip install -r requirements.txt
```

Then you can simply use `jupyter notebook xxx.ipynb` to run the corresponding notebook.
25 changes: 25 additions & 0 deletions data_type/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
Cartopy==0.24.1
dask==2024.11.2
fsspec==2024.10.0
geopandas==1.0.1
gcsfs==2024.10.0
geoviews==1.13.0
h5py==3.12.1
holoviews==1.20.0
hvplot==0.11.1
jupyter==1.1.1
laspy[lazrs,laszip]==2.5.4
matplotlib==3.9.2
netCDF4==1.7.2
numpy==2.1.3
pandas==2.2.3
panel==1.5.4
plotly==5.24.1
pyarrow==18.0.0
rasterio==1.4.2
Requests==2.32.3
rio_cogeo==5.3.6
rioxarray==0.18.0
scipy==1.14.1
xarray==2024.10.0
zarr==2.18.3
Binary file added data_type/sample_data/point_clouds/simple.las
Binary file not shown.
Loading