As AI-as-a-Service developer for a new road weather forecasting model applied to the road weather datasets of the Open Data Hub, I would like to integrate some weather forecasts data in the Open Data Hub, so that the implementation of the model is simpler

[rcavaliere]

• set up module for data extraction (see https://pypi.org/project/dwdGribExtractor/)
• define technical specifications on how to integrate the data in the Open Data Hub
• implementation of the Data Collector

[clezag]

@rcavaliere are you sure about this library?

I have taken a quick look and initially failed to install it, because in their own install instructions they had a typo, so the package name was wrong.
The repo hasn't gotten any commits for 3 years, has 0 issues opened, no forks, and from the typo in the install instructions still being there I assume nobody is using this.

I'd just like to avoid putting in a lot of work and then realizing the library doesn't work right for whatever reason (in 3 years maybe the file formats changed or something similar), or we have to maintain the library ourselves if future issues come up.

EDIT: nevermind, I've looked at the actual code, and it's just a few hundred lines, so we can probably manage if there are no alternatives

[rcavaliere]

@clezag CISMA has provided me now some more details, I will upgrade the technical details on how to extract the data here by preparing the usual technical specification document. I will however work on this starting next week

[ohnewein]

@dulvui is in an e-mail thread investigating together with @rcavaliere and CISMA about this issue.

[rcavaliere]

Based on some information received yesterday by CISMA, the technical feasibility should be confirmed, @ohnewein

[rcavaliere]

Instruction for preliminary JSON creation consolidated with CISMA:

240902_SpecificheIntegrazione_WeatherForecasts_NOI_v1.1.docx

@dulvui can you provide some JSON examples, so that we can design the integration in our database?

[clezag]

Here's a test JSON:
test.json

If you want to extract some yourself, you can use this script as extract.sh
(GH doesn't allow upload of .sh files)

#!/bin/bash
wget -qO - $1 | bzip2 -dc > tmp
docker run --name grib2json --rm -v ./tmp:/input -e "ARGS=--names --data /input" alexwoif/docker-grib2json
rm tmp

And then just run it like this:

./extract.sh https://opendata.dwd.de/weather/nwp/icon-d2/grib/21/t_2m/icon-d2_germany_regular-lat-lon_single-level_2024120821_020_2d_t_2m.grib2.bz2 > test.json

It didn't work on the icosahedral type files, I assume we'd have to upgrade some dependencies in this tool, it hasn't been maintained for 7 years.