review/ksz2050

[wingechr]

We have uploaded the first three tables for the ksz2050 scenario and submit them to the review process:

• model_draft.ksz2050_r2_ks95_ghg_emissions_overview
• model_draft.ksz2050_r2_ks80_ghg_emissions_overview
• model_draft.ksz2050_r2_ams2012_ghg_emissions_overview

We added the meta data json files in a new branch: https://github.com/OpenEnergyPlatform/data-preprocessing/tree/review/ksz2050 (but no pull request yet).

[l-emele]

We have uploaded further tables for the ksz2050 scenario to be reviewed:

• model_draft.ksz2050_r2_ams2012_co2_emissions_industrial_processes
• model_draft.ksz2050_r2_ams2012_electricity_consumption
• model_draft.ksz2050_r2_ams2012_electricity_overview
• model_draft.ksz2050_r2_ams2012_energy_and_climate_targets
• model_draft.ksz2050_r2_ams2012_energy_consumption_other_energy_industries
• model_draft.ksz2050_r2_ams2012_fugitive_emissions
• model_draft.ksz2050_r2_ams2012_ghg_emissions_detailed_data
• model_draft.ksz2050_r2_ams2012_ghg_emissions_from_combustion_by_fuel
• model_draft.ksz2050_r2_ams2012_parameters_assumptions
• model_draft.ksz2050_r2_ams2012_primary_and_final_energy_consumption
• model_draft.ksz2050_r2_ks80_co2_emissions_industrial_processes
• model_draft.ksz2050_r2_ks80_electricity_consumption
• model_draft.ksz2050_r2_ks80_electricity_overview
• model_draft.ksz2050_r2_ks80_energy_and_climate_targets
• model_draft.ksz2050_r2_ks80_fugitive_emissions
• model_draft.ksz2050_r2_ks80_ghg_emissions_detailed_data
• model_draft.ksz2050_r2_ks80_ghg_emissions_from_combustion_by_fuel
• model_draft.ksz2050_r2_ks80_parameters_assumptions
• model_draft.ksz2050_r2_ks80_primary_and_final_energy_consumption
• model_draft.ksz2050_r2_ks95_co2_emissions_industrial_processes
• model_draft.ksz2050_r2_ks95_electricity_consumption
• model_draft.ksz2050_r2_ks95_electricity_overview
• model_draft.ksz2050_r2_ks95_energy_and_climate_targets
• model_draft.ksz2050_r2_ks95_fugitive_emissions
• model_draft.ksz2050_r2_ks95_ghg_emissions_detailed_data
• model_draft.ksz2050_r2_ks95_ghg_emissions_from_combustion_by_fuel
• model_draft.ksz2050_r2_ks95_parameters_assumptions
• model_draft.ksz2050_r2_ks95_primary_and_final_energy_consumption

All in the review/ksz2050 branch: https://github.com/OpenEnergyPlatform/data-preprocessing/tree/review/ksz2050

[christian-rli]

I had a first look at the metadata. Unfortunately the metadata-structure does not conform to the latest standard (our fault for not keeping you up to date). I'm going to update the structure to "OEP-1.4.0" and then see further.

[christian-rli]

Specifically, the following issues are addressed in the restructuring:

• "fundingAuthority" -> "fundingAgency"
• "fundingAuthorityLogo" -> "fundingAgencyLogo"
• missing key "publisherLogo"
• all "none" values should be null
• "timeseries" structure in "temporal" missing
• • "timestamp" -> "alignment"
• • missing "aggregationType"
• "licenses" structure in "sources" missing

[han-f]

We updated the metadata another time and added publisher logo, corrected contact email etc. There is a pull request now (#39 ) since we think the data including metadata is now final and can be moved to the scenario schema.

[christian-rli]

I moved the tables to the scenario schema, but metadata aren't visible yet. Version 1.4 is not fully supported yet. Maybe I can convert it back to the old version to have the metadata show in the mean time.

[han-f]

Is omi not yet able to do this?
From my perspective it would be very useful if the metadata could be shown very soon. It includes a phrase about the funding, which needs to be shown in any case. I am thus wondering whether there is an easy way to only include that phrase for now and then make the 1.4 metadata shown once the platform is ready.

[han-f]

Many thanks for moving the data. In the process, all tags we assigned got lost. At least they are not attached to the tables any longer. I have opened a separate ticket for this: OpenEnergyPlatform/oeplatform#426

[christian-rli]

Sorry about the lost tags @han-f . Hopefully we can restore them.

omi ist able to understand version 1.4, but support/implementation into the oep is not complete. This is a pending issue already called for here, but I just realized there is no actual issue for this. So I created a new one here: OpenEnergyPlatform/oeplatform#427 . Sorry @MGlauer :)

The older version of our metadata (1.3) does not have a "fundingAgency" or equivalent key. A temporary solution would be to add the funding phrase into the description of the older version. I tried to parse the metadata into version 1.3, but ran into new issues.

[han-f]

Thanks @christian-rli. The tags are back, many thanks!
Interestingly, however if I click a tag in the most popular tags sidebar - I do not get any results.?
If it worked out to have the funding phrase in the description this would be great too.

[han-f]

@christian-rli : any news on adding the funding phrase into the description field of metadata version 1.3?

[christian-rli]

Sorry for the wait @han-f . Unfortunately converting backwards still has some issues. I decided to upload the string in version 1.4.0 to all tables. Now it doesn't look pretty, but at least the information is there. When support for 1.4.0 finally arrives these will be the first tables to work properly (and a good reference point for the implementation as well).

[l-emele]

I just had an look on this issue. This issue is now pending five months and to me it is not exactly clear what changes in out metadata are needed that we can finalize the review of the metadata and create a pull request.

@christian-rli: Could you please exactly specifcy the changes needed in our metadata (e.g. as a to do list)?

[christian-rli]

Sorry if this issue seems pending @l-emele . No changes need to be done. The strings look good and were uploaded to the OEDB. The only remaining issue is that they don't look nice on the OEP yet. No strings of version 1.4 do up until this moment. This issue was addressed in OpenEnergyPlatform/oeplatform#427 , so once the next release is out, everything should look fine, as well. I kept this issue open as a reminder to see if it actually does. If you prefer, we can close this.

[han-f]

Does that mean that the data in the database is correct, but what is currently displayed on OEP is wrong? I only ask because one thing is different between what we submitted here and what is displayed currently on OEP: the email address for contact.

[l-emele]

On the OEP also in our metadata file it shows [email protected]. That is the correct one.

[han-f]

Oh interesting - on the OEP the (not pretty) metadata showed [email protected] for me. Let me check if this was a browser hiccup

[han-f]

On the OEP the email address at the beginning is correct, the one under contributors is still [email protected], can this easily be corrected for all KSZ tables?

[jh-RLI]

What @christian-rli meant is that the representation of the metadata string in version 1.4 on the oep does not look good yet. The string is displayed as plain text and not formatted. This will change after the relaunch.

[jh-RLI]

To change the content of the metadata on the oep the metadata files on github have to be updated. @christian-rli can then append the corrected version to the tables.

[han-f]

Thanks @jh-RLI and yes I know I need to update them here. I did this now by opening each file once and changing it. I just wondered above whether it would be possible to do it more efficiently than opening each single file, as there are many.
@christian-rli could you please append the updated version to the tables. I have changed the email address under contributors to the correct one. Apologies for not adding this as a header to each commit message.

[jh-RLI]

@han-f oh okay. Unfortunately, I do not know a better solution than search and replace. Maybe an idea for another tool that can adjust multiple metadata files at once.

[christian-rli]

The updated versions are now attached to the tables @han-f . Thank you @Ludee for being my remote sql-console and fixing my little errors :)

[han-f]

Thanks. That probably means this ticket can be closed.