Investigation of different voltage levels on the Transmission Capacity using IT and DE resource files

[GbotemiB]

This PR Investigates the effect of the different voltage levels on the resulted transmission capacity in base_network.

[ekatef]

@GbotemiB great to see the progress. The implementation of get_i_nom_by_voltage( ) looks amazing!

A couple of preliminary comments:

1. Same comment for if_country_in_entsoe as in AT-MK notebook: probably, we may revise filtering making it cleaner and remove if_country_in_entsoe
2. Could you please move the voltage dictionary into arguments list of get_i_nom_by_voltage( )?
3. Regarding units in calculate_s_nom( ): in PyPSA standard types voltage is in kV, i_nom in kA which should give MVA

[ekatef]

Copying here @davide-f advice related to design of the comparison experiment:

The last notebook is for beyond 200kV. I think that we may first try to calibrate Eur to start
below 220, for sure 33kV, 66kV, 110kV, 132kV
but if the comparison for 220kV+ doesn't work, adding lower voltages is of secondary importance I believe

[davide-f]

Btw, 33kV is usually medium voltage

[ekatef]

@GbotemiB great, very interesting results! Thanks a lot advancing the code and adding the data. My feeling is that we are very close to finalising an eccensial part of the analysis.

A couple of presentation suggestions:

1. I think adding markdown headers (#Header 1, ##Header 2 etc.) would be helpful to get a quick understanding of the notebooks structure.

2. I wouldn't hesitate to change s_nom_aux into something more straight-forward, like s_nom_single or s_nom_circuit.

3. Would be perfect to have the tables side-by-side for pypsa-earth-base and ENTSO. Not sure it's easy to have a two-column layout in markdown, but probably we could try to make both tables horizontal? May be helpful for long tables like # table comparing auxiliary s_nom with s_nom in base_csv for DE.

4. It feels like we may use power of facet plots (a simple implementation like this or this would do the job) to present results of our parametric analysis.

As an idea, I'd suggest to vary voltage values and line types for each voltage (meaning different i_nom for the same voltage), and present barplots organised in facets by voltage-i_nom combinations. What is your feeling about that?

Currently, I have an impression that we may want to add a forth voltage value, like [u_low, 220., 300., 380.], taking u_low as 66kV or 110kV or 132kV.

5. It would be great to add a basic s_nom comparison between base.nc and base_network_csv for the updated elec.nc dataset with an increased threshold_voltage.

[GbotemiB]

@ekatef. thanks for the insightful review.
I have recently just pushed my commit changes. The notebook is ready for another review.

[ekatef]

@ekatef. thanks for the insightful review. I have recently just pushed my commit changes. The notebook is ready for another review.

Great improvements, @GbotemiB! Thanks for the great work.

I think that the notebook is basically ready, but it can inspire some changes in the modeling, which would need updates in the data and refreshing the notebook content. So, I wouldn't merge it right now to leave some space for us to work with modeling parameters.

But you may count this work as a thing done: that is establishing of a framework for analysis, which will be really helpful 🎉

[ekatef]

@GbotemiB thanks for improving the conclusion! However, I fear that the result is currently in a direct contradiction with the conclusion of AT-MK PR #67 . Would you mind to elaborate a bit?

I'd suggest to start from writing some plan like a draft I proposed in the review to #68. What do you think?

[ekatef]

Hello @GbotemiB! I do agree with your conclusion, but feel that it can be a bit more comprehensive. Would you mind to elaborate a bit?

My feeling is that making a plan for a conclusion may be very helpful. Can you do that? As a suggestion, you may probably find some inspiration in our work on the conclusion for #67, like a draft for a plan proposed here.

[GbotemiB]

Cool, thanks. I will adopt the plan from #67 here.

[ekatef]

Hello @GbotemiB! A great next iteration, and I think we are converging :)

The conclusion looks much more clear and conscience now. The only major comment is to explain that the aim of the analysis has been to investigate base.nc. You have just shown in AT-MK notebook that base.csv is quite close to the reference data, while there seem to be some troubles in the code which translates base.csv input into base.nc network. My feeling is that it could be worth to highlight it both in the introduction and in the conclusion.

A comment which relate to the whole analysis: note please that s_nom is a limit of the apparent power, not the power itself (see https://pypsa.readthedocs.io/en/latest/components.html#line). It could make sense to check that s_nom is explained correctly throughout the notebook.

I'd also recommend to go through the comments and text blocks and ensure that you understand and like all the explanations. Sometimes, it looks like grammar may be fixed (like respecting a capital in the phrase begin), sometimes text can be clarified a bit (like in the phrase func calc s_nom using the s_nom defined formula from pypsa earth). The analysis you have done is very helpful for further modeling improvements, and it would be great to make it as clear as we can.

Thanks for the great work!

[GbotemiB]

Much appreciated feedback from @ekatef,
I will let you know as soon as I am done with effecting the comments.

[ekatef]

Hello @GbotemiB!
Great work with making the improvements! Looks clear and neat now.

The only comment I have: would be nice to revise this introductory fragment to make more how to start with the notebook.

# Todo: Automate download process

# links to download data files
# https://drive.google.com/drive/folders/18dV790r11hHKIwpbyDBaxMV4XbhBFQde?usp=drive_link

It would suffice to replace the current Todo with an invitation to download the underlaying data manually. Something like that would do: This notebook uses a prepared base.nc network model which should be loaded manually and placed into subfolder of the current working folder "osm_data_validation_AT_MK_IT_DE " (please adjust as you feel like).

After that, I think we can count this work done for now 🙂

[GbotemiB]

Thank you very much @ekatef for the wonderful support. I have completed the changes as requested.

[ekatef]

Hello @GbotemiB!
Looks perfect 😄 Great work! My feeling is that we can count the notebook as the point get done for now.

However, I'd keep the PR open, until we'll find a solution for this issue, for which this notebook will be definitely very helpful. My feeling is that you have developed a perfect tool for analysis, while the improvements we have to introduce into the model are likely to change the comparison results significantly.

[ekatef]

@GbotemiB Thank you for the notebook! It is really handy to verify OSM. Just to keep track of events, posting here validation results also for Norway:

• one of the reasons of the OSM vs ENTSOe discrepancy is voltage mapping in base_network rule, this PR introduces a simple fix for that, but with OSM-extracted data, we still overestimate an overall transmission capacity twice;

• interestingly, data OSM for high-voltage grid seem to be not exactly accurate, while generally data availability is quite high for OSM:

My feeling is the the results confirm that a part of the found discrepancy wrong definition of a voltage level in OSM data. It's also worth to keep in mind that the highest voltage level in ENTSOe database (380 kV) do not necessarily correspond to the real highest voltage in the grid (420 kV in case of Norway).

[ekatef]

Ah, one more thing: it makes sense to check that the set-up instructions are completely clear. Would be great to describe in a bit more details which files and folders should be extracted and check that the downloading links work.

[GbotemiB]

Hi @ekatef, I have revised the instructions, verified the current download links, and added more links.
There is a new fail in the CI. I also use pre-commit before pushing.