Wp flex changed reinforcement #395

birgits · 2024-01-18T22:35:54Z

Description

This PR mainly changes the implementation of reduced_analysis. reduced_analysis can be used to conduct the grid reinforcement on a reduced number of time steps to avoid running the power flow for all time steps, including ones without grid issues, and thus speed up calculations. As it was implemented so far, the reduced_analysis could not be used to run the reinforcement for single LV grids or a subset of time steps. The latter is needed to use it in the catch_convergence_reinforce_grid function to conduct grid reinforcement for the converged time steps first.

The PR also now allows to weight time steps by estimated grid expansion costs analogous to how it was already possible in the selection of the most critical time intervals. This can be useful to further reduce the number of time steps to the most relevant ones.

Further, a fixes small bug in selection of time intervals.

Type of change

Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

New and adjusted code is formatted using the pre-commit hooks
~~New and adjusted code includes type hinting now~~
I have commented my code, particularly in hard-to-understand areas
~~I have made corresponding changes to the documentation~~
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
The Read the Docs documentation is compiling correctly
If new packages are needed, I added them the setup.py, and if needed the rtd_requirements.txt, the eDisGo_env.yml and the eDisGo_env_dev.yml.
I have added new features to the corresponding whatsnew file

This reverts commit 43552cf.

AnyaHe

Hi @birgits ,

I left a few comments, but in general I think everything is good and this PR can be merged with minor changes. I do not fully understand the enhanced reinforcement so there, I have to trust that everything is correct.

Thanks for these improvements 👍

AnyaHe · 2024-02-16T08:33:47Z

edisgo/tools/temporal_complexity_reduction.py

+        The costs don't convey the actual costs but are an estimation, as
+        the real number of parallel lines needed is not determined and the whole feeder
+        length is used instead of the length over two-thirds of the feeder.
+        If False, the severity of each feeder's voltage issue is set to be the same.


Is it really the severity, that is set to be the same or rather the weights?

No, it's actually the severity. It is here set to 1. We could also change that to maybe use the maximum voltage deviation?

If there is no specific reason to do so, I would probably change it, so it is consistent with the overloading. Or why would we handle the voltage deviation differently?

It was set to 1 before and I just left it as it was, but was also wondering about it. I changed it now so that the maximum voltage deviation in the feeder is used.

AnyaHe · 2024-02-16T10:45:02Z

edisgo/tools/temporal_complexity_reduction.py

@@ -661,7 +959,9 @@ def get_most_critical_time_steps(
    steps = loading_scores[:num_steps_loading].index


I know this has not changed but I was wondering whether it might be interesting to know how many timesteps there were to begin with, if you choose to run only a limited number of timesteps. So maybe add an else-statement to the previous if statement with a logger.info(f"{num_steps_loading} of a total {len(loading_scores)} relevant time steps are chosen for the reinforcement evaluation.")? This way, the user can at least see which share of the relevant time steps is evaluated in the end. Same comment for line 975 with a limited number of voltage issues.

Good idea, I added it!

AnyaHe · 2024-02-16T10:59:15Z

edisgo/tools/temporal_complexity_reduction.py

+    if weight_by_costs:
+        # weight line violations with expansion costs
+        costs = _costs_per_line_and_transformer(edisgo_obj)
+        crit_lines_score = crit_lines_score * costs.loc[crit_lines_score.columns]


This is just a thought and not required for this PR, but might be an enhancement. Would it not make sense to apply np.ceil and round to the next integer value (after subtracting 1)? After all it is more an integer decision, how many new lines we need and does not depend on the severity of the overloading in between integer values.

That's a good point. I thought about for a while and I think both the way we do it now and what you are suggesting are not really correct as the relative loading doesn't convey the number of additional lines needed to solve the problem.
E.g. when there are two 1 MW lines and the loading is 3.2 MW, then we would need two more lines of the same type to solve the problem. So in that case subtracting 1 and ceiling would not be correct, because relative loading is 1.6 and subtracting 1 and ceiling would only yield 1 line. But when there is only one 1 MW line and the loading is 3.2 MW, then subtracting 1 and ceiling would yield the right number of lines.
The correct way would probably be to use the absolute overloading and the s_nom of the standard line in the respective voltage level. But I think since this should just be an estimation of the costs I would leave it like it is for now. I will make a ToDo there, so we can maybe change it at some point.

Ah true, I did not think of parallel lines. Yes I guess it is fine for now and we can think about adapting it some other time.

AnyaHe · 2024-02-16T13:06:19Z

tests/flex_opt/test_reinforce_grid.py

        )
+        assert len(res_reduced.i_res) == 2


In general in this test, I feel like there are not many things that are being tested against. Is everything covered in other tests or might it be an idea to at least specify the length of the equipment changes or total costs that we would expect with every mode?

Alot more is for the reinforcement tested in the EDisGo class tests. At this point I just wanted to check, whether the usage of two time steps is followed through throughout the reinforcement. Do you still think more tests need to be added or is it okay?

Ah then it is probably okay. It seems like it is never tested for specific values (e.g. the sum of reinforcement costs in a specific case). Is that on purpose? Would we not want to be able to detect if something changes with the values as well, not just the number of reinforced elements?

It's not on purpose and could be added some time. I would rather test the content of equipment changes then, because the costs function test already tests the derivation of the costs from the equipment changes. I'll add a ToDo for now.

AnyaHe · 2024-02-16T13:28:04Z

edisgo/flex_opt/reinforce_grid.py

@@ -737,6 +766,7 @@ def reinforce():
            "reinforcement."
        )
        selected_timesteps = converging_timesteps
+        troubleshooting_mode = troubleshooting_mode_set
        reinforce()

    # Run reinforcement for time steps that did not converge after initial reinforcement


I know this did not change with this PR and it is a very specific case but I would add a "and not converging_timesteps.empty" in the if-statement above. Otherwise you run the same function twice which is unnecessary. If no timesteps converged, there is no reinforcement and the situation does not change for the other timesteps that do not converge.

I don't understand what you mean here, sorry. The if statement above this line is already if not converging_timesteps.empty.

I mean to add the respective statement here:

eDisGo/edisgo/flex_opt/reinforce_grid.py

Line 773 in 70c3264

if not non_converging_timesteps.empty:

It is just in the case that no timesteps converged, it is unnecessary to run lines 774-780, because the timesteps did not converge before and the grid did not change. So it is certain that the timesteps will not converge again. So instead of "if not non_converging_timesteps.empty:", I would write "if not non_converging_timesteps.empty and not converging_timesteps.empty:"
Does that make sense?

You're right! I changed it now, so that it is only tried to be reinforced in case it was previously reinforced for converged time steps.

AnyaHe · 2024-02-16T14:19:20Z

setup.py

-    "pandas >= 1.4.0",
+    # newer pandas versions don't work with specified sqlalchemy versions, but upgrading
+    # sqlalchemy leads to new errors.. should be fixed at some point
+    "pandas < 2.2.0",


I think this requirement should also be added to the yml files, for all windows users :)

You're right, I added it.

AnyaHe · 2024-02-16T14:32:18Z

edisgo/flex_opt/reinforce_grid.py

@@ -746,6 +776,7 @@ def reinforce():
            "reinforcement."
        )
        selected_timesteps = non_converging_timesteps
+        troubleshooting_mode = False


I have to be honest. I dont get why troubleshooting is turned off in between. But then again, I am also not very familiar with the enhanced_reinforce_grid. I trust this is correct, but dont feel like I understand enough to properly assess this.

Here I turned off troubleshooting mode, as in case power flow still not converges for all time steps, we want to go straight to the iterative grid reinforcement instead of running the power flow with reduced time series until it converges in get_most_critical_time_steps(). However, now that I think of it a bit more, the troubleshooting mode probably never makes sense for the grid reinforcement because it is only used to determine the most critical time steps, but in power flows afterwards in reinforce_grid() it would still results in convergence issues. Or can you think of a case where it might be helpful to have that parameter in the reinforcement?

From the top of my head, I cannot think of any.

I set the default to False, as it makes more sense. The user can still set it to True if he or she wants to.

birgits · 2024-02-16T22:10:50Z

Thank you so much @AnyaHe for your valuable comments. I implemented some of your suggestions and also have a few points I would like to discuss further. See comments above :)

…f grid has beed reinforced in between

…equally

…grid reinforcement

Stackoverflow links for some reason now fail when checked with github actions, even though the link is correct. They are therefore for now ignored.

AnyaHe · 2024-02-21T08:35:09Z

Thanks @birgits for addressing all the comments and the improvements in this PR. From my side, you can merge now. Great work! :)

birgits added 30 commits October 22, 2023 13:23

Delete ToDo

5073260

Fix use same time index

4f6fe13

Allow selecting most critical time steps from subset of time steps

8c6e57a

Change how reduced_analysis is set

475a2f3

More changes to change how reduced analysis is set

b7886fc

Change LV reniforcement

43552cf

Fix type hinting

0f15fab

Bug fix hand analyze parameters to get_most_critical_time_steps

b17d714

Allow not running an inital analyze when using reduced analysis

b54ebc9

Quick bug fix

70748ce

Bug fix when analyze mode is lv, station node voltage cannot be checked

708bdb3

Change back as this wasn't the problem

bd5c888

Add return in case no timesteps for reinforcement exist

c832690

Bug fix add parameters when function is called

44a2203

Fix only use reduced analysis when power flow converges

51b53de

Add helper function temporarily

1bd354b

Bug fix

9cc3e67

Merge branch 'dev' into wp_flex_changed_reinforcement

e5add23

Merge branch 'dev' into wp_flex_changed_reinforcement

7b7a7e1

Revert "Change LV reniforcement"

74381c3

This reverts commit 43552cf.

Fix test - setup class needs to be rerun

4aff553

Get rid of redundant code and parameter documentation

db325bc

Allow weighting by costs or without weighting factor

716c636

Fix spelling

8d046b3

Bugfix and remove time intervals without grid issues

c2e45ad

Adapt tests to bugfix and new parameter

edd4ba3

Adapt test

0595d57

Move approximation of costs to separate functions and add tests

4979e3b

Add weighting by costs in time steps selection

70b8da5

Add tests

cf0c412

birgits added 3 commits January 30, 2024 16:25

Add test

139fd7f

Limit pandas version

b6d2f9e

Add changes to whatsnew

70c3264

birgits marked this pull request as ready for review February 1, 2024 01:40

birgits requested a review from AnyaHe February 1, 2024 01:40

AnyaHe approved these changes Feb 16, 2024

View reviewed changes

birgits added 3 commits February 16, 2024 10:19

Add logging information

85b49f4

Add ToDo for how function could be improved

9e0d85d

Restrict pandas version

c272fda

birgits added 7 commits February 16, 2024 14:30

Change links that are now redirected

dfb71ae

Only conduct reinforcement for previously non-converging time steps i…

7ec2ec2

…f grid has beed reinforced in between

Use maximum deviation in feeder rather than weighting all deviations …

a1ac8c8

…equally

Add ToDo

b99abba

Change default for use_troubleshotting_mode as True is not useful in …

f04bc2f

…grid reinforcement

Try fixing failing link check github action

0657cd1

Ignore stackoverflow links when checking links

79b5a31

Stackoverflow links for some reason now fail when checked with github actions, even though the link is correct. They are therefore for now ignored.

birgits merged commit 0f603cc into dev Feb 21, 2024
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wp flex changed reinforcement #395

Wp flex changed reinforcement #395

birgits commented Jan 18, 2024 •

edited

Loading

AnyaHe left a comment

AnyaHe Feb 16, 2024

birgits Feb 16, 2024

AnyaHe Feb 17, 2024

birgits Feb 19, 2024

AnyaHe Feb 16, 2024

birgits Feb 16, 2024

AnyaHe Feb 16, 2024

birgits Feb 16, 2024

AnyaHe Feb 17, 2024

AnyaHe Feb 16, 2024

birgits Feb 16, 2024

AnyaHe Feb 17, 2024

birgits Feb 19, 2024

AnyaHe Feb 16, 2024

birgits Feb 16, 2024

AnyaHe Feb 16, 2024

birgits Feb 16, 2024 •

edited

Loading

AnyaHe Feb 16, 2024

birgits Feb 16, 2024

AnyaHe Feb 16, 2024

birgits Feb 16, 2024

AnyaHe Feb 17, 2024

birgits Feb 20, 2024

birgits commented Feb 16, 2024

AnyaHe commented Feb 21, 2024

		@@ -661,7 +959,9 @@ def get_most_critical_time_steps(
		steps = loading_scores[:num_steps_loading].index

Wp flex changed reinforcement #395

Wp flex changed reinforcement #395

Conversation

birgits commented Jan 18, 2024 • edited Loading

Description

Type of change

Checklist:

AnyaHe left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

birgits Feb 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

birgits commented Feb 16, 2024

AnyaHe commented Feb 21, 2024

birgits commented Jan 18, 2024 •

edited

Loading

birgits Feb 16, 2024 •

edited

Loading