Clustering isolated buses fails in simplify_network #908
Comments
|
Hello @dhoop-iidev! Thanks a lot for reporting. As a quick workaround, you may set Not sure about the reason of this issue. As a double-check: have you cleaned-up results of the previous runs before building the model for Argentina? Could you also please share your config settings? I'll try to reproduce the issue locally in meantime. |
|
Hello @ekatef ! Thanks for your quick answer. |
|
@dhoop-iidev thanks for the config! Unfortunately, I can't reproduce the issue you reported. So, the usual trouble-shooting should hopefully resolve the issue. As a suggestion, could you please remove all the output and re-run the simulation setting |
|
Alright I'll keep you posted if I ever find the solution |
|
Hi dhoop-iidev Did you find the solution to the problem ? I ask because I have the same problem. I look forward to your comments Thanks |
|
Hello @Javier-Andres-Gonzalez, |
|
Hello @dhoop-iidev If I can fix the problem, I will share the solution! I have spent a long time trying to find the problem, but still nothing. Do you speak spanish? |
|
Yes I live in Uruguay :) But let's continue in English so that the rest of the community can understand |
|
Are you also trying to model Argentina or another country ? |
|
I live in Argentina. I'm trying to follow the tutorial: (https://pypsa-eur.readthedocs.io/en/latest/tutorial.html) I am running this command: ( Error) snakemake -call results/test-elec/networks/elec_s_6_ec_lcopt_Co2L-24H.nc --configfile config/test/config.electricity.yaml |
|
Hello @Javier-Andres-Gonzalez, as discussed, the issue has been connected with different design of PyPSA-Earth and PyPSA-Europe. Since you are interested in modeling Argentina, you need pypsa-earth, which tutorial is available here https://pypsa-earth.readthedocs.io/en/latest/short_tutorial.html Thanks for reaching out and looking forward to hearing about your progress. |
|
Hello @dhoop-iidev! Do you have any updates regarding the issue you faced with A short trouble-shooting list which may resolve many problems with running the modeling workflow (have added a point to our documentation wish-list on that):
|
|
Thanks @ekatef, I think I've already done that in some order, but I'll run it again and let you know. If not I'll reinstall the whole thing from scratch. |
|
Alright I delete everything and cloned pypsa earth again from scratch and this time it worked. I don't know what the error was. |
@dhoop-iidev Thanks a lot for notifying! |
Checklist
Describe the Bug
Hello everyone,
I receive an "Assertion Error" during the simplify_network operation, more precisely when merging isolated nodes. It says that In Bus cluster bus_id, the values of attribute bus_id do not agree and then lists some buses.
I want to create a model for Argentina. The only changes I made from the default config.yaml is changing the country to ["AR"], creating a new cutout (ar-2013), and change the snapshot to start: "2013-01-01" and end "2013-02-01".
Thanks in advance for your help !
Error Message
INFO:snakemake.logging: [Tue Oct 24 11:37:10 2023] INFO:snakemake.logging:[Tue Oct 24 11:37:10 2023] rule simplify_network: input: networks/elec.nc, resources/costs.csv, resources/bus_regions/regions_onshore.geojson, resources/bus_regions/regions_offshore.geojson output: networks/elec_s.nc, resources/bus_regions/regions_onshore_elec_s.geojson, resources/bus_regions/regions_offshore_elec_s.geojson, resources/bus_regions/busmap_elec_s.csv, resources/bus_regions/connection_costs_s.csv log: logs/simplify_network/elec_s.log jobid: 5 benchmark: benchmarks/simplify_network/elec_s reason: Missing output files: networks/elec_s.nc, resources/bus_regions/regions_offshore_elec_s.geojson, resources/bus_regions/regions_onshore_elec_s.geojson; Input files updated by another job: resources/bus_regions/regions_offshore.geojson, resources/bus_regions/regions_onshore.geojson, networks/elec.nc wildcards: simpl= resources: tmpdir=C:\Users\le_re\AppData\Local\Temp, mem_mb=4000, mem_mib=3815 INFO:snakemake.logging:rule simplify_network: input: networks/elec.nc, resources/costs.csv, resources/bus_regions/regions_onshore.geojson, resources/bus_regions/regions_offshore.geojson output: networks/elec_s.nc, resources/bus_regions/regions_onshore_elec_s.geojson, resources/bus_regions/regions_offshore_elec_s.geojson, resources/bus_regions/busmap_elec_s.csv, resources/bus_regions/connection_costs_s.csv log: logs/simplify_network/elec_s.log jobid: 5 benchmark: benchmarks/simplify_network/elec_s reason: Missing output files: networks/elec_s.nc, resources/bus_regions/regions_offshore_elec_s.geojson, resources/bus_regions/regions_onshore_elec_s.geojson; Input files updated by another job: resources/bus_regions/regions_offshore.geojson, resources/bus_regions/regions_onshore.geojson, networks/elec.nc wildcards: simpl= resources: tmpdir=C:\Users\le_re\AppData\Local\Temp, mem_mb=4000, mem_mib=3815 INFO:snakemake.logging: This is the repository path: C:\Users\le_re\Documents\PyPSA\pypsa-earth Had to go 0 folder(s) up. INFO:pypsa.io:Imported network elec.nc has buses, carriers, generators, lines, loads, storage_units, transformers INFO:__main__:Mapping all network lines onto a single 380kV layer INFO:__main__:Simplifying connected link components INFO:__main__:Removing stubs C:\Users\le_re\Documents\PyPSA\pypsa-earth\.snakemake\scripts\tmpypqh1c2l.simplify_network.py:306: DeprecatedWarning: aggregategenerators is deprecated. Use `aggregateoneport` instead. generators, generators_pnl = aggregategenerators( INFO:__main__:Dropped 21 buses. A resulted load discrepancy is 4e+00% and 3e+00% for average load and generation capacity, respectively Traceback (most recent call last): File "C:\Users\le_re\Documents\PyPSA\pypsa-earth\.snakemake\scripts\tmpypqh1c2l.simplify_network.py", line 1007, in <module> n, merged_nodes_map = merge_isolated_nodes( File "C:\Users\le_re\Documents\PyPSA\pypsa-earth\.snakemake\scripts\tmpypqh1c2l.simplify_network.py", line 833, in merge_isolated_nodes clustering = get_clustering_from_busmap( File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\site-packages\pypsa\clustering\spatial.py", line 534, in get_clustering_from_busmap buses = aggregatebuses(n, busmap, custom_strategies=bus_strategies) File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\site-packages\pypsa\clustering\spatial.py", line 341, in aggregatebuses aggregated = n.buses.groupby(busmap).agg(strategies) File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\site-packages\pandas\core\groupby\generic.py", line 894, in aggregate result = op.agg() File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\site-packages\pandas\core\apply.py", line 169, in agg return self.agg_dict_like() File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\site-packages\pandas\core\apply.py", line 486, in agg_dict_like results = { File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\site-packages\pandas\core\apply.py", line 487, in <dictcomp> key: obj._gotitem(key, ndim=1).agg(how) for key, how in arg.items() File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\site-packages\pandas\core\groupby\generic.py", line 297, in aggregate return self._python_agg_general(func, *args, **kwargs) File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\site-packages\pandas\core\groupby\groupby.py", line 1654, in _python_agg_general result = self.grouper.agg_series(obj, f) File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\site-packages\pandas\core\groupby\ops.py", line 1081, in agg_series result = self._aggregate_series_pure_python(obj, func) File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\site-packages\pandas\core\groupby\ops.py", line 1104, in _aggregate_series_pure_python res = func(group) File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\site-packages\pandas\core\groupby\groupby.py", line 1640, in <lambda> f = lambda x: func(x, *args, **kwargs) File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\site-packages\pypsa\clustering\spatial.py", line 136, in consense assert (x == v).all() or x.isnull().all(), ( AssertionError: In Bus cluster bus_id, the values of attribute bus_id do not agree: Bus 46 46 341 341 346 346 420 420 423 423 451 451 545 545 555 555 571 571 579 579 581 581 606 606 619 619 621 621 628 628 645 645 650 650 651 651 673 673 679 679 690 690 699 699 704 704 732 732 733 733 740 740 751 751 754 754 757 757 760 760 794 794 799 799 803 803 954 954 Name: bus_id, dtype: object [Tue Oct 24 11:37:20 2023] INFO:snakemake.logging:[Tue Oct 24 11:37:20 2023] Error in rule simplify_network: jobid: 5 input: networks/elec.nc, resources/costs.csv, resources/bus_regions/regions_onshore.geojson, resources/bus_regions/regions_offshore.geojson output: networks/elec_s.nc, resources/bus_regions/regions_onshore_elec_s.geojson, resources/bus_regions/regions_offshore_elec_s.geojson, resources/bus_regions/busmap_elec_s.csv, resources/bus_regions/connection_costs_s.csv log: logs/simplify_network/elec_s.log (check log file(s) for error details) ERROR:snakemake.logging:Error in rule simplify_network: jobid: 5 input: networks/elec.nc, resources/costs.csv, resources/bus_regions/regions_onshore.geojson, resources/bus_regions/regions_offshore.geojson output: networks/elec_s.nc, resources/bus_regions/regions_onshore_elec_s.geojson, resources/bus_regions/regions_offshore_elec_s.geojson, resources/bus_regions/busmap_elec_s.csv, resources/bus_regions/connection_costs_s.csv log: logs/simplify_network/elec_s.log (check log file(s) for error details) RuleException: CalledProcessError in file C:\Users\le_re\Documents\PyPSA\pypsa-earth\Snakefile, line 592: Command 'C:/Users/le_re/mambaforge/envs/pypsa-earth/python.exe "C:\Users\le_re\Documents\PyPSA\pypsa-earth\.snakemake\scripts\tmpypqh1c2l.simplify_network.py"' returned non-zero exit status 1. File "C:\Users\le_re\Documents\PyPSA\pypsa-earth\Snakefile", line 592, in __rule_simplify_network File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\concurrent\futures\thread.py", line 58, in run ERROR:snakemake.logging:RuleException: CalledProcessError in file C:\Users\le_re\Documents\PyPSA\pypsa-earth\Snakefile, line 592: Command 'C:/Users/le_re/mambaforge/envs/pypsa-earth/python.exe "C:\Users\le_re\Documents\PyPSA\pypsa-earth\.snakemake\scripts\tmpypqh1c2l.simplify_network.py"' returned non-zero exit status 1. File "C:\Users\le_re\Documents\PyPSA\pypsa-earth\Snakefile", line 592, in __rule_simplify_network File "C:\Users\le_re\mambaforge\envs\pypsa-earth\lib\concurrent\futures\thread.py", line 58, in run Removing output files of failed job simplify_network since they might be corrupted: resources/bus_regions/connection_costs_s.csv WARNING:snakemake.logging:Removing output files of failed job simplify_network since they might be corrupted: resources/bus_regions/connection_costs_s.csv Shutting down, this might take some time. WARNING:snakemake.logging:Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message ERROR:snakemake.logging:Exiting because a job execution failed. Look above for error message Complete log: .snakemake\log\2023-10-24T104521.772466.snakemake.log WARNING:snakemake.logging:Complete log: .snakemake\log\2023-10-24T104521.772466.snakemake.logThe text was updated successfully, but these errors were encountered: