MissingOutputException in rule prepare_gas_network

[doneachh]

Checklist

• I am using the current main branch or the latest release. Please indicate.
• I am running on an up-to-date pypsa-africa environment. Update via conda env update -f envs/environment.yaml.

Describe the Bug

We don't create those pngs when running in the following block:
else:
print(
"The following countries have no existing Natural Gas network between the chosen bus regions:\n"
+ ", ".join(bus_regions_onshore.country.unique().tolist())
)

    # Create an empty DataFrame with the specified column names
    pipelines = {"bus0": [], "bus1": [], "capacity": [], "length": [], "GWKm": []}

    pipelines = pd.DataFrame(pipelines)
    pipelines.to_csv(snakemake.output.clustered_gas_network, index=False)

Error Message

Waiting at most 5 seconds for missing files.
MissingOutputException in rule prepare_gas_network in file /mnt/l/Git/Namibia/pypsa-earth-sec/Snakefile, line 148:
Job 25  completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
resources/gas_networks/existing_gas_pipelines__10.png
resources/gas_networks/clustered_gas_pipelines__10.png
Removing output files of failed job prepare_gas_network since they might be corrupted:
resources/gas_networks/gas_network_elec_s_10.csv

[davide-f]

Personally, as mentioned in the PR, I'd recommend to keep the workflow consistent and keep the plotting functions as separate rules.
What do all think about it?
@energyLS @hazemakhalek @Eddy-JV

P.S. @doneachh I really recommend you to attend a meeting as we are discussing the merge and I believe it is relevant for you

[doneachh]

@davide-f sounds like a good solution. Sadly i can not join today as i have another meeting at the same time, but keep me updated what the others say to this issue and I'll adjust this PR afterwards :)

[energyLS]

Personally, as mentioned in the PR, I'd recommend to keep the workflow consistent and keep the plotting functions as separate rules. What do all think about it? @energyLS @hazemakhalek @Eddy-JV

P.S. @doneachh I really recommend you to attend a meeting as we are discussing the merge and I believe it is relevant for you

Totally agree here, plotting/analysis should be separate.