Errors during extraction of datafiles (tar directory targets)

[euronion]

During rasterisation a lot of files where in the wrong location or were not unzip-ped / un-tar-ed correctly.

I had a few unsuccesfull runs of rasterize_datasets(cleanup=:limited) and download_datasets(). Maybe that caused the problem?

The affected files and directories I had to manually cleanup were:

• unzip NUTS.zip
• renaming of the WDPA files rename 's/^WDPA_Apr2020/WDPA/' WDPA_*
• create the temp_ssp2 folder and populate it
• mv GlobalEnergyGIS/ETOPO1_Ice_c_geotiff/ETOPO1_Ice_c_geotiff.tif GlobalEnergyGIS/ETOPO1_Ice_c_geotiff.tif
• unzip timezones-with-oceans.shapefile.zip into timezones-with-oceans.shapefile

[niclasmattsson]

I just completed a fresh install on a Windows laptop. The download and rasterization took several hours on this slow and low-memory machine, but it completed and cleaned up without the problems you had. But I'll leave this issue open until I get the chance to test a Linux box as well.

I do appreciate the reports though. Please keep them coming if you have more problems, but I hope things become smoother for you from now on.

[euronion]

I did a clean install and tried to download the datafiles again.
It works up to the step where it tries to unpack the tar file temp_ssp2.tar and fails with the following error:

Unpacking archive: temp_ssp2.tar
tar: /home/<usr>/share/GlobalEnergyGIS/temp_ssp2: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
ERROR: failed process: Process(`tar xf /home/<usr>/share/GlobalEnergyGIS/temp_ssp2.tar --directory=/home/<usr>/share/GlobalEnergyGIS/temp_ssp2`, ProcessExited(2)) [2]

Stacktrace:
 [1] pipeline_error at ./process.jl:525 [inlined]
 [2] run(::Cmd; wait::Bool) at ./process.jl:440
 [3] run at ./process.jl:438 [inlined]
 [4] unpack(::String, ::String, ::String) at /home/<usr>/.julia/packages/GlobalEnergyGIS/gFPzK/src/downloaddatasets.jl:146
 [5] download_datasets(::Int64) at /home/<usr>/.julia/packages/GlobalEnergyGIS/gFPzK/src/downloaddatasets.jl:75
 [6] download_datasets() at /home/<usr>/.julia/packages/GlobalEnergyGIS/gFPzK/src/downloaddatasets.jl:6
 [7] top-level scope at REPL[1]:1

It seems as if the directory temp_ssp2 is not created before trying to extract the tar archive. In the background BinDeps is relying on 7z for Windows and the tar command for Linux. Both behave differently on output directories, i.e. I think you need to create all directories where you want to untar tar archives to first.