New CDS API and more

This is a lone overdue release as new features and typos/bus have been corrected. This new release now supports the new CDS API for ERA5 data download.

WARNING: There may be some compatibility issues with workflows based on previous release.

• New tool to fetch Copernicus DEM [19abe84] with paralleisation [1d834bd]
• improvement of path directory code [72dcd3e]
• debug of masking [3713f83]
• function to export to FSM2oshd [1ed7b6d]
• correction of a bug when elevation was lower than the lower pressure level [e133c6e]
• adding compatibility with IFS data [134f7ce] and script to feth IFS data [69c9845]
• implementation of an export function to FSM2 model [749a0e3]
• improvements in the routine for forecast [cd75c5a]
• added rioxarray to requirements [de8ae0c]
• added export function for SURFEX model [f5ad251]
• added explicit get_era5() function in workflow [7b35bcd]
• fixed typos and bug in topo_sim.py [327fe4f] et al.
• added dask chuncking to stop memory leaks [fd2adc4]
• output format of the ERA5 data is now dynamic and not hard coded to netcdf [3cb0b49]. now support for
• update of new variable keywords of CDS-beta [ef13238]
• replaced raise warning ... by simple prints as it led to crash [f8da401]
• new feature to download ERA5 from zarr repositories [3eb160d]
• added possibilities to downscale at other timestep than 1h only [d77d988]

addressing multiple typos

This release includes a working version for exporting to FSM2ohd, and a number of other correction in the code. These are minor typos that led to inconsistencies.

FSM2oshd support & Arbitray domain mask

New release bringing latest development of new features into TopoPyScale:

• Support to export to FSM2oshd format 3fe1e1a
• added support for masking out domain or split domain in independent regions 81298bf
• added support for custom outputs directory 951451e
• rename files containign FSM simulation routines to sim_fsm.py
• added support for C-implementation of solar position in pvlib library 074710a
• point_id can be anything from int, float to string 1a32a5d

and a number of minor bug fixes

v0.2.3 JOSS release.

This new version of the package is justified by the release of the JOSS publication following the review process. In the meantime a number of features have been added.

• support for SLURM server type
• realtime processing, downloading latest ERA5 data. For operational setup c0fb49f
• git commit hash and package version added to netcdf file metadata. This ensure to know which version of TopoPyScale was used to create data.
• climate input can now be located outside of the project directory. This prevent to copy multiple times the same data. 77fd990
• Outputs directory can be named independently of default
• export FSM as netcdf
• streamlined the environment.yml file 5af8c8d
• fix Dask indexing: 4b00cd6
• option to give specifc name to cluster points and not depend on the default point_id 7695450
• function to export griddedata 420f80a
• update xarray functions that were deprecated 3662908
• possibility to run FSM in parallel mode 91c8528
• added snow routine for steep slope 4d31fd1

Update to Python 3.9 and latest `xarray`

This release includes:

• upgrade to Python 3.9 and Xarray 2.3
• no more problem of memory byt using open_mfdataset
• new testing script with github action
• added units to default downscaled dataset
• n_core option is less restrictive now.
• solved a number of path issues
• a number of improvements of the FSM simulation topo_sim.py routines
• corrected projection issues when interpolating the climate dataset
• to_cryogrid() now compatible with Cryogrid.jl
• added an environment.yml file for version controlled install of VE
• plotting function map_variable() has been generalized
• and a number of bugs have been caught along the way.

Another release for less bugs

New release resolving a number of bugs:

• a crucial bug occurring when storing datasets during downscalng in topo_scale.py. More details in issues: #60 and commit a33e79f
• a bug resolved in to_crocus() 4e73489
• a bug when computing level elevation from geopotential 6a4c68e
• a bug in to_musa() 81e8978

Parallelization is up and running :)

TopoPyScale is improving and being tested by more users. New features, improvements and bug fixes justify a new release. The whole downscaling process gain speed by spreading the load on multicores. This was tested on laptops and servers.

WARNING: This new release requires Python >= 3.8. Otherwise, python runs into a memory leak and crashes.

New features:

• multicore, paralleization of topo_scale
• tools to explore optimal number of clusters in topo_sub
• implementation of feature importance for clustering. New field in config file needed
• added precipitation lapse-rate option, commit

Improvements:

• simplification of path management using the library pathlib
• remove all usage of open_mfdataset() or to_mfdataset that are not as efficient as expected
• simplification of tmp files during downscaling. 85a4149
• added error message in case upper pressure level is not high enough

Bug fixes:

• to_netcdf() default case bug fixed
• plotting routines for variable mapping fixed
• fixed bug in time-batch handling of files
• bug in fetch_WMO() fixed

pypi install

This release includes a new options in setup.py that the package can be installed in a single line using Pypi

pip install topopyscale

Time batch debug

• Corrected bugs associated to merge of time_batch branch into main.
• Updated and tested install

Time batches and JOSS

This version brings the possibility to run downscaling in batch of time. This improves the workflow when running long time series for large domain. It also includes the latest JOSS manuscript