Davai WW#3

[AlexandreMary]

Addressed issues/improvements:

• eccodes paths to be removed and positionned by vortex according to the executable #35 (Thibault) => Solved (update in Vortex)
• Experts for ModelTLAD and EnsembleRead miss stdeo files at ECMWF #10 (Thibault) => Solved (update in Vortex)
• Issue with GMKFILE and OMPIIFC2302DP.AA (Chris/Daan) => workedaround, to be reported to Ryad for fixing: gmkpack does not find the root pack if gmkfile does not exist at Ryad's (gmk_support) and variable GMKFILE is not properly set (GMK_SUPPORT seems to not be fully implemented yet)
• Compilation with cmake/hirlam #41 (Roel) - initially proposed steps:
  • first step should be to support compiling with cmake from HarmonieCSC within IAL-build
  • then in a second step setup a usecase where starting from HarmonieCSC, compiling with cmake/hirlam, and run a subset of tests
  • Conclusion of the week:
    • using cmake/hirlam to compile IAL binaries require rearranging of cmake/hirlam, for now too bound to teh HarmonieCSC structure
    • we could have a separate usecase of Davai, starting from a HarmonieCSC git ref, then running with it a subset of tasks. However this requires the user to have access to Hirlam repositories. And the cmake/hirlam generated binaries require specific environments to be ran (otherwise some linked libraries are missing), which for now prevents from running Davai tests with them.
• Dissociate git2pack & pack2bin into 2 jobs #31 (Alexandre) => done, requires update of vortex (deployed in vortex-olive-dev) and DAVAI-env (in branch develop)
• Make davai test suite faster #34 (Alexandre) => merged in DV50T1_dev
• PPF jobs : consistent orography between PGD and atmosphere #12 (Daan): extended with C923-step1 / WIP
• Harmonie-Arome test #40 (Daniel/Chris): WIP, PR to come together with a bunch of changes in IAL
• Alaro[0|1] Single Precision crashes (Olda): a set of fixes from LACE (?) has been tested in Davai and it indeed fixes ALARO configurations (but Alaro1+SFX/SP) => to be commited to CY50T1
• Proposal of a CY50T0 gathering Alaro/SP+HarmonieArome fixes so as to have an extended set of tests for testing CY50T1 contributions
• Packaging DAVAI-env and DAVAI-tests + making a Davai experiment a virtualenv => work started by Thibault, cf. Packages git-cloned or in virtualenv instead of links #37 and Make DAVAI-env pip installable DAVAI-env#11
• 2 minor bugs were found in IAL-build and IAL-expertise, fixed in respective develop branches

Discussion on namelists and other "static data" in a Git repo + Git provider in Vortex:

• By "static data" we mean data that is not "flow data", i.e.:
  • flow data: produced by one task to be used by the next (such as historic files, analysis, etc...)
  • static data: namelists, PGD or clim files, RRTM/RTTOV coefficients, etc...
• MF wants to replace their current genv/gget/uenv/uget tools for handling the static data, for various reasons including ergonomy, maintenance, version control
• one idea to replace that is to use git repository(ies) in which this data is stored and version controled, and therefore git commands to retrieve these data in the first step (fetching inoput data) of a task
• for static data such as namelists or other (small) text files, this is standard, but for big binary data (such as PGD files) the idea is to use git-annex to store the data not on the git server (e.g. github) but on specific remote spaces (e.g. hendrix.meteo.fr, ecfs, cloud service...)
• a prototype "gitenv-demo" has been made by Thibault, containing a raw extract of an arpege "genv" environment, to show an example of how git and git-annex coexist and examples of version controling
• the architecture of such a git repo handling these static data remains to be figured out
• the sharedness of such a repo throughout ACCORD is to be figured out, but different such repositories could coexist side by side - though they would benefit from having the same internal architecture, so as to be able to switch easily from one to another
• A first step would be to use the Git repo and Git provider implemented in Vortex in Davai and then get feedback on its usage

Presentation of IAL-bundle and IAL-build bundling tools

• => need for more documentation, started on IAL-build's README.md
• ...

Discussion on automated testing

• Interest in using automated run of Davai through "github actions" so as to:
  • make more visible the validation of PRs
  • ensure testing is not skipped
• But should be seen as complementary to manual launching of Davai, necessary during development phase