Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Containerization of MDTF #486

Open
aradhakrishnanGFDL opened this issue Sep 22, 2023 · 2 comments · May be fixed by #503
Open

Containerization of MDTF #486

aradhakrishnanGFDL opened this issue Sep 22, 2023 · 2 comments · May be fixed by #503
Assignees
Labels
framework Issue pertains to the framework code tool Scripts or features that assist with data or code management that are not part of the framework src

Comments

@aradhakrishnanGFDL
Copy link
Collaborator

**One minute to MDTF: usability across all platforms (sorta) **

**Issue to track the container readiness (build as part of CI, run specific tests in CI and from two other platforms) and to record challenges, issues, future directions **

  • Scope of the first deliverable is limited to seeing how an example POD can be run using an MDTF container, and see how it may be built as part of an automated pipeline.

** A developer/contributor mode and an explorer mode from a containerized build may need to come with some best practices: document things learnt**

@aradhakrishnanGFDL aradhakrishnanGFDL self-assigned this Sep 22, 2023
@wrongkindofdoctor wrongkindofdoctor added framework Issue pertains to the framework code tool Scripts or features that assist with data or code management that are not part of the framework src labels Sep 25, 2023
@aradhakrishnanGFDL
Copy link
Collaborator Author

aradhakrishnanGFDL commented Oct 5, 2023

Use my fork and the main branch for testing:

https://github.com/aradhakrishnanGFDL/MDTF-diagnostics

HOW TO TEST THE MDTF CONTAINER

  1. Install Docker desktop if you don’t have docker already.

  2. From your terminal, do the following:

docker pull public.ecr.aws/c6h4h4s8/mdtf/mdtf-test:latest
The above will pull the latest docker image built in the GitHub actions workflow associated with this container22 branch.

  1. If you haven't already, clone the MDTF github repo. The first cut of the docker image does not include the diagnostics to allow one to dev and contribute to github more swiftly. We may include it later.

  2. You likely have your MDTF sample inputdata in your environment. We will simply point to that and the docker light-weight for this release.
    If you don't have sample input datasets, please follow this documentation here
    general organization of inputdata: https://mdtf-diagnostics.readthedocs.io/en/main/sphinx/start_config.html

sample datasets from an FTP server: https://mdtf-diagnostics.readthedocs.io/en/main/sphinx/start_install.html#obtaining-supporting-data

  1. Ensure inputdata is present at some_path/inputdata. We will use this in the next step

  2. Type this from the terminal, refer 7 and then run with suggested changes.

docker run -it -v /Users/ar46/mdtf/MDTF-diagnostics/diagnostics/:/proj/MDTF-diagnostics/diagnostics/ -v /Users/ar46/mdtf/inputdata/:/proj/inputdata/ -v /Users/ar46/mdtf-dock/wkdir/:/proj/wkdir/ -p 8080:80 public.ecr.aws/c6h4h4s8/mdtf/mdtf-test where,

  1. Replace /Users/ar46/mdtf in /Users/ar46/mdtf/inputdata and /Users/ar46/mdtf/MDTF-diagnostics/diagnostics/ with your path to the inputdata and the diagnostics directory (from the cloned repo)
    Replace /Users/ar46/mdtf-dock/wkdir/ by a working directory in your host machine.
    Replace
    We are mounting volumes to the docker container such that they are available for our MDTF runs.

  2. From the docker container that is now in the "run" mode from the previous step, you may run MDTF!

micromamba activate _MDTF_base
mdtf_framework.py -f /proj/inputdata/tests/docker_test_set1.jsonc
  1. Where do I see the results?
    From your localhost, just open the HTML file from your work directory.
    In this example, I open the following from my web browser:
    /Users/ar46/mdtf-dock/wkdir/MDTF_GFDL.v2.Synthetic_1_10/index.html

@mengxi16 mengxi16 mentioned this issue Oct 9, 2023
3 tasks
@aradhakrishnanGFDL
Copy link
Collaborator Author

including @jkrasting, I have done limited testing on my mac. Someone else other than me testing and improving the instructions would be helpful.

@aradhakrishnanGFDL aradhakrishnanGFDL linked a pull request Jan 25, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
framework Issue pertains to the framework code tool Scripts or features that assist with data or code management that are not part of the framework src
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants