Test sepal_ui tools with latest sepal environment

[dfguerrerom]

With the latest sepal Ubuntu release, we'll have to make sure all our apps are working properly in the test server.
I'm creating this issue to track the progress of this process, and it's open to anyone who wants to help on the testing.

The steps /objectives are:

SEPAL_UI

- `git clone [email protected]:12rambau/sepal_ui.git`
- `cd sepal_ui`
- `git checkout module_venv`  # this branch contains the module_venv bin file to mimic the way as sepal does the venvs
- `pip install -e . --config-settings editable_mode=strict`  **

** I'm still not knowing why it doesn't work with only pip install -e . as on my local installation... there must be something that doesn't allow this behavior... after losing too many time I could make it work with the extra args, but not the "compat" previous one...
It happens that even though the editable .pth is located in .local/lib/python3.10/site-packages it will still preferring the not editable... I think this deserves to open an issue in setuptools because I couldn't go to the root of the problem.

Repository to test

• git clone $repository_to_test
• cd $folder_to_respository_to_test

edit requirements.txt file

• remove gdal and pyproj, these packages are installed by the module_venv script and cannot be downgraded to an older version.

Create local module virtual environment and test the app

• module_venv # This bin file will create the virtual environment with the current

• Test that the environment was successfully created and fix any issues, I know for some of them we'll have issues... specially for the packages that requires gdal, or scikit-learn, numpy or so pinned versions.

• [email protected]:sepal-contrib/alert_module.git

• [email protected]:sepal-contrib/alos_mosaics.git

• [email protected]:sepal-contrib/basin-rivers.git

• [email protected]:sepal-contrib/bfast-explorer.git

• [email protected]:sepal-contrib/bfastspatial.git

• [email protected]:sepal-contrib/bfast_gpu.git

• [email protected]:sepal-contrib/clip-time-series.git

• [email protected]:sepal-contrib/coverage_analysis.git

• [email protected]:sepal-contrib/cumsum_change.git

• [email protected]:sepal-contrib/damage_proxy_maps.git

• [email protected]:sepal-contrib/eSBAE_notebooks.git

• [email protected]:sepal-contrib/fcdm.git

• [email protected]:sepal-contrib/forest_at_risk.git

• [email protected]:sepal-contrib/geemap.git

• [email protected]:sepal-contrib/gee_source.git

• [email protected]:sepal-contrib/gfc_wrapper_python.git

• [email protected]:sepal-contrib/glad_alerts.git

• [email protected]:sepal-contrib/goe4lup.git

• [email protected]:sepal-contrib/gwb.git

• [email protected]:sepal-contrib/mspa-analysis.git

• [email protected]:sepal-contrib/planet-order.git

• [email protected]:sepal-contrib/planet_active_fires_explorer.git

• [email protected]:sepal-contrib/reclassify.git

• [email protected]:sepal-contrib/sdg_15.3.1.git

• [email protected]:sepal-contrib/se.plan.git

• [email protected]:sepal-contrib/sepal-geemap.git

• [email protected]:sepal-contrib/sepal-leafmap.git

• [email protected]:sepal-contrib/sepal-sbae.git

• [email protected]:sepal-contrib/sepal_mgci.git

[12rambau]

At this point don't you think it would make more sense to sit 2-3 days and set up some tests in the Jenkins pipeline ?

[dfguerrerom]

At this point don't you think it would make more sense to sit 2-3 days and set up some tests in the Jenkins pipeline ?

We definitely have to create some type of test to avoid doing this in the future, but I think the situation is a bit complex now because:

• we are using a newer version of GDAL which is incompatible with all the modules that had a GDAL pinned version in the past, we have to remove it from the GDAL requirements.
• we are forced to create all the venvs from scratch and some problems come up with this... i.e., a module that requires sepal_ui==2.6.4 will require install geemap, geemap has a earhengine-api pinned version >= to the one of the openforis fork and so, it will shadow it... meaning that we have to upgrade to a newer sepal_ui version that doesn't require geemap... doing this process require to test the whole tool and it's a bit slow.

[dfguerrerom]

Another issue, some modules required rasterio==1.1.5, that version is incompatible with GDAL==3.6.4, meaning that we have to manually change to a one that works and check that everything works.

[dfguerrerom]

@cdanielw, @lpaolini (and just for curiosity @12rambau )
We should find a way to make the apps fully isolated in their own environment, using their own C Libraries, so we won't end in situations like this.

• conda?
• docker?
  I think docker will take so long and it's not an option, but if I'm not wrong, conda can manage a separate copy of the GDAL library for example, but we would need to investigate the pros/cons