chore: migrate OC Python projects to uv (SMR-8)

[tschaffter]

Contributes to https://sagebionetworks.jira.com/browse/SMR-8

Description

This PR migrates two OC Python project to the package manager uv.

The projects are:

• openchallenges-data-lambda
• openchallenges-etl-data

Changelog

• Update the Nx plugin for Python and set uv as its preferred package manager.
• Set Python version of openchallenges-data-lambda to 3.13.1.
• Set Python version of openchallenges-edam-etl to 3.12.8.
• Implement openchallenges-data-lambda:serve.
• Update the Dockerfile of openchallenges-data-lambda to use uv.
• Update the Dockerfile of openchallenges-edam-etl to use uv and multi-stage build.
• Add .pdm-build to workspace-level .gitignore.
• Remove more folders with workspace-nuke.

Notes

The Dockerfile of the lambda project was adapted from this example provided by uv.

Preview

Install with uv (18 seconds, 12x faster than current setup)

$ time nx prepare openchallenges-data-lambda

> nx run openchallenges-data-lambda:prepare

> uv sync

Using CPython 3.13.1
Creating virtual environment at: .venv
Resolved 21 packages in 5ms
   Built openchallenges-data-lambda @ file:///workspaces/sage-monorepo/apps/openchallenges/data-lambda
Prepared 21 packages in 2.61s
░░░░░░░░░░░░░░░░░░░░ [0/21] Installing wheels...                                                                                                                                                                                     warning: Failed to hardlink files; falling back to full copy. This may lead to degraded performance.
         If the cache and target directories are on different filesystems, hardlinking may not be supported.
         If this is intentional, set `export UV_LINK_MODE=copy` or use `--link-mode=copy` to suppress this warning.
Installed 21 packages in 491ms
 + cachetools==5.5.1
 + certifi==2024.12.14
 + charset-normalizer==3.4.1
 + google-auth==2.38.0
 + google-auth-oauthlib==1.2.1
 + gspread==6.1.4
 + idna==3.10
 + numpy==2.1.0
 + oauthlib==3.2.2
 + openchallenges-data-lambda==0.1.0 (from file:///workspaces/sage-monorepo/apps/openchallenges/data-lambda)
 + pandas==2.2.3
 + pyasn1==0.6.1
 + pyasn1-modules==0.4.1
 + python-dateutil==2.9.0.post0
 + pytz==2024.2
 + requests==2.32.3
 + requests-oauthlib==2.0.0
 + rsa==4.9
 + six==1.17.0
 + tzdata==2025.1
 + urllib3==2.3.0

————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 NX   Successfully ran target prepare for project openchallenges-data-lambda (12s)


real    0m18.362s
user    0m6.117s
sys     0m7.386s

Installation with Pyenv and Poetry (3m40s)

$ time nx prepare openchallenges-data-lambda

> nx run openchallenges-data-lambda:prepare

> ./install.sh

Downloading Python-3.11.10.tar.xz...
-> https://www.python.org/ftp/python/3.11.10/Python-3.11.10.tar.xz
Installing Python-3.11.10...
Installed Python-3.11.10 to /home/ubuntu/.pyenv/versions/3.11.10
Creating virtualenv openchallenges-data-lambda-hLSYwqCz-py3.11 in /home/ubuntu/.cache/pypoetry/virtualenvs
Using virtualenv: /home/ubuntu/.cache/pypoetry/virtualenvs/openchallenges-data-lambda-hLSYwqCz-py3.11
Installing dependencies from lock file

Package operations: 20 installs, 0 updates, 0 removals

  - Installing certifi (2024.12.14)
  - Installing charset-normalizer (3.4.1)
  - Installing idna (3.10)
  - Installing pyasn1 (0.6.1)
  - Installing urllib3 (2.3.0)
  - Installing cachetools (5.5.1)
  - Installing oauthlib (3.2.2)
  - Installing pyasn1-modules (0.4.1)
  - Installing requests (2.32.3)
  - Installing rsa (4.9)
  - Installing google-auth (2.38.0)
  - Installing requests-oauthlib (2.0.0)
  - Installing six (1.17.0)
  - Installing google-auth-oauthlib (1.2.1)
  - Installing numpy (2.1.0)
  - Installing python-dateutil (2.9.0.post0)
  - Installing pytz (2024.2)
  - Installing tzdata (2025.1)
  - Installing gspread (6.1.4)
  - Installing pandas (2.2.3)

Installing the current project: openchallenges-data-lambda (0.1.0)

———————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 NX   Successfully ran target prepare for project openchallenges-data-lambda (4m)


real    3m40.618s
user    7m57.510s
sys     1m16.697s

How to test

1. Checkout this feature branch.
2. Remove .venv and other untracked folders from apps/openchallenges/data-lambda.
3. Run nx prepare openchallenges-data-lambda from the workspace directory.
4. Run the commands listed in the apps/openchallenges/data-lambda/README.md to verify that the lambda function works properly.

[vpchung]

Works great!

 $ nx prepare openchallenges-data-lambda

> nx run openchallenges-data-lambda:prepare

> uv sync

...
... [truncated]

————————————————————————

 NX   Successfully ran target prepare for project openchallenges-data-lambda (11s)

No external changes found 👍🏼 :

$ nx run openchallenges-data-lambda:invoke --event events/event.json

> nx run openchallenges-data-lambda:invoke --event events/event.json

> curl -X POST 'http://localhost:9000/2015-03-31/functions/function/invocations' --data @events/event.json

{"statusCode": 200, "body": "{\"message\": \"Data successfully pulled from OC Data google sheet.\"}"}
————————————————————————

 NX   Successfully ran target invoke for project openchallenges-data-lambda (5s)

      With additional flags:
        --event=events/event.json