Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jupyter notebook support #574

Merged
merged 51 commits into from
Feb 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
4b114e1
chore: Make start-persistent-session method tool specific
amolenaar Jan 31, 2023
17c621d
fix: Set initial session state in create_session function
amolenaar Feb 2, 2023
da422ce
feat: Deploy Jupyter images
amolenaar Feb 2, 2023
5e3d7a9
feat: deploy an ingress in front of the jupyter container
amolenaar Feb 3, 2023
0fe6b99
fix: Remove removed property also from frontend schema
amolenaar Feb 3, 2023
91eb661
fix: typo in CONTRIBUTING
amolenaar Feb 3, 2023
17bd313
feat: Launch jupyter notebook alongside Capella image
amolenaar Feb 3, 2023
66a8728
fix: Make Guacamole creds optional
amolenaar Feb 6, 2023
18faefc
fix: Use a map for ports, so we can properly name them
amolenaar Feb 7, 2023
064a4d3
fix: revert parameter order change
amolenaar Feb 7, 2023
5267b3d
fix: move jupyter startup to a script
amolenaar Feb 9, 2023
39f2d39
fix: Capella sessions work again
amolenaar Feb 9, 2023
49d9ea5
feat: Jupyter notebooks are now an "integration"
amolenaar Feb 9, 2023
4a49682
feat: build jupyter image by default
amolenaar Feb 9, 2023
2603d5b
feat: Add OpenShift route support
amolenaar Feb 9, 2023
63e19d9
fix: host for jupyter session ingress/route
amolenaar Feb 9, 2023
f500099
fix: Be restrictive on what we install in the Jupyter image
amolenaar Feb 10, 2023
244d4ff
fix: export telemetry for jupyter container to Prometheus
amolenaar Feb 10, 2023
9eb4dc8
fix: set lower resource limits for jupyter notebook
amolenaar Feb 10, 2023
a0f2174
fix: Make readonly and backup images not required
amolenaar Feb 10, 2023
da10213
fix: delete openshift route on session shutdown
amolenaar Feb 13, 2023
2cdf338
fix: remove file server port for capella instances
amolenaar Feb 13, 2023
3145a61
Only set persistent docker image for jupyter
amolenaar Feb 14, 2023
2101ec7
fix: Build jupyter by default
amolenaar Feb 14, 2023
1824475
fix: proper name for migration
amolenaar Feb 14, 2023
56690c1
fix: move session state fields back to route
amolenaar Feb 14, 2023
1120adb
fix: service account for openshift
amolenaar Feb 14, 2023
d5bd6e7
fix: Build jupyter image in capella-dockerimages
amolenaar Feb 14, 2023
9e258db
fix: Alembic history is linear again
amolenaar Feb 17, 2023
e8c4df1
fix: No need to cd in backend
amolenaar Feb 17, 2023
ba54f85
fix: forgot to remove old resource authorization
amolenaar Feb 17, 2023
9b94119
chore: add type annotation for jupyter environment vars
amolenaar Feb 17, 2023
5a0ab6f
chore: only delete ingress if not on openshift
amolenaar Feb 17, 2023
3cac9c9
fix: be more defensive with assigned ports
amolenaar Feb 17, 2023
1c3e091
chore: Move networking api to the constructor
amolenaar Feb 17, 2023
b96b78f
chore: import opensift modules, instead of classes
amolenaar Feb 17, 2023
4c703cc
fix: Change syntax to create/delete routes
amolenaar Feb 17, 2023
e866d0e
fix: Do assign API group to service account
amolenaar Feb 17, 2023
f8cec8b
chore: Tinker with openshift route
amolenaar Feb 17, 2023
e731389
fix: use a special domain name for jupyter sessions on openshift
amolenaar Feb 17, 2023
f3b7c55
feat: allow sessions to be served from a different host name
amolenaar Feb 20, 2023
cd8d39b
fix: Use extension config for Jupyter
amolenaar Feb 20, 2023
69a3382
fix: Add custom host permission to service account
amolenaar Feb 20, 2023
afdcbd0
fix: sync with latest dockerimages changes
amolenaar Feb 21, 2023
0dc223f
feat: all session should be able to provide setup states
amolenaar Feb 21, 2023
7851398
fix: readonly session state should be returned
amolenaar Feb 21, 2023
a3d2ae2
chore: refactor, clean up session creation code
amolenaar Feb 22, 2023
2e04c78
fix: update docs for jupyter
amolenaar Feb 22, 2023
f46348c
fix: Update T4C client version
amolenaar Feb 24, 2023
9a7bdbf
fix: remove ssl-redirect annotation
amolenaar Feb 24, 2023
505d235
chore: Properly separate Guacamole and Jupyter sessions
amolenaar Feb 24, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# SPDX-FileCopyrightText: Copyright DB Netz AG and the capella-collab-manager contributors
# SPDX-License-Identifier: CC0-1.0

logs/
.venv*
/.provision-backend
/.provision-guacamole
Expand Down
1 change: 0 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,6 @@ To create an upgrade script automatically (this will compare the current databas
with the models):

```sh
cd capellacollab
alembic revision --autogenerate -m "Commit message"
```

Expand Down
9 changes: 6 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ SESSION_NAMESPACE = collab-sessions
PORT ?= 8080

# List of Capella versions, e.g.: `5.0.0 5.2.0 6.0.0`
CAPELLA_VERSIONS ?= 5.2.0
CAPELLA_VERSIONS ?= 6.0.0
# List of T4C versions, e.g., `5.2.0 6.0.0`
T4C_CLIENT_VERSIONS ?= 5.2.0
T4C_CLIENT_VERSIONS ?= 6.0.0

TIMEOUT ?= 10m

Expand All @@ -29,7 +29,7 @@ export DOCKER_BUILDKIT=1
SHELL = /bin/bash
.SHELLFLAGS = -euo pipefail -c

build: backend frontend docs guacamole
build: backend frontend docs guacamole jupyter

backend: IMAGE=capella/collab/backend
backend:
Expand All @@ -54,6 +54,9 @@ capella:
t4c-client:
$(CAPELLA_DOCKERIMAGES) CAPELLA_VERSIONS="$(T4C_CLIENT_VERSIONS)" t4c/client/remote t4c/client/backup

jupyter:
$(CAPELLA_DOCKERIMAGES) jupyter-notebook

docs:
docker build -t capella/collab/docs -t $(LOCAL_REGISTRY_NAME):$(REGISTRY_PORT)/capella/collab/docs docs/user
docker push $(LOCAL_REGISTRY_NAME):$(REGISTRY_PORT)/capella/collab/docs
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# SPDX-FileCopyrightText: Copyright DB Netz AG and the capella-collab-manager contributors
dominik003 marked this conversation as resolved.
Show resolved Hide resolved
# SPDX-License-Identifier: Apache-2.0

"""Define Jupyter as integration

Revision ID: 61d36288afe9
Revises: f3d2dedd7906
Create Date: 2023-02-09 11:57:07.345877

"""
import sqlalchemy as sa
from alembic import op

# revision identifiers, used by Alembic.
revision = "61d36288afe9"
down_revision = "f3d2dedd7906"
branch_labels = None
depends_on = None


def upgrade():
op.add_column(
"tool_integrations",
sa.Column(
"jupyter", sa.Boolean(), nullable=False, server_default="false"
),
)


def downgrade():
op.drop_column("tool_integrations", "jupyter")
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# SPDX-FileCopyrightText: Copyright DB Netz AG and the capella-collab-manager contributors
# SPDX-License-Identifier: Apache-2.0

"""Commit message

Revision ID: f3d2dedd7906
Revises: 4df9c82766e2
Create Date: 2023-02-03 14:31:55.776520

"""
import sqlalchemy as sa
from alembic import op

# revision identifiers, used by Alembic.
revision = "f3d2dedd7906"
down_revision = "f7c1a89af5d7"
branch_labels = None
depends_on = None


def upgrade():
op.add_column(
"sessions", sa.Column("jupyter_token", sa.String(), nullable=True)
)


def downgrade():
op.drop_column("sessions", "jupyter_token")
8 changes: 8 additions & 0 deletions backend/capellacollab/config/config_schema.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,14 @@ properties:
type: string
password:
type: string
jupyter:
type: object
additionalProperties: false
required:
- publicURI
properties:
publicURI:
type: string
authentication:
type: object
required:
Expand Down
14 changes: 14 additions & 0 deletions backend/capellacollab/core/database/migration.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@
)
from capellacollab.tools import crud as tools_crud
from capellacollab.tools import models as tools_models
from capellacollab.tools.integrations import crud as integrations_crud
from capellacollab.tools.integrations import models as integrations_models
from capellacollab.users import crud as users_crud
from capellacollab.users import models as users_models
from capellacollab.users.events import crud as events_crud
Expand Down Expand Up @@ -109,6 +111,16 @@ def create_tools(db):
)
tools_crud.create_tool(db, capella)
tools_crud.create_tool(db, papyrus)
jupyter = tools_models.Tool(
name="Jupyter",
docker_image_template=f"{registry}/jupyter-notebook:$version",
)
tools_crud.create_tool(db, jupyter)
integrations_crud.update_integrations(
db,
jupyter.integrations,
integrations_models.PatchToolIntegrations(jupyter=True),
)

default_version = tools_crud.create_version(db, capella.id, "6.0.0", True)
tools_crud.create_version(db, capella.id, "5.2.0")
Expand All @@ -117,6 +129,8 @@ def create_tools(db):
tools_crud.create_version(db, papyrus.id, "6.1")
tools_crud.create_version(db, papyrus.id, "6.0")

tools_crud.create_version(db, jupyter.id, "python-3.11")

default_nature = tools_crud.create_nature(db, capella.id, "model")
tools_crud.create_nature(db, capella.id, "library")

Expand Down
1 change: 1 addition & 0 deletions backend/capellacollab/sessions/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ class DatabaseSession(Base):
guacamole_username: str = sa.Column(sa.String)
guacamole_password: str = sa.Column(sa.String)
guacamole_connection_id: str = sa.Column(sa.String)
jupyter_token: str = sa.Column(sa.String)
host: str = sa.Column(sa.String)
type: WorkspaceType = sa.Column(sa.Enum(WorkspaceType), nullable=False)
project_id: str = sa.Column(
Expand Down
Loading