diff --git a/CHANGELOG.md b/CHANGELOG.md index eb3620dc..a9e87a92 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,7 +14,27 @@ Please refer to the respective repositories for a more in depth changelog of sin |Backend|| |DSF Feasibility Plugin|| |FLARE|| -|Blaze FHIR server|| +|TORCH|| + + +## [5.0.0-alpha] - 2024-10-21 + +### Features + +| Feature | Affected Components | +| -- | -- | +|UI Re-Desig, Restructuring of Code|UI, Backend| +|Extended Criteria Search (Elastic Search)|UI, Backend, Ontology Generation| +|Add OAuth2 to triangle components|TORCH, FLARE| +|Added Dataselection and Extraction |UI, Backend, Ontology Generation, TORCH| +|Migrated from Mapping code system tree strcture to poly tree structure to support non strict hierarchical code systems like sct |UI, Backend, Ontology Generation, TORCH, FLARE| +|Loading and displaying of criteria availability |UI, Backend, Ontology Generation| + +### Overall + +- Updated all components to new versions +- Added TORCH component for data selection and extraction in the triangle ## [4.1.0] - 2024-07-16 diff --git a/README.md b/README.md index 637626cd..20efe81d 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Feasibility Deploy -[![version](https://img.shields.io/badge/version-4.0.0-green.svg)](https://github.com/medizininformatik-initiative/feasibility-deploy/releases) +[![version](https://img.shields.io/badge/version-5.0.0-green.svg)](https://github.com/medizininformatik-initiative/feasibility-deploy/releases) This feasibility deployment repository offers an example deployment repository using docker-compose and official images to set up a feasibility portal (central) as well as feasibility triangle (decentral - at site) diff --git a/feasibility-portal/README.md b/feasibility-portal/README.md index dd9e0e15..53c200ba 100644 --- a/feasibility-portal/README.md +++ b/feasibility-portal/README.md @@ -66,6 +66,39 @@ The portal is configured by default to start the following services: - UI - Keycloak +For the reverse proxy you need to choose the configuration (variable `FEASIBILITY_PORTAL_PROXY_NGINX_CONFIG` in +[proxy/.env](./proxy/.env)) which also decides what the changes to the `.env` files you have to make: + +- [./subdomains.nginx.conf](./proxy/subdomains.nginx.conf) with separate domains for the services (Backend, UI, Keycloak) + - All subdomains must point to the host machine the portal will run. + + - Set the service hostnames (`BACKEND_HOSTNAME`, `KEYCLOAK_HOSTNAME` and `GUI_HOSTNAME`, depending on which services you need) in [proxy/.env](./proxy/.env). +- Change the following variables in [keycloak/.env](./keycloak /.env): + - `FEASIBILITY_KC_HOSTNAME_URL`and `FEASIBILITY_KC_HOSTNAME_ADMIN_URL`: set the domain part to the value you set for `KEYCLOAK_HOSTNAME` before. + -` FEASIBILITY_KC_HTTP_RELATIVE_PATH`: set to `/auth`. +- Change the values for the variables `FEASIBILITY_BACKEND_API_BASE_URL` in [backend/.env](./backend/.env) and `FEASIBILITY_BACKEND_ALLOWED_ORIGINS` in [backend /.env](./backend/.env) + to the base url of your feasibility portal backend. In the [backend/.env](./backend/.env) change the values for the variable `FEASIBILITY_BACKEND_KEYCLOAK_BASE_URL_ISSUER` to the base url of your feasibility portal keycloak. +- Change the following variables in [gui/deploy-config.json](./gui/deploy-config.json): + - `uiBackendApi > baseUrl`: set the domain part of the local feasibility portal backend. + - `auth > baseUrl`: set the domain part of the local feasibility portal keycloak. +- On the [proxy/.env] use this variable `FEASIBILITY_PORTAL_PROXY_NGINX_CONFIG=./subdomains.nginx.conf`. + +- [./context-paths.nginx.conf](./proxy/context-paths.nginx.conf) which requires only one domain and uses context paths (`/auth` for keycloak,`/api` for backend and `/`) for user interface. +- The domain must point to the host machine the portal will run. +- On the [proxy/.env] use this variable`FEASIBILITY_PORTAL_PROXY_NGINX_CONFIG=./context-paths.nginx.conf` +- Change the following variable `FEASIBILITY_KC_HOSTNAME_URL` and `FEASIBILITY_KC_HOSTNAME_ADMIN_URL` in [keycloak/.env]: set the domain part of your domain. The path must be set to /auth at the end of the url. For example, https://example.org/auth. +- Add `/auth` in the following variable `FEASIBILITY_KC_HTTP_RELATIVE_PATH` in [keycloak/.env] +- Change the following variable `FEASIBILITY_BACKEND_API_BASE_URL` in [backend/.env]: set the domain part of your domain. The path must be set to /api at the end of the url. For example, https://example.org/api. +- Change the following variable `FEASIBILITY_BACKEND_ALLOWED_ORIGINS` in [backend/.env]: set the domain part of your domain. For example, https://example.org. +- Change the following variable`FEASIBILITY_BACKEND_KEYCLOAK_BASE_URL_ISSUER` in [backend/.env]: set the domain part of your domain. The path must be set to /api at the end of the url. For example, https://example.org/auth. +- Add `/auth` in the following variable `FEASIBILITY_BACKEND_KEYCLOAK_BASE_URL_JWK` in [backend/.env] +- Change the variable `FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_AUTH_OAUTH_ISSUER_URL` when using the bundled keycloak in [backend/.env]replace the values with https://DOMAIN:REV_PROXY_PORT/auth/realms/blaze where DOMAIN is your domain and REV_PROXY_PORT is the port number set in rev-proxy/.env (default 444). For example, https://example.org:444/auth/realms/blaze. +- On the [gui/deploy-config.json] change the following variables: + - `uiBackendApi > baseUrl`: set the domain part of the local feasibility portal backend with the context path `/api`. For example https://example.org/api. + - `auth > baseUrl`: set the domain part of the local feasibility portal keycloak the context path `/auth`. For example https://example.org/auth. + +Please note that the keycloak provided here is an example setup, and we strongly recommend for each site to adjust the keycloak installation to their local security requirements or connect the local feasibility portal to a keycloak already provided at the site. + For more details on the environment variables see the paragraph **Configurable environment variables** of this README. ### Step 6 - Start the feasibility portal diff --git a/feasibility-portal/backend/.env.default b/feasibility-portal/backend/.env.default index 50d9d9ff..6766d42c 100644 --- a/feasibility-portal/backend/.env.default +++ b/feasibility-portal/backend/.env.default @@ -1,56 +1,57 @@ # ----- app -FEASIBILITY_BACKEND_CQL_TRANSLATE_ENABLED=true -FEASIBILITY_BACKEND_FHIR_TRANSLATE_ENABLED=false -FEASIBILITY_BACKEND_API_BASE_URL=https://api.datenportal.localhost -FEASIBILITY_BACKEND_ALLOWED_ORIGINS=https://datenportal.localhost -FEASIBILITY_BACKEND_ONTOLOGY_ORDER="Diagnose, Prozedur, Person, Laboruntersuchung, Medikamentenverabreichung, Bioprobe, Einwilligung" -FEASIBILITY_BACKEND_MAX_SAVED_QUERIES_PER_USER=100 +DATAPORTAL_BACKEND_CQL_TRANSLATE_ENABLED=true +DATAPORTAL_BACKEND_FHIR_TRANSLATE_ENABLED=false +DATAPORTAL_BACKEND_API_BASE_URL=https://api.datenportal.localhost +DATAPORTAL_BACKEND_ALLOWED_ORIGINS=https://datenportal.localhost +DATAPORTAL_BACKEND_ONTOLOGY_ORDER="Diagnose, Prozedur, Person, Laboruntersuchung, Medikamentenverabreichung, Bioprobe, Einwilligung" +DATAPORTAL_BACKEND_MAX_SAVED_QUERIES_PER_USER=100 # ---- db config -FEASIBILITY_BACKEND_DATASOURCE_HOST=feasibility-gui-backend-db -FEASIBILITY_BACKEND_DATASOURCE_PORT=5432 -FEASIBILITY_BACKEND_DATASOURCE_USERNAME=guidbuser -FEASIBILITY_BACKEND_DATASOURCE_PASSWORD=guidbpw +DATAPORTAL_BACKEND_DATASOURCE_HOST=dataportal-postgres +DATAPORTAL_BACKEND_DATASOURCE_PORT=5432 +DATAPORTAL_BACKEND_DATASOURCE_USERNAME=guidbuser +DATAPORTAL_BACKEND_DATASOURCE_PASSWORD=guidbpw # ---- auth -FEASIBILITY_BACKEND_KEYCLOAK_ENABLED=true -FEASIBILITY_BACKEND_KEYCLOAK_ALLOWED_ROLE=FeasibilityUser -FEASIBILITY_BACKEND_KEYCLOAK_POWER_ROLE=FeasibilityPowerUser -FEASIBILITY_BACKEND_KEYCLOAK_ADMIN_ROLE=FeasibilityAdmin -FEASIBILITY_BACKEND_KEYCLOAK_BASE_URL_ISSUER=https://auth.datenportal.localhost -FEASIBILITY_BACKEND_KEYCLOAK_BASE_URL_JWK=http://auth:8080 -FEASIBILITY_BACKEND_KEYCLOAK_REALM=feasibility +DATAPORTAL_BACKEND_KEYCLOAK_ENABLED=true +DATAPORTAL_BACKEND_KEYCLOAK_ALLOWED_ROLE=FeasibilityUser +DATAPORTAL_BACKEND_KEYCLOAK_POWER_ROLE=FeasibilityPowerUser +DATAPORTAL_BACKEND_KEYCLOAK_ADMIN_ROLE=FeasibilityAdmin +DATAPORTAL_BACKEND_KEYCLOAK_BASE_URL_ISSUER=https://auth.datenportal.localhost +DATAPORTAL_BACKEND_KEYCLOAK_BASE_URL_JWK=http://auth:8080 +DATAPORTAL_BACKEND_KEYCLOAK_REALM=feasibility #---- Direct broker -FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_ENABLED=true -FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_USE_CQL=false -FEASIBILITY_BACKEND_BROKER_CLIENT_OBFUSCATE_RESULT_COUNT=false -FEASIBILITY_BACKEND_FLARE_WEBSERVICE_BASE_URL=http://flare:8080 -FEASIBILITY_BACKEND_CQL_SERVER_BASE_URL=http://fhir-server:8080/fhir -FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_AUTH_BASIC_USERNAME= -FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_AUTH_BASIC_PASSWORD= -FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_AUTH_OAUTH_ISSUER_URL=https://keycloak.localhost:444/realms/blaze -FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_AUTH_OAUTH_CLIENT_ID=account -FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_AUTH_OAUTH_CLIENT_SECRET=insecure +DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_ENABLED=true +DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_USE_CQL=false +DATAPORTAL_BACKEND_BROKER_CLIENT_OBFUSCATE_RESULT_COUNT=false +DATAPORTAL_BACKEND_FLARE_WEBSERVICE_BASE_URL=http://flare:8080 +DATAPORTAL_BACKEND_CQL_SERVER_BASE_URL=http://fhir-server:8080/fhir +DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_AUTH_BASIC_USERNAME= +DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_AUTH_BASIC_PASSWORD= +DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_AUTH_OAUTH_ISSUER_URL=https://keycloak.localhost:444/realms/blaze +DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_AUTH_OAUTH_CLIENT_ID=account +DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_AUTH_OAUTH_CLIENT_SECRET=insecure # ---- DSF broker -FEASIBILITY_BACKEND_DSF_ENABLED=false -FEASIBILITY_BACKEND_DSF_CACERT=/opt/codex-feasibility-security/ca.pem -FEASIBILITY_BACKEND_DSF_DSF_SECURITY_KEYSTORE_P12FILE=/opt/codex-feasibility-security/test-user.p12 -FEASIBILITY_BACKEND_DSF_SECURITY_KEYSTORE_PASSWORD=password -FEASIBILITY_BACKEND_DSF_WEBSERVICE_BASE_URL=https://dsf-zars-fhir-proxy/fhir -FEASIBILITY_BACKEND_DSF_WEBSOCKET_URL=wss://dsf-zars-fhir-proxy:443/fhir/ws -FEASIBILITY_BACKEND_DSF_ORGANIZATION_ID=Test_ZARS +DATAPORTAL_BACKEND_DSF_ENABLED=false +DATAPORTAL_BACKEND_DSF_CACERT=/opt/dataportal-security/ca.pem +DATAPORTAL_BACKEND_DSF_DSF_SECURITY_KEYSTORE_P12FILE=/opt/dataportal-security/test-user.p12 +DATAPORTAL_BACKEND_DSF_SECURITY_KEYSTORE_PASSWORD=password +DATAPORTAL_BACKEND_DSF_WEBSERVICE_BASE_URL=https://dsf-zars-fhir-proxy/fhir +DATAPORTAL_BACKEND_DSF_WEBSOCKET_URL=wss://dsf-zars-fhir-proxy:443/fhir/ws +DATAPORTAL_BACKEND_DSF_ORGANIZATION_ID=Test_ZARS # ---- privacy -FEASIBILITY_BACKEND_PRIVACY_QUOTA_SOFT_CREATE_AMOUNT=3 -FEASIBILITY_BACKEND_PRIVACY_QUOTA_SOFT_CREATE_INTERVALMINUTES=1 -FEASIBILITY_BACKEND_PRIVACY_QUOTA_HARD_CREATE_AMOUNT=50 -FEASIBILITY_BACKEND_PRIVACY_QUOTA_HARD_CREATE_INTERVALMINUTES=10080 -FEASIBILITY_BACKEND_PRIVACY_QUOTA_READ_SUMMARY_POLLINGINTERVALSECONDS=5 -FEASIBILITY_BACKEND_PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_POLLINGINTERVALSECONDS=10 -FEASIBILITY_BACKEND_PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_AMOUNT=10 -FEASIBILITY_BACKEND_PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_INTERVALSECONDS=7200 -FEASIBILITY_BACKEND_PRIVACY_THRESHOLD_RESULTS=0 -FEASIBILITY_BACKEND_PRIVACY_THRESHOLD_SITES=1 -FEASIBILITY_BACKEND_QUERYRESULT_EXPIRY_MINUTES=5 +DATAPORTAL_BACKEND_PRIVACY_QUOTA_SOFT_CREATE_AMOUNT=3 +DATAPORTAL_BACKEND_PRIVACY_QUOTA_SOFT_CREATE_INTERVALMINUTES=1 +DATAPORTAL_BACKEND_PRIVACY_QUOTA_HARD_CREATE_AMOUNT=50 +DATAPORTAL_BACKEND_PRIVACY_QUOTA_HARD_CREATE_INTERVALMINUTES=10080 +DATAPORTAL_BACKEND_PRIVACY_QUOTA_READ_SUMMARY_POLLINGINTERVALSECONDS=5 +DATAPORTAL_BACKEND_PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_POLLINGINTERVALSECONDS=10 +DATAPORTAL_BACKEND_PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_AMOUNT=10 +DATAPORTAL_BACKEND_PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_INTERVALSECONDS=7200 +DATAPORTAL_BACKEND_PRIVACY_THRESHOLD_RESULTS=0 +DATAPORTAL_BACKEND_PRIVACY_THRESHOLD_SITES=1 +DATAPORTAL_BACKEND_PRIVACY_THRESHOLD_SITES_RESULT=0 +DATAPORTAL_BACKEND_QUERYRESULT_EXPIRY_MINUTES=5 # ---- logging -FEASIBILITY_BACKEND_LOG_LEVEL_SQL=info -FEASIBILITY_BACKEND_LOG_LEVEL=info +DATAPORTAL_BACKEND_LOG_LEVEL_SQL=info +DATAPORTAL_BACKEND_LOG_LEVEL=info diff --git a/feasibility-portal/backend/docker-compose.yml b/feasibility-portal/backend/docker-compose.yml index 33c474c2..631cdcdc 100644 --- a/feasibility-portal/backend/docker-compose.yml +++ b/feasibility-portal/backend/docker-compose.yml @@ -1,95 +1,152 @@ services: - feasibility-gui-backend: - image: ghcr.io/medizininformatik-initiative/feasibility-backend:5.0.1 + dataportal-backend: + restart: unless-stopped + image: ghcr.io/medizininformatik-initiative/feasibility-backend:6.0.0-alpha.3 ports: - - ${FEASIBILITY_BACKEND_PORT:-127.0.0.1:8091}:8090 + - ${DATAPORTAL_BACKEND_PORT:-127.0.0.1:8091}:8090 depends_on: - - feasibility-gui-backend-db + dataportal-postgres: + condition: service_started + dataportal-elastic: + condition: service_healthy + init-elasticsearch: + condition: service_completed_successfully environment: # ----- app - QUERY_VALIDATION_ENABLED: ${FEASIBILITY_BACKEND_QUERY_VALIDATION_ENABLED:-true} - CQL_TRANSLATE_ENABLED: ${FEASIBILITY_BACKEND_CQL_TRANSLATE_ENABLED:-true} - FHIR_TRANSLATE_ENABLED: ${FEASIBILITY_BACKEND_FHIR_TRANSLATE_ENABLED:-false} - API_BASE_URL: ${FEASIBILITY_BACKEND_API_BASE_URL:-https://localhost/api/} - ALLOWED_ORIGINS: ${FEASIBILITY_BACKEND_ALLOWED_ORIGINS:-https://localhost} - QUERYRESULT_EXPIRY_MINUTES: ${FEASIBILITY_BACKEND_QUERYRESULT_EXPIRY_MINUTES:-5} - ONTOLOGY_ORDER: ${FEASIBILITY_BACKEND_ONTOLOGY_ORDER:-"Diagnose, Prozedur, Person, Laboruntersuchung, Medikamentenverabreichung, Bioprobe, Einwilligung"} - MAX_SAVED_QUERIES_PER_USER: ${FEASIBILITY_BACKEND_MAX_SAVED_QUERIES_PER_USER:-100} + QUERY_VALIDATION_ENABLED: ${DATAPORTAL_BACKEND_QUERY_VALIDATION_ENABLED:-true} + CQL_TRANSLATE_ENABLED: ${DATAPORTAL_BACKEND_CQL_TRANSLATE_ENABLED:-true} + FHIR_TRANSLATE_ENABLED: ${DATAPORTAL_BACKEND_FHIR_TRANSLATE_ENABLED:-false} + API_BASE_URL: ${DATAPORTAL_BACKEND_API_BASE_URL:-https://localhost/api/} + ALLOWED_ORIGINS: ${DATAPORTAL_BACKEND_ALLOWED_ORIGINS:-https://localhost} + QUERYRESULT_EXPIRY_MINUTES: ${DATAPORTAL_BACKEND_QUERYRESULT_EXPIRY_MINUTES:-5} + ONTOLOGY_ORDER: ${DATAPORTAL_BACKEND_ONTOLOGY_ORDER:-"Diagnose, Prozedur, Person, Laboruntersuchung, Medikamentenverabreichung, Bioprobe, Einwilligung"} + MAX_SAVED_QUERIES_PER_USER: ${DATAPORTAL_BACKEND_MAX_SAVED_QUERIES_PER_USER:-100} # ---- db config - FEASIBILITY_DATABASE_HOST: ${FEASIBILITY_BACKEND_DATASOURCE_HOST:-feasibility-gui-backend-db} - FEASIBILITY_DATABASE_PORT: ${FEASIBILITY_BACKEND_DATASOURCE_PORT:-5432} - FEASIBILITY_DATABASE_USER: ${FEASIBILITY_BACKEND_DATASOURCE_USERNAME:-guidbuser} - FEASIBILITY_DATABASE_PASSWORD: ${FEASIBILITY_BACKEND_DATASOURCE_PASSWORD:-guidbpw} + DATABASE_HOST: ${DATAPORTAL_BACKEND_DATASOURCE_HOST:-dataportal-backend-db} + DATABASE_PORT: ${DATAPORTAL_BACKEND_DATASOURCE_PORT:-5432} + DATABASE_USER: ${DATAPORTAL_BACKEND_DATASOURCE_USERNAME:-dataportaluser} + DATABASE_PASSWORD: ${DATAPORTAL_BACKEND_DATASOURCE_PASSWORD:-dataportalpw} + DATABASE_DBNAME: ${DATAPORTAL_BACKEND_DATASOURCE_DBNAME:-dataportal} + # ---- ontology + ONTOLOGY_FILES_FOLDER_UI: ${DATAPORTAL_BACKEND_ONTOLOGY_FILES_FOLDER:-/opt/dataportal-backend/ontology} + ONTOLOGY_DB_MIGRATION_FOLDER: ${DATAPORTAL_BACKEND_ONTOLOGY_DB_MIGRATION_FOLDER:-/opt/dataportal-backend/ontology/migration} # ---- auth - KEYCLOAK_ENABLED: ${FEASIBILITY_BACKEND_KEYCLOAK_ENABLED:-true} - KEYCLOAK_ALLOWED_ROLE: ${FEASIBILITY_BACKEND_KEYCLOAK_ALLOWED_ROLE:-FeasibilityUser} - KEYCLOAK_POWER_ROLE: ${FEASIBILITY_BACKEND_KEYCLOAK_POWER_ROLE:-FeasibilityPowerUser} - KEYCLOAK_ADMIN_ROLE: ${FEASIBILITY_BACKEND_KEYCLOAK_ADMIN_ROLE:-FeasibilityAdmin} - KEYCLOAK_BASE_URL_ISSUER: ${FEASIBILITY_BACKEND_KEYCLOAK_BASE_URL_ISSUER:-http://auth:8080} - KEYCLOAK_BASE_URL_JWK: ${FEASIBILITY_BACKEND_KEYCLOAK_BASE_URL_JWK:-http://auth:8080} - KEYCLOAK_REALM: ${FEASIBILITY_BACKEND_KEYCLOAK_REALM:-feasibility} + KEYCLOAK_ENABLED: ${DATAPORTAL_BACKEND_KEYCLOAK_ENABLED:-true} + KEYCLOAK_BASE_URL: ${DATAPORTAL_BACKEND_KEYCLOAK_BASE_URL:-http://keycloak:8080} + KEYCLOAK_CLIENT_ID: ${DATAPORTAL_BACKEND_KEYCLOAK_CLIENT_ID:-dataportal-gui} + KEYCLOAK_ALLOWED_ROLE: ${DATAPORTAL_BACKEND_KEYCLOAK_ALLOWED_ROLE:-DataportalUser} + KEYCLOAK_POWER_ROLE: ${DATAPORTAL_BACKEND_KEYCLOAK_POWER_ROLE:-DataportalPowerUser} + KEYCLOAK_ADMIN_ROLE: ${DATAPORTAL_BACKEND_KEYCLOAK_ADMIN_ROLE:-DataportalAdmin} + KEYCLOAK_BASE_URL_ISSUER: ${DATAPORTAL_BACKEND_KEYCLOAK_BASE_URL_ISSUER:-http://auth:8080} + KEYCLOAK_BASE_URL_JWK: ${DATAPORTAL_BACKEND_KEYCLOAK_BASE_URL_JWK:-http://auth:8080} + KEYCLOAK_REALM: ${DATAPORTAL_BACKEND_KEYCLOAK_REALM:-dataportal} + #---- Mock broker + BROKER_CLIENT_MOCK_ENABLED: ${DATAPORTAL_BACKEND_BROKER_CLIENT_MOCK_ENABLED:-false} #---- Direct broker - BROKER_CLIENT_DIRECT_ENABLED: ${FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_ENABLED:-false} - BROKER_CLIENT_DIRECT_USE_CQL: ${FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_USE_CQL:-false} - BROKER_CLIENT_OBFUSCATE_RESULT_COUNT: ${FEASIBILITY_BACKEND_BROKER_CLIENT_OBFUSCATE_RESULT_COUNT:-false} - FLARE_WEBSERVICE_BASE_URL: ${FEASIBILITY_BACKEND_FLARE_WEBSERVICE_BASE_URL:-http://flare:8080} - CQL_SERVER_BASE_URL: ${FEASIBILITY_BACKEND_CQL_SERVER_BASE_URL:-http://fhir-server:8080/fhir} - BROKER_CLIENT_DIRECT_AUTH_BASIC_USERNAME: ${FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_AUTH_BASIC_USERNAME} - BROKER_CLIENT_DIRECT_AUTH_BASIC_PASSWORD: ${FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_AUTH_BASIC_PASSWORD} - BROKER_CLIENT_DIRECT_AUTH_OAUTH_ISSUER_URL: ${FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_AUTH_OAUTH_ISSUER_URL:-https://keycloak.localhost:444/realms/blaze} - BROKER_CLIENT_DIRECT_AUTH_OAUTH_CLIENT_ID: ${FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_AUTH_OAUTH_CLIENT_ID:-account} - BROKER_CLIENT_DIRECT_AUTH_OAUTH_CLIENT_SECRET: ${FEASIBILITY_BACKEND_BROKER_CLIENT_DIRECT_AUTH_OAUTH_CLIENT_SECRET:-insecure} + BROKER_CLIENT_DIRECT_ENABLED: ${DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_ENABLED:-false} + BROKER_CLIENT_DIRECT_USE_CQL: ${DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_USE_CQL:-false} + BROKER_CLIENT_OBFUSCATE_RESULT_COUNT: ${DATAPORTAL_BACKEND_BROKER_CLIENT_OBFUSCATE_RESULT_COUNT:-false} + BROKER_CLIENT_DIRECT_AUTH_BASIC_USERNAME: ${DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_AUTH_BASIC_USERNAME} + BROKER_CLIENT_DIRECT_AUTH_BASIC_PASSWORD: ${DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_AUTH_BASIC_PASSWORD} + BROKER_CLIENT_DIRECT_AUTH_OAUTH_ISSUER_URL: ${DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_AUTH_OAUTH_ISSUER_URL:-https://keycloak.localhost:444/realms/blaze} + BROKER_CLIENT_DIRECT_AUTH_OAUTH_CLIENT_ID: ${DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_AUTH_OAUTH_CLIENT_ID:-account} + BROKER_CLIENT_DIRECT_AUTH_OAUTH_CLIENT_SECRET: ${DATAPORTAL_BACKEND_BROKER_CLIENT_DIRECT_AUTH_OAUTH_CLIENT_SECRET:-insecure} + FLARE_WEBSERVICE_BASE_URL: ${DATAPORTAL_BACKEND_FLARE_WEBSERVICE_BASE_URL:-http://flare:8080} + CQL_SERVER_BASE_URL: ${DATAPORTAL_BACKEND_CQL_SERVER_BASE_URL:-http://fhir-server:8080/fhir} # ---- Aktin broker - BROKER_CLIENT_AKTIN_ENABLED: ${FEASIBILITY_BACKEND_AKTIN_ENABLED:-false} - AKTIN_BROKER_BASE_URL: ${FEASIBILITY_BACKEND_AKTIN_BROKER_BASE_URL:-http://aktin-broker:8080/broker/} - AKTIN_BROKER_API_KEY: ${FEASIBILITY_BACKEND_AKTIN_BROKER_API_KEY:-xxxApiKeyAdmin123} + BROKER_CLIENT_AKTIN_ENABLED: ${DATAPORTAL_BACKEND_AKTIN_ENABLED:-false} + AKTIN_BROKER_BASE_URL: ${DATAPORTAL_BACKEND_AKTIN_BROKER_BASE_URL:-http://aktin-broker:8080/broker/} + AKTIN_BROKER_API_KEY: ${DATAPORTAL_BACKEND_AKTIN_BROKER_API_KEY:-xxxApiKeyAdmin123} # ---- DSF broker - BROKER_CLIENT_DSF_ENABLED: ${FEASIBILITY_BACKEND_DSF_ENABLED:-false} - DSF_SECURITY_CACERT: ${FEASIBILITY_BACKEND_DSF_CACERT:-/opt/codex-feasibility-security/ca.pem} - DSF_SECURITY_KEYSTORE_P12FILE: ${FEASIBILITY_BACKEND_DSF_DSF_SECURITY_KEYSTORE_P12FILE:-/opt/codex-feasibility-security/test-user.p12} - DSF_SECURITY_KEYSTORE_PASSWORD: ${FEASIBILITY_BACKEND_DSF_SECURITY_KEYSTORE_PASSWORD:-password} - DSF_PROXY_HOST: ${FEASIBILITY_BACKEND_DSF_PROXY_HOST} - DSF_PROXY_USERNAME: ${FEASIBILITY_BACKEND_DSF_PROXY_USERNAME} - DSF_PROXY_PASSWORD: ${FEASIBILITY_BACKEND_DSF_PROXY_PASSWORD} - DSF_WEBSERVICE_BASE_URL: ${FEASIBILITY_BACKEND_DSF_WEBSERVICE_BASE_URL:-https://dsf-zars-fhir-proxy/fhir} - DSF_WEBSOCKET_URL: ${FEASIBILITY_BACKEND_DSF_WEBSOCKET_URL:-wss://dsf-zars-fhir-proxy:443/fhir/ws} - DSF_ORGANIZATION_ID: ${FEASIBILITY_BACKEND_DSF_ORGANIZATION_ID:-Test_ZARS} + BROKER_CLIENT_DSF_ENABLED: ${DATAPORTAL_BACKEND_DSF_ENABLED:-false} + DSF_SECURITY_CACERT: ${DATAPORTAL_BACKEND_DSF_CACERT:-/opt/dataportal-security/ca.pem} + DSF_SECURITY_KEYSTORE_P12FILE: ${DATAPORTAL_BACKEND_DSF_DSF_SECURITY_KEYSTORE_P12FILE:-/opt/dataportal-security/test-user.p12} + DSF_SECURITY_KEYSTORE_PASSWORD: ${DATAPORTAL_BACKEND_DSF_SECURITY_KEYSTORE_PASSWORD:-password} + DSF_PROXY_HOST: ${DATAPORTAL_BACKEND_DSF_PROXY_HOST} + DSF_PROXY_USERNAME: ${DATAPORTAL_BACKEND_DSF_PROXY_USERNAME} + DSF_PROXY_PASSWORD: ${DATAPORTAL_BACKEND_DSF_PROXY_PASSWORD} + DSF_WEBSERVICE_BASE_URL: ${DATAPORTAL_BACKEND_DSF_WEBSERVICE_BASE_URL:-https://dsf-zars-fhir-proxy/fhir} + DSF_WEBSOCKET_URL: ${DATAPORTAL_BACKEND_DSF_WEBSOCKET_URL:-wss://dsf-zars-fhir-proxy:443/fhir/ws} + DSF_ORGANIZATION_ID: ${DATAPORTAL_BACKEND_DSF_ORGANIZATION_ID:-Test_ZARS} # ---- privacy - PRIVACY_QUOTA_SOFT_CREATE_AMOUNT: ${FEASIBILITY_BACKEND_PRIVACY_QUOTA_SOFT_CREATE_AMOUNT:-3} - PRIVACY_QUOTA_SOFT_CREATE_INTERVALMINUTES: ${FEASIBILITY_BACKEND_PRIVACY_QUOTA_SOFT_CREATE_INTERVALMINUTES:-1} - PRIVACY_QUOTA_HARD_CREATE_AMOUNT: ${FEASIBILITY_BACKEND_PRIVACY_QUOTA_HARD_CREATE_AMOUNT:-50} - PRIVACY_QUOTA_HARD_CREATE_INTERVALMINUTES: ${FEASIBILITY_BACKEND_PRIVACY_QUOTA_HARD_CREATE_INTERVALMINUTES:-10080} - PRIVACY_QUOTA_READ_SUMMARY_POLLINGINTERVALSECONDS: ${FEASIBILITY_BACKEND_PRIVACY_QUOTA_READ_SUMMARY_POLLINGINTERVALSECONDS:-10} - PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_POLLINGINTERVALSECONDS: ${FEASIBILITY_BACKEND_PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_POLLINGINTERVALSECONDS:-10} - PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_AMOUNT: ${FEASIBILITY_BACKEND_PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_AMOUNT:-3} - PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_INTERVALSECONDS: ${FEASIBILITY_BACKEND_PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_INTERVALSECONDS:-7200} - PRIVACY_THRESHOLD_RESULTS: ${FEASIBILITY_BACKEND_PRIVACY_THRESHOLD_RESULTS:-20} - PRIVACY_THRESHOLD_SITES: ${FEASIBILITY_BACKEND_PRIVACY_THRESHOLD_SITES:-3} + PRIVACY_QUOTA_SOFT_CREATE_AMOUNT: ${DATAPORTAL_BACKEND_PRIVACY_QUOTA_SOFT_CREATE_AMOUNT:-3} + PRIVACY_QUOTA_SOFT_CREATE_INTERVALMINUTES: ${DATAPORTAL_BACKEND_PRIVACY_QUOTA_SOFT_CREATE_INTERVALMINUTES:-1} + PRIVACY_QUOTA_HARD_CREATE_AMOUNT: ${DATAPORTAL_BACKEND_PRIVACY_QUOTA_HARD_CREATE_AMOUNT:-50} + PRIVACY_QUOTA_HARD_CREATE_INTERVALMINUTES: ${DATAPORTAL_BACKEND_PRIVACY_QUOTA_HARD_CREATE_INTERVALMINUTES:-10080} + PRIVACY_QUOTA_READ_SUMMARY_POLLINGINTERVALSECONDS: ${DATAPORTAL_BACKEND_PRIVACY_QUOTA_READ_SUMMARY_POLLINGINTERVALSECONDS:-10} + PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_POLLINGINTERVALSECONDS: ${DATAPORTAL_BACKEND_PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_POLLINGINTERVALSECONDS:-10} + PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_AMOUNT: ${DATAPORTAL_BACKEND_PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_AMOUNT:-3} + PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_INTERVALSECONDS: ${DATAPORTAL_BACKEND_PRIVACY_QUOTA_READ_DETAILED_OBFUSCATED_INTERVALSECONDS:-7200} + PRIVACY_THRESHOLD_RESULTS: ${DATAPORTAL_BACKEND_PRIVACY_THRESHOLD_RESULTS:-20} + PRIVACY_THRESHOLD_SITES: ${DATAPORTAL_BACKEND_PRIVACY_THRESHOLD_SITES:-3} + PRIVACY_THRESHOLD_SITES_RESULT: ${DATAPORTAL_BACKEND_PRIVACY_THRESHOLD_SITES_RESULT:-0} QUERYRESULT_DISABLE_LOG_FILE_ENCRYPTION: "true" + # ---- Elastic Search + ELASTIC_SEARCH_ENABLED: ${DATAPORTAL_BACKEND_ELASTIC_SEARCH_ENABLED:-true} + ELASTIC_SEARCH_HOST: ${DATAPORTAL_BACKEND_ELASTIC_SEARCH_HOST:-dataportal-elastic} + ELASTIC_SEARCH_FILTER: ${DATAPORTAL_BACKEND_ELASTIC_SEARCH_FILTER:-context,terminology,kds_module} # ---- logging - LOG_LEVEL_SQL: ${FEASIBILITY_BACKEND_LOG_LEVEL_SQL:-warn} - LOG_LEVEL: ${FEASIBILITY_BACKEND_LOG_LEVEL:-warn} + LOG_LEVEL_SQL: ${DATAPORTAL_BACKEND_LOG_LEVEL_SQL:-warn} + LOG_LEVEL: ${DATAPORTAL_BACKEND_LOG_LEVEL:-warn} extra_hosts: - - "keycloak.localhost:host-gateway" - restart: unless-stopped + - "auth.localhost:host-gateway" volumes: - - ${FEASIBILITY_BACKEND_CERTS_PATH:-../secrets}:/opt/codex-feasibility-security - - ./certs:/opt/codex-feasibility-backend/certs + - ${DATAPORTAL_BACKEND_CERTS_PATH:-../secrets}:/opt/dataportal-security + - ./certs:/opt/dataportal-backend/certs - feasibility-gui-backend-db: - image: 'postgres:15-alpine' + dataportal-postgres: + image: 'postgres:16-alpine' ports: - - ${FEASIBILITY_BACKEND_DB_PORT:-127.0.0.1:5432}:5432 + - ${DATAPORTAL_BACKEND_DB_PORT:-127.0.0.1:5432}:5432 environment: - POSTGRES_USER: ${FEASIBILITY_BACKEND_DATASOURCE_USERNAME:-guidbuser} - POSTGRES_PASSWORD: ${FEASIBILITY_BACKEND_DATASOURCE_PASSWORD:-guidbpw} - POSTGRES_DB: codex_ui + POSTGRES_USER: ${DATAPORTAL_BACKEND_DATASOURCE_USERNAME:-dataportaluser} + POSTGRES_PASSWORD: ${DATAPORTAL_BACKEND_DATASOURCE_PASSWORD:-dataportalpw} + POSTGRES_DB: dataportal restart: unless-stopped volumes: - - type: volume - source: feas-backend-db-data - target: /var/lib/postgresql/data + - type: volume + source: dataportal-postgres-data + target: /var/lib/postgresql/data + dataportal-elastic: + image: docker.elastic.co/elasticsearch/elasticsearch:8.15.0 + ports: + - "127.0.0.1:9200:9200" + - "127.0.0.1:9300:9300" + healthcheck: + test: [ "CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1" ] + interval: 30s + timeout: 30s + retries: 3 + environment: + discovery.type: single-node + ES_JAVA_OPTS: -Xmx512m -Xms512m + node.name: es01 + cluster.name: elasticsearch + xpack.security.enabled: false + volumes: + - type: volume + source: dataportal-elastic-data + target: /usr/share/elasticsearch/data + init-elasticsearch: + image: curlimages/curl:8.8.0 + depends_on: + dataportal-elastic: + condition: service_healthy + environment: + ELASTIC_HOST: http://dataportal-elastic:9200 + ELASTIC_GIT_TAG: v3.0.0-test.2 + ELASTIC_FILEPATH: https://github.com/medizininformatik-initiative/fhir-ontology-generator/raw/TAGPLACEHOLDER/example/fdpg-ontology/ + ELASTIC_FILENAME: elastic.zip + # if set to false, existing indices are not overridden. + OVERRIDE_EXISTING: true + entrypoint: [ "sh", "-c", "/tmp/init.sh" ] + volumes: + - type: bind + source: ./elastic-init.sh + target: /tmp/init.sh volumes: - feas-backend-db-data: - name: "feas-backend-db-data" + dataportal-postgres-data: + name: "dataportal-postgres-data" + dataportal-elastic-data: + name: "dataportal-elastic-data" \ No newline at end of file diff --git a/feasibility-portal/backend/elastic-init.sh b/feasibility-portal/backend/elastic-init.sh new file mode 100755 index 00000000..8bdff646 --- /dev/null +++ b/feasibility-portal/backend/elastic-init.sh @@ -0,0 +1,53 @@ +#!/bin/sh + +until curl -X GET "$ELASTIC_HOST/_cluster/health" | grep -q '"status":"green"\|"status":"yellow"'; do + echo "Waiting for Elasticsearch..." + sleep 5 +done + +ABSOLUTE_FILEPATH=$(echo "$ELASTIC_FILEPATH" | sed "s/TAGPLACEHOLDER/$ELASTIC_GIT_TAG/")"$ELASTIC_FILENAME" +echo "Downloading $ABSOLUTE_FILEPATH" +response_onto_dl=$(curl --write-out "%{http_code}" -sLO "$ABSOLUTE_FILEPATH") + +if [ "$response_onto_dl" -ne 200 ]; then + echo "Could not download ontology file. Maybe the tag $ELASTIC_GIT_TAG does not exist? Error code was $response_onto_dl" + exit 1 +fi + +unzip -o "$ELASTIC_FILENAME" + +if [ "$OVERRIDE_EXISTING" = "true" ]; then + echo "(Trying to) delete existing indices" + curl -s -DELETE "$ELASTIC_HOST/ontology" + curl -s -DELETE "$ELASTIC_HOST/codeable_concept" +fi + +echo "Creating ontology index..." +response_onto=$(curl --write-out "%{http_code}" -s --output /dev/null -XPUT -H 'Content-Type: application/json' "$ELASTIC_HOST/ontology" -d @elastic/ontology_index.json) +echo "Creating codeable concept index..." +response_cc=$(curl --write-out "%{http_code}" -s --output /dev/null -XPUT -H 'Content-Type: application/json' "$ELASTIC_HOST/codeable_concept" -d @elastic/codeable_concept_index.json) +echo "Done" + +for FILE in elastic/*; do + if [ -f "$FILE" ]; then + BASENAME=$(basename "$FILE") + if echo "$BASENAME" | grep -q "^onto_es__ontology.*\.json$"; then + if [ "$response_onto" -eq 200 ] || [ "$OVERRIDE_EXISTING" = "true" ]; then + echo "Uploading $BASENAME" + curl -s --output /dev/null -XPOST -H 'Content-Type: application/json' --data-binary @"$FILE" "$ELASTIC_HOST/ontology/_bulk?pretty" + else + echo "Skipping $BASENAME because index was already existing. Set OVERRIDE_EXISTING to true to force creating a new index" + fi + fi + if echo "$BASENAME" | grep -q "^onto_es__codeable_concept.*\.json$"; then + if [ "$response_cc" -eq 200 ] || [ "$OVERRIDE_EXISTING" = "true" ]; then + echo "Uploading $BASENAME" + curl -s --output /dev/null -XPOST -H 'Content-Type: application/json' --data-binary @"$FILE" "$ELASTIC_HOST/codeable_concept/_bulk?pretty" + else + echo "Skipping $BASENAME because index was already existing. Set OVERRIDE_EXISTING to true to force creating a new index" + fi + fi + fi +done + +echo "All done" diff --git a/feasibility-portal/gui/deploy-config.json.default b/feasibility-portal/gui/deploy-config.json.default index b5d0ff54..ad9fe91c 100644 --- a/feasibility-portal/gui/deploy-config.json.default +++ b/feasibility-portal/gui/deploy-config.json.default @@ -6,10 +6,10 @@ "baseUrl": "/api" }, "uiBackendApi": { - "baseUrl": "https://api.datenportal.localhost/api/v3" + "baseUrl": "https://api.datenportal.localhost/api/v4" }, "auth": { - "baseUrl": "https://auth.datenportal.localhost", + "baseUrl": "https://example.org/auth", "realm": "feasibility", "clientId": "feasibility-webapp", "roles": ["FeasibilityUser"] diff --git a/feasibility-portal/gui/docker-compose.yml b/feasibility-portal/gui/docker-compose.yml index 3f833f4f..73ad4fd0 100644 --- a/feasibility-portal/gui/docker-compose.yml +++ b/feasibility-portal/gui/docker-compose.yml @@ -1,8 +1,6 @@ services: dataportal-ui: - image: ghcr.io/medizininformatik-initiative/feasibility-gui:5.0.0 + image: ghcr.io/medizininformatik-initiative/feasibility-gui:6.0.0-alpha.4 restart: unless-stopped - ports: - - "127.0.0.1:8080:8080" volumes: - ./deploy-config.json:/usr/share/nginx/html/assets/config/config.deploy.json diff --git a/feasibility-portal/keycloak/.env.default b/feasibility-portal/keycloak/.env.default index 3b653f1e..bf2f01c3 100644 --- a/feasibility-portal/keycloak/.env.default +++ b/feasibility-portal/keycloak/.env.default @@ -2,10 +2,10 @@ FEASIBILITY_KC_DB=keycloakdb FEASIBILITY_KC_DB_USER=keycloakdbuser FEASIBILITY_KC_DB_PW=keycloakdbpw FEASIBILITY_KC_ADMIN_USER=admin -FEASIBILITY_KC_ADMIN_PW=adminpw -FEASIBILITY_KC_HTTP_RELATIVE_PATH=/ -FEASIBILITY_KC_HOSTNAME_URL=https://auth.datenportal.localhost -FEASIBILITY_KC_HOSTNAME_ADMIN_URL=https://auth.datenportal.localhost +FEASIBILITY_KC_ADMIN_PW=admin +FEASIBILITY_KC_HTTP_RELATIVE_PATH=/auth +FEASIBILITY_KC_HOSTNAME_URL=https://example.org/auth +FEASIBILITY_KC_HOSTNAME_ADMIN_URL=https://example.org/auth FEASIBILITY_KC_LOG_LEVEL=info FEASIBILITY_KC_PROXY=edge diff --git a/feasibility-portal/keycloak/docker-compose.yml b/feasibility-portal/keycloak/docker-compose.yml index d82b36a9..0aa53a8a 100644 --- a/feasibility-portal/keycloak/docker-compose.yml +++ b/feasibility-portal/keycloak/docker-compose.yml @@ -1,3 +1,4 @@ + services: auth-db: image: postgres:15-alpine @@ -16,13 +17,13 @@ services: environment: KC_DB: postgres KC_DB_URL: "jdbc:postgresql://auth-db:5432/${FEASIBILITY_KC_DB}" - KC_DB_USERNAME: ${FEASIBILITY_KC_DB_USER} - KC_DB_PASSWORD: ${FEASIBILITY_KC_DB_PW} - KEYCLOAK_ADMIN: ${FEASIBILITY_KC_ADMIN_USER} - KEYCLOAK_ADMIN_PASSWORD: ${FEASIBILITY_KC_ADMIN_PW} - KC_HTTP_RELATIVE_PATH: ${FEASIBILITY_KC_HTTP_RELATIVE_PATH} - KC_HOSTNAME: ${FEASIBILITY_KC_HOSTNAME_URL:-https://auth.datenportal.localhost} - KC_HOSTNAME_ADMIN: ${FEASIBILITY_KC_HOSTNAME_ADMIN_URL} + KC_DB_USERNAME: ${FEASIBILITY_KC_DB_USER:-keycloakdbuser} + KC_DB_PASSWORD: ${FEASIBILITY_KC_DB_PW:-keycloakdbpw} + KEYCLOAK_ADMIN: ${FEASIBILITY_KC_ADMIN_USER:-} + KEYCLOAK_ADMIN_PASSWORD: ${FEASIBILITY_KC_ADMIN_PW:-} + KC_HTTP_RELATIVE_PATH: ${FEASIBILITY_KC_HTTP_RELATIVE_PATH:-/auth} + KC_HOSTNAME: ${FEASIBILITY_KC_HOSTNAME_URL:-https://auth.localhost} + KC_HOSTNAME_ADMIN: ${FEASIBILITY_KC_HOSTNAME_ADMIN_URL:-https://auth.localhost} KC_LOG_LEVEL: ${FEASIBILITY_KC_LOG_LEVEL:-info} KC_PROXY: ${FEASIBILITY_KC_PROXY:-edge} volumes: diff --git a/feasibility-portal/proxy/.env.default b/feasibility-portal/proxy/.env.default index 42e49f69..ff8ebef3 100644 --- a/feasibility-portal/proxy/.env.default +++ b/feasibility-portal/proxy/.env.default @@ -1,6 +1,13 @@ +# Set separate hostnames if 'subdomains.nginx.conf' is used (see FEASIBILITY_PORTAL_PROXY_NGINX_CONFIG below). +# For 'context-paths.nginx.conf' these values are ignored. BACKEND_HOSTNAME=api.datenportal.localhost KEYCLOAK_HOSTNAME=auth.datenportal.localhost GUI_HOSTNAME=datenportal.localhost -PROXY_CERTIFICATE_PATH=../auth/cert.pem -PROXY_CERTIFICATE_KEY_PATH=../auth/key.pem -PROXY_NGINX_CONFIG_PATH=./nginx.conf + +# Comment one of the FEASIBILITY_PORTAL_PROXY_NGINX_CONFIG depending if it is used 'subdomains.nginx.conf' or 'context-paths.nginx.conf' +FEASIBILITY_PORTAL_PROXY_NGINX_CONFIG=./subdomains.nginx.conf +#FEASIBILITY_PORTAL_PROXY_NGINX_CONFIG=./context-paths.nginx.conf +FEASIBILITY_PORTAL_PROXY_CERTIFICATE_PATH=../auth/cert.pem +FEASIBILITY_PORTAL_PROXY_CERTIFICATE_KEY_PATH=../auth/key.pem + + diff --git a/feasibility-portal/proxy/conf.d/backend.conf.template b/feasibility-portal/proxy/conf.d/backend.conf.template index 9d0ca7a2..6c5d595b 100644 --- a/feasibility-portal/proxy/conf.d/backend.conf.template +++ b/feasibility-portal/proxy/conf.d/backend.conf.template @@ -1,12 +1,22 @@ - server { +server { + listen 8443 ssl; + http2 on; + server_name ${BACKEND_HOSTNAME}; - listen 8443 ssl; - server_name ${BACKEND_HOSTNAME}; - - location / { - proxy_pass http://feasibility-gui-backend:8090/; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header X-Forwarded-Host $host; - } + # redirect server error pages to the static page /50x.html + # + error_page 500 502 503 504 /50x.html; + location = /50x.html { + root /usr/share/nginx/html; + } + location / { + #Used with 'subdomains.nginx.conf' + proxy_pass http://dataportal-backend:8090/; + + #Used with 'context-paths.nginx.conf' + #set $backend_upstream http://dataportal-backend:8090; + #proxy_pass http://$backend_upstream$request_uri; } + +} \ No newline at end of file diff --git a/feasibility-portal/proxy/conf.d/gui.conf.template b/feasibility-portal/proxy/conf.d/gui.conf.template index 874b4757..eed3716e 100644 --- a/feasibility-portal/proxy/conf.d/gui.conf.template +++ b/feasibility-portal/proxy/conf.d/gui.conf.template @@ -1,17 +1,23 @@ server { + listen 8443 ssl; + http2 on; + server_name ${GUI_HOSTNAME}; - listen 8443 ssl; - server_name ${GUI_HOSTNAME}; + # redirect server error pages to the static page /50x.html + # + error_page 500 502 503 504 /50x.html; + location = /50x.html { + root /usr/share/nginx/html; + } - location / { - proxy_pass http://dataportal-ui:8080/; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header Host $host; - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection "upgrade"; - proxy_read_timeout 86400; - } + location / { + #Used with 'subdomains.nginx.conf' + proxy_pass http://dataportal-ui:8080/; + + #Used with 'context-paths.nginx.conf' + #set $gui_upstream http://dataportal-ui:8080; + #proxy_pass http://$gui_upstream$request_uri; } + +} diff --git a/feasibility-portal/proxy/conf.d/keycloak.conf.template b/feasibility-portal/proxy/conf.d/keycloak.conf.template index fb27b44b..87f3f559 100644 --- a/feasibility-portal/proxy/conf.d/keycloak.conf.template +++ b/feasibility-portal/proxy/conf.d/keycloak.conf.template @@ -1,30 +1,21 @@ server { + listen 8443 ssl; + http2 on; + server_name ${KEYCLOAK_HOSTNAME}; - listen 8443 ssl; - server_name ${KEYCLOAK_HOSTNAME}; + # redirect server error pages to the static page /50x.html + error_page 500 502 503 504 /50x.html; + location = /50x.html { + root /usr/share/nginx/html; + } - location / { - proxy_buffers 4 128k; - proxy_busy_buffers_size 128k; - proxy_buffer_size 64k; - proxy_pass http://auth:8080; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header X-Forwarded-Host $host; - proxy_set_header X-Forwarded-Port $server_port; - proxy_set_header X-Real-IP $remote_addr; - } + location / { + #Used with 'subdomains.nginx.conf' + proxy_pass http://auth:8080; - #location /keycloakadmin { - # proxy_buffers 4 128k; - # proxy_busy_buffers_size 128k; - # proxy_buffer_size 64k; - # proxy_pass http://auth:8080; - # proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - # proxy_set_header X-Forwarded-Proto $scheme; - # proxy_set_header X-Forwarded-Host $host; - # proxy_set_header X-Forwarded-Port $server_port; - # proxy_set_header X-Real-IP $remote_addr; - # } + #Used with 'context-paths.nginx.conf' + #set $auth_upstream http://auth:8080; + #proxy_pass http://$auth_upstream$request_uri; } +} \ No newline at end of file diff --git a/feasibility-portal/proxy/context-paths.nginx.conf b/feasibility-portal/proxy/context-paths.nginx.conf new file mode 100644 index 00000000..1646a2f9 --- /dev/null +++ b/feasibility-portal/proxy/context-paths.nginx.conf @@ -0,0 +1,112 @@ +pid /tmp/nginx.pid; + +events { + worker_connections 1024; +} + +http { + proxy_read_timeout 7d; + proxy_set_header X-Real-IP $remote_addr; + + include /etc/nginx/mime.types; + gzip on; + gzip_vary on; + gzip_min_length 10240; + gzip_proxied expired no-cache no-store private auth; + gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml; + gzip_disable "MSIE [1-6]\."; + + # Check if a X-Forwarded-Proto header (set by reverse-proxy) is already present. If not take the scheme used to call our nginx server. + map $http_x_forwarded_proto $x_forwarded_proto { + default $http_x_forwarded_proto; + "" $scheme; # Note that if the reverse-proxy does not add a X-Forwarded-Proto header, it may be incorrect if the protocol used by the reverse proxy is not the same as the one on which your nginx server is listening. In this case you have no solution than harcode the correct value. + } + + # Check if a X-Forwarded-Host header (set by reverse-proxy) is already present. If not take the value of the 'Host' header. + map $http_x_forwarded_host $x_forwarded_host { + default $http_x_forwarded_host; + "" $http_host; + } + + # Set the default port of each scheme/protocol (80 for http, 443 for https) + map $x_forwarded_proto $default_http_port { + default 80; + "https" 443; + } + + # Extract the real port of the client request url (unfortunatly nginx has no variable to get this info) + map $http_host $request_port { + default $default_http_port; # If port not explicitely defined in url take the default one associated to the calling scheme/protocol (80 for http, 443 for https) + "~^[^\:]+:(?

\d+)$" $p; + } + + # Check if a X-Forwarded-Port header (set by reverse-proxy) is already present. If not take the port from the client request url + map $http_x_forwarded_port $x_forwarded_port { + default $http_x_forwarded_port; + "" $request_port; + } + + server { + listen 8443 ssl; + http2 on; + + # DNS resolver needed for Docker + resolver 127.0.0.11 valid=10s; + + # SSL-Certificate and private key + ssl_certificate /etc/nginx/certs/cert.pem; + ssl_certificate_key /etc/nginx/certs/key.pem; + + # The supported SSL Protocols + ssl_protocols TLSv1.2 TLSv1.3; + + # NGINX can impose its TLS cipher suite choices over those of a connecting browser, provided the browser supports them. + ssl_prefer_server_ciphers on; + + # The supported SSL Ciphers + ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-RC4-SHA:ECDHE-RSA-RC4-SHA:ECDH-ECDSA-RC4-SHA:ECDH-RSA-RC4-SHA:RC4-SHA'; + + ssl_session_cache builtin:1000 shared:SSL:10m; + + # OCSP Stapling + # When enabled, NGINX will make OCSP requests on behalf of connecting browsers. The response received from the OCSP server is added to NGINX’s browser response, which eliminates the need for browsers to verify a certificate’s revocation status by connecting directly to an OCSP server. + ssl_stapling on; + ssl_stapling_verify on; + + # Header Options + add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always; + add_header X-Content-Type-Options nosniff; + add_header X-Frame-Options SAMEORIGIN; + + # redirect server error pages to the static page /50x.html + # + error_page 500 502 503 504 /50x.html; + location = /50x.html { + root /usr/share/nginx/html; + } + + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $x_forwarded_proto; + proxy_set_header X-Forwarded-Host $x_forwarded_host; + proxy_set_header X-Forwarded-Port $x_forwarded_port; + + location /api { + set $backend_upstream http://feasibility-gui-backend:8090; + proxy_pass $backend_upstream; + + proxy_buffer_size 8k; + proxy_request_buffering off; + client_max_body_size 100M; + } + + location /auth { + set $auth_upstream http://auth:8080; + proxy_pass $auth_upstream; + } + + location / { + set $gui_upstream http://dataportal-ui:8080; + proxy_pass $gui_upstream; + } + } +} diff --git a/feasibility-portal/proxy/docker-compose.yml b/feasibility-portal/proxy/docker-compose.yml index 30101839..14de4be8 100644 --- a/feasibility-portal/proxy/docker-compose.yml +++ b/feasibility-portal/proxy/docker-compose.yml @@ -1,7 +1,8 @@ +# Comment one of the FEASIBILITY_PORTAL_PROXY_NGINX_CONFIG depending if it is used 'subdomains.nginx.conf' or 'context-paths.nginx.conf' services: dataportal-nginx: restart: unless-stopped - image: nginxinc/nginx-unprivileged:1.23-alpine + image: nginxinc/nginx-unprivileged:1.27-alpine environment: BACKEND_HOSTNAME: ${BACKEND_HOSTNAME:-api.datenportal.localhost} GUI_HOSTNAME: ${GUI_HOSTNAME:-datenportal.localhost} @@ -9,9 +10,10 @@ services: ports: - 443:8443 volumes: - - ${PROXY_CERTIFICATE_PATH:-../auth/cert.pem}:/etc/nginx/conf.d/cert.pem - - ${PROXY_CERTIFICATE_KEY_PATH:-../auth/key.pem}:/etc/nginx/conf.d/key.pem - - ${PROXY_NGINX_CONFIG_PATH:-./nginx.conf}:/etc/nginx/nginx.conf:ro - - ./conf.d:/etc/nginx/templates + - ${FEASIBILITY_PORTAL_PROXY_CERTIFICATE_PATH:-../auth/cert.pem}:/etc/nginx/certs/cert.pem + - ${FEASIBILITY_PORTAL_PROXY_CERTIFICATE_KEY_PATH:-../auth/key.pem}:/etc/nginx/certs/key.pem + #- ${FEASIBILITY_PORTAL_PROXY_NGINX_CONFIG:-./context-paths.nginx.conf}:/etc/nginx/nginx.conf:ro + - ${FEASIBILITY_PORTAL_PROXY_NGINX_CONFIG:-./subdomains.nginx.conf}:/etc/nginx/nginx.conf:ro + - ./conf.d:/etc/nginx/templates:ro diff --git a/feasibility-portal/proxy/nginx.conf b/feasibility-portal/proxy/nginx.conf deleted file mode 100644 index 0ffdd73b..00000000 --- a/feasibility-portal/proxy/nginx.conf +++ /dev/null @@ -1,45 +0,0 @@ -pid /tmp/nginx.pid; - -events { - worker_connections 1024; - } - -http { - - gzip on; - gzip_vary on; - gzip_min_length 10240; - gzip_proxied expired no-cache no-store private auth; - gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml; - gzip_disable "MSIE [1-6]\."; - - # SSL-Certificate and private key - ssl_certificate /etc/nginx/conf.d/cert.pem; - ssl_certificate_key /etc/nginx/conf.d/key.pem; - - # The supported SSL Protocols - ssl_protocols TLSv1.2 TLSv1.3; - - # NGINX can impose its TLS cipher suite choices over those of a connecting browser, provided the browser supports them. - ssl_prefer_server_ciphers on; - - # The supported SSL Ciphers - ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-RC4-SHA:ECDHE-RSA-RC4-SHA:ECDH-ECDSA-RC4-SHA:ECDH-RSA-RC4-SHA:RC4-SHA'; - - # Path to the Diffie-Hellman Prime - # to create: openssl dhparam -out /etc/nginx/dhparam.pem 4096 - # ssl_dhparam /etc/nginx/dhparam.pem; - - - ssl_session_cache builtin:1000 shared:SSL:10m; - - # OCSP Stapling - # When enabled, NGINX will make OCSP requests on behalf of connecting browsers. The response received from the OCSP server is added to NGINX’s browser response, which eliminates the need for browsers to verify a certificate’s revocation status by connecting directly to an OCSP server. - ssl_stapling on; - ssl_stapling_verify on; - - # Header Options - add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always; - add_header X-Content-Type-Options nosniff; - - include /etc/nginx/conf.d/*.conf; -} diff --git a/feasibility-portal/proxy/subdomains.nginx.conf b/feasibility-portal/proxy/subdomains.nginx.conf new file mode 100644 index 00000000..6dc10c88 --- /dev/null +++ b/feasibility-portal/proxy/subdomains.nginx.conf @@ -0,0 +1,79 @@ +pid /tmp/nginx.pid; + +events { + worker_connections 1024; + } + +http { + + + # Check if a X-Forwarded-Proto header (set by reverse-proxy) is already present. If not take the scheme used to call our nginx server. + map $http_x_forwarded_proto $x_forwarded_proto { + default $http_x_forwarded_proto; + "" $scheme; # Note that if the reverse-proxy does not add a X-Forwarded-Proto header, it may be incorrect if the protocol used by the reverse proxy is not the same as the one on which your nginx server is listening. In this case you have no solution than harcode the correct value. + } + + # Check if a X-Forwarded-Host header (set by reverse-proxy) is already present. If not take the value of the 'Host' header. + map $http_x_forwarded_host $x_forwarded_host { + default $http_x_forwarded_host; + "" $http_host; + } + + # Set the default port of each scheme/protocol (80 for http, 443 for https) + map $x_forwarded_proto $default_http_port { + default 80; + "https" 443; + } + + # Extract the real port of the client request url (unfortunatly nginx has no variable to get this info) + map $http_host $request_port { + default $default_http_port; # If port not explicitely defined in url take the default one associated to the calling scheme/protocol (80 for http, 443 for https) + "~^[^\:]+:(?

\d+)$" $p; + } + + # Check if a X-Forwarded-Port header (set by reverse-proxy) is already present. If not take the port from the client request url + map $http_x_forwarded_port $x_forwarded_port { + default $http_x_forwarded_port; + "" $request_port; + } + + proxy_read_timeout 7d; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $x_forwarded_proto; + proxy_set_header X-Forwarded-Host $x_forwarded_host; + proxy_set_header X-Forwarded-Port $x_forwarded_port; + + gzip on; + gzip_vary on; + gzip_min_length 10240; + gzip_proxied expired no-cache no-store private auth; + gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml; + gzip_disable "MSIE [1-6]\."; + + # SSL-Certificate and private key + ssl_certificate /etc/nginx/certs/cert.pem; + ssl_certificate_key /etc/nginx/certs/key.pem; + + # The supported SSL Protocols + ssl_protocols TLSv1.2 TLSv1.3; + + # NGINX can impose its TLS cipher suite choices over those of a connecting browser, provided the browser supports them. + ssl_prefer_server_ciphers on; + + # The supported SSL Ciphers + ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-RC4-SHA:ECDHE-RSA-RC4-SHA:ECDH-ECDSA-RC4-SHA:ECDH-RSA-RC4-SHA:RC4-SHA'; + ssl_session_cache builtin:1000 shared:SSL:10m; + + # OCSP Stapling + # When enabled, NGINX will make OCSP requests on behalf of connecting browsers. The response received from the OCSP server is added to NGINX’s browser response, which eliminates the need for browsers to verify a certificate’s revocation status by connecting directly to an OCSP server. + ssl_stapling on; + ssl_stapling_verify on; + + # Header Options + add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always; + add_header X-Content-Type-Options nosniff; + add_header X-Frame-Options SAMEORIGIN; + + include /etc/nginx/conf.d/*.conf; +} diff --git a/feasibility-triangle/.DS_Store b/feasibility-triangle/.DS_Store new file mode 100644 index 00000000..ffe9c66c Binary files /dev/null and b/feasibility-triangle/.DS_Store differ diff --git a/feasibility-triangle/README.md b/feasibility-triangle/README.md index e7e9bba0..de1cc2cc 100644 --- a/feasibility-triangle/README.md +++ b/feasibility-triangle/README.md @@ -10,8 +10,9 @@ The Feasibility Triangle is composed of four components: 1. A Middleware Client (DSF) 2. A Feasibility Analysis Request Executor (FLARE) -3. A FHIR Server (Blaze) -4. Reverse Proxy (NGINX) +3. A Dataselection and Extraction Executor (TORCH) +4. A FHIR Server (Blaze) +5. Reverse Proxy (NGINX) The reverse proxy allows for integration into a site's multi-server infrastructure. It also provides basic auth capability for FHIR server and FLARE components. @@ -61,6 +62,7 @@ Running this setup safely at your site requires a valid certificate and domains. - FHIR server - FLARE +- TORCH - Keycloak (optional, see step 8) You will require two .pem files: a `cert.pem` (certificate) and `cert.key` (private key). @@ -94,7 +96,7 @@ For the reverse proxy you need to choose the configuration (variable `FEASIBILIT - [./subdomains.nginx.conf](./rev-proxy/subdomains.nginx.conf) with separate domains for the services (fhir-server (incl) and optionally flare and keycloak) - All subdomains must point to the host machine the triangle will run on. - - Set the service hostnames (`FLARE_HOSTNAME`, `FHIR_HOSTNAME` and `KEYCLOAK_HOSTNAME`, depending on which services you need) in [rev-proxy/.env](./rev-proxy/.env). + - Set the service hostnames (`FLARE_HOSTNAME`, `FHIR_HOSTNAME`, `TORCH_HOSTNAME` and `KEYCLOAK_HOSTNAME`, depending on which services you need) in [rev-proxy/.env](./rev-proxy/.env). - You can change the default external port the reverse proxy listens on in [rev-proxy/.env](./rev-proxy/.env) (variable `FEASIBILITY_TRIANGLE_REV_PROXY_PORT`). Any value other than `443` needs to be added to all external url's in the `.env` files and to the url's used for accessing the triangle from outside. The default value is `444` to avoid a conflict with the proxy of the local feasibility portal when it is deployed on the same host. @@ -140,6 +142,7 @@ For the reverse proxy you need to choose the configuration (variable `FEASIBILIT The triangle is configured by default to start the following services: - FLARE: A Rest Service, which is needed to translate, execute and evaluate a feasibility query on a FHIR Server using FHIR Search +- TORCH: A Rest Service, which is needed to execute Dataselection and Extraction queries (CRTDLs - clinical-resource-transfer-definition-languge) - BLAZE: The FHIR Server which holds the patient data for feasibility queries - Keycloak (optional): OpenID Connect provider for authorization used by BLAZE component - We recommend using your own keyloak and configuring a blaze realm there @@ -156,7 +159,7 @@ to `false` in `/opt/feasibility-deploy/feasibility-triangle/fhir-server/.env`. To start the triangle execute `/opt/feasibility-deploy/feasibility-triangle/start-triangle.sh`. This starts the following default triangle: -FLARE (FHIR Search executor) - BLAZE (FHIR Server) - Keycloak (optional) +FLARE (FHIR Search executor) - TORCH (FHIR Data Extractor) - BLAZE (FHIR Server) - Keycloak (optional) If you would like to pick other component combinations you can start each component individually by setting your compose project (`export FEASIBILITY_COMPOSE_PROJECT=feasibility-deploy`) navigating to the respective components folder and executing: @@ -186,6 +189,7 @@ These are the URLs for access to the webclients via nginx: | Component | URL | User | Password | |-------------|------------------------------------------------------------------|------------------|------------------| | Flare | `https://your-flare-subdomain.your-domain:configured-port/` | chosen in step 3 | chosen in step 3 | +| TORCH | `https://your-torch-subdomain.your-domain:configured-port/` | chosen in step 3 | chosen in step 3 | | FHIR Server | `https://your-fhir-subdomain.your-domain:configured-port/fhir` | chosen in step 3 | chosen in step 3 | > [!NOTE] @@ -201,7 +205,8 @@ Accessible service via localhost: | Component | URL | Authentication Type | Notes | |-------------|----------------------------------|---------------------|----------------------| | Flare | | None required | | -| FHIR Server | | Bearer Token | Configured in step 8 | +| TORCH | | None required | | +| FHIR Server | | Bearer Token | Configured in step 8 | Please be aware that you will need to set up an ssh tunnel to your server and forward the respective ports if you would like to access the services on localhost without a password. @@ -322,6 +327,15 @@ If new search parameters have been added follow the "fhir-server/README.md -> Re | FHIR_SERVER_HOSTNAME | change the default value of the domain names where the services are reachable | http://fhir-server:8080 | | REV-PROXY| | KEYCLOAK_HOSTNAME | change the default value of the domain names where the services are reachable | https://keycloak.localhost:444/realms/blaze | | REV-PROXY | | FLARE_HOSTNAME |change the default value of the domain names where the services are reachable | http://fhir-server:8080/fhir | | REV-PROXY | +| TORCH_FHIR_URL | The base URL of the FHIR server which contains the patient data Torch is used to extract. | http://fhir-server:8080/fhir | | TORCH | +| TORCH_FLARE_URL | The base URL of the FLARE component if used. | http://flare:8080 | | TORCH | +| TORCH_RESULTS_PERSISTENCE | | PT12H30M5S | | TORCH | +| TORCH_LOG_LEVEL | Log level | debug | | TORCH | +| TORCH_NGINX_FILELOCATION | The base URL of the NGINX, which is used to serve the extracted patient data. | https://torch.localhost:444/fileserver | | TORCH | +| TORCH_BATCHSIZE | The number of patients Torch processes in one batch. | 100 | | TORCH | +| TORCH_MAXCONCURRENCY | Maximum number of parallel threads Torch uses to process the data extractions. | 4 | | TORCH | +| TORCH_USE_CQL | Whether or not to use CQL - if false FLARE is used. | true | | TORCH | +| TORCH_ENABLED | Whether or not Torch should be started. | true | | TORCH | ### Support for self-singed certificates diff --git a/feasibility-triangle/down-triangle.sh b/feasibility-triangle/down-triangle.sh index 27de6e72..b55baf90 100755 --- a/feasibility-triangle/down-triangle.sh +++ b/feasibility-triangle/down-triangle.sh @@ -4,6 +4,7 @@ COMPOSE_PROJECT=${FEASIBILITY_COMPOSE_PROJECT:-feasibility-deploy} BASE_DIR="$( cd -- "$(dirname "$0")" >/dev/null 2>&1 || exit 1 ; pwd -P )" docker compose -p "$COMPOSE_PROJECT" -f "$BASE_DIR"/rev-proxy/docker-compose.yml \ + -f "$BASE_DIR"/torch/docker-compose.yml \ -f "$BASE_DIR"/fhir-server/docker-compose.yml \ -f "$BASE_DIR"/fhir-server/keycloak.docker-compose.yml \ -f "$BASE_DIR"/flare/docker-compose.yml down -v diff --git a/feasibility-triangle/flare/docker-compose.yml b/feasibility-triangle/flare/docker-compose.yml index f874107b..cdfc31a5 100644 --- a/feasibility-triangle/flare/docker-compose.yml +++ b/feasibility-triangle/flare/docker-compose.yml @@ -1,6 +1,6 @@ services: flare: - image: ghcr.io/medizininformatik-initiative/flare:2.3.0 + image: ghcr.io/medizininformatik-initiative/flare:2.4.0-alpha.2 ports: - ${FEASIBILITY_FLARE_PORT:-127.0.0.1:8084}:8080 environment: diff --git a/feasibility-triangle/initialise-triangle-env-files.sh b/feasibility-triangle/initialise-triangle-env-files.sh index 39fdc8ac..b32415ac 100755 --- a/feasibility-triangle/initialise-triangle-env-files.sh +++ b/feasibility-triangle/initialise-triangle-env-files.sh @@ -1,7 +1,7 @@ #!/bin/bash BASE_DIR="$( cd -- "$(dirname "$0")" >/dev/null 2>&1 || exit 1 ; pwd -P )" -envfiles=( "$BASE_DIR/fhir-server/.env" "$BASE_DIR/flare/.env" "$BASE_DIR/rev-proxy/.env") +envfiles=( "$BASE_DIR/fhir-server/.env" "$BASE_DIR/flare/.env" "$BASE_DIR/torch/.env" "$BASE_DIR/rev-proxy/.env") for file in "${envfiles[@]}" do diff --git a/feasibility-triangle/rev-proxy/.env.default b/feasibility-triangle/rev-proxy/.env.default index 32de70ff..8f20a522 100644 --- a/feasibility-triangle/rev-proxy/.env.default +++ b/feasibility-triangle/rev-proxy/.env.default @@ -3,6 +3,7 @@ FHIR_SERVER_HOSTNAME=fhir.example.org KEYCLOAK_HOSTNAME=auth.example.org FLARE_HOSTNAME=flare.example.org +TORCH_HOSTNAME=torch.localhost FEASIBILITY_TRIANGLE_REV_PROXY_NGINX_CONFIG=./context-paths.nginx.conf FEASIBILITY_TRIANGLE_REV_PROXY_PORT=444 diff --git a/feasibility-triangle/rev-proxy/conf.d/torch.conf.template b/feasibility-triangle/rev-proxy/conf.d/torch.conf.template new file mode 100644 index 00000000..516e5fe5 --- /dev/null +++ b/feasibility-triangle/rev-proxy/conf.d/torch.conf.template @@ -0,0 +1,29 @@ +server { + listen 8443 ssl; + http2 on; + server_name ${TORCH_HOSTNAME}; + + # redirect server error pages to the static page /50x.html + # + error_page 500 502 503 504 /50x.html; + location = /50x.html { + root /usr/share/nginx/html; + } + + location / { + auth_basic "Restricted"; + auth_basic_user_file /etc/nginx/.htpasswd; + set $upstream torch:8080; + proxy_pass http://$upstream$request_uri; + proxy_set_header Authorization ""; + } + + location /fileserver/ { + auth_basic "Restricted"; + auth_basic_user_file /etc/nginx/.htpasswd; + set $upstream torch-nginx:8080; + rewrite ^/fileserver/(.*)$ /$1 break; + proxy_pass http://torch-nginx:8080; + proxy_set_header Authorization ""; + } +} diff --git a/feasibility-triangle/rev-proxy/context-paths.nginx.conf b/feasibility-triangle/rev-proxy/context-paths.nginx.conf index c2c2732f..0ffe0719 100644 --- a/feasibility-triangle/rev-proxy/context-paths.nginx.conf +++ b/feasibility-triangle/rev-proxy/context-paths.nginx.conf @@ -109,6 +109,24 @@ http { proxy_pass $auth_upstream; } + location /torch/ { + auth_basic "Restricted"; + auth_basic_user_file /etc/nginx/.htpasswd; + set $torch_upstream http://torch:8080; + rewrite ^/torch/(.*)$ /$1 break; + proxy_pass $torch_upstream; + proxy_set_header Authorization ""; + } + + location /torch/fileserver/ { + auth_basic "Restricted"; + auth_basic_user_file /etc/nginx/.htpasswd; + set $torch_nginx_upstream http://torch-nginx:8080; + rewrite ^/torch/fileserver/(.*)$ /$1 break; + proxy_pass $torch_nginx_upstream; + proxy_set_header Authorization ""; + } + location / { auth_basic "Restricted"; auth_basic_user_file /etc/nginx/.htpasswd; @@ -116,5 +134,7 @@ http { proxy_pass $flare_upstream; proxy_set_header Authorization ""; } + + } } diff --git a/feasibility-triangle/rev-proxy/docker-compose.yml b/feasibility-triangle/rev-proxy/docker-compose.yml index b9f94b3b..8e1bfa70 100644 --- a/feasibility-triangle/rev-proxy/docker-compose.yml +++ b/feasibility-triangle/rev-proxy/docker-compose.yml @@ -6,6 +6,7 @@ services: FHIR_SERVER_HOSTNAME: ${FHIR_SERVER_HOSTNAME:-fhir.localhost} KEYCLOAK_HOSTNAME: ${KEYCLOAK_HOSTNAME:-auth.localhost} FLARE_HOSTNAME: ${FLARE_HOSTNAME:-flare.localhost} + TORCH_HOSTNAME: ${TORCH_HOSTNAME:-torch.localhost} ports: - ${FEASIBILITY_TRIANGLE_REV_PROXY_PORT:-444}:8443 volumes: diff --git a/feasibility-triangle/start-triangle.sh b/feasibility-triangle/start-triangle.sh index ecd21f57..d06df279 100755 --- a/feasibility-triangle/start-triangle.sh +++ b/feasibility-triangle/start-triangle.sh @@ -40,3 +40,14 @@ else rm "$BASE_DIR/rev-proxy/conf.d/flare.conf" fi fi + +if [ -f "$BASE_DIR/torch/.env" ] && grep -qE '^TORCH_ENABLED=true\s*$' "$BASE_DIR/torch/.env"; then + if [ ! -f "$BASE_DIR/rev-proxy/conf.d/torch.conf" ]; then + cp "$BASE_DIR/rev-proxy/conf.d/torch.conf.template" "$BASE_DIR/rev-proxy/conf.d/torch.conf" + fi + COMPOSE_IGNORE_ORPHANS=True docker compose -p "$COMPOSE_PROJECT" -f "$BASE_DIR"/torch/docker-compose.yml up -d +else + if [ -f "$BASE_DIR/rev-proxy/conf.d/torch.conf" ]; then + rm "$BASE_DIR/rev-proxy/conf.d/torch.conf" + fi +fi \ No newline at end of file diff --git a/feasibility-triangle/stop-triangle.sh b/feasibility-triangle/stop-triangle.sh index b32e0e60..6a57880f 100755 --- a/feasibility-triangle/stop-triangle.sh +++ b/feasibility-triangle/stop-triangle.sh @@ -5,6 +5,7 @@ COMPOSE_PROJECT=${FEASIBILITY_COMPOSE_PROJECT:-feasibility-deploy} BASE_DIR="$( cd -- "$(dirname "$0")" >/dev/null 2>&1 || exit 1 ; pwd -P )" docker compose -p "$COMPOSE_PROJECT" -f "$BASE_DIR"/flare/docker-compose.yml \ + -f "$BASE_DIR"/torch/docker-compose.yml \ -f "$BASE_DIR"/fhir-server/docker-compose.yml \ -f "$BASE_DIR"/fhir-server/keycloak.docker-compose.yml \ -f "$BASE_DIR"/rev-proxy/docker-compose.yml stop diff --git a/feasibility-triangle/torch/.env.default b/feasibility-triangle/torch/.env.default new file mode 100644 index 00000000..2fb1f8fa --- /dev/null +++ b/feasibility-triangle/torch/.env.default @@ -0,0 +1,14 @@ +TORCH_FHIR_URL=http://torch-data-store:8080/fhir +TORCH_FLARE_URL=http://torch-flare:8080 +TORCH_RESULTS_PERSISTENCE=PT12H30M5S +TORCH_LOG_LEVEL=debug +TORCH_NGINX_FILELOCATION=http://localhost:80 +TORCH_BATCHSIZE=100 +TORCH_MAXCONCURRENCY=4 +TORCH_USE_CQL=true +TORCH_ENABLED=true +TORCH_FHIR_USER= +TORCH_FHIR_PASSWORD= +TORCH_FHIR_OAUTH_ISSUER_URI=https://auth.localhost:444/realms/blaze +TORCH_FHIR_OAUTH_CLIENT_ID=account +TORCH_FHIR_OAUTH_CLIENT_SECRET=insecure \ No newline at end of file diff --git a/feasibility-triangle/torch/docker-compose.yml b/feasibility-triangle/torch/docker-compose.yml new file mode 100644 index 00000000..61eee20a --- /dev/null +++ b/feasibility-triangle/torch/docker-compose.yml @@ -0,0 +1,39 @@ +services: + torch-nginx: + restart: unless-stopped + image: nginxinc/nginx-unprivileged:1.25.5-alpine + ports: + - ${TORCH_NGINX_PORT:-127.0.0.1:80}:8080 + volumes: + - ./torch.nginx.conf:/etc/nginx/nginx.conf + - triangle-torch-data-store:/app/output + torch: + restart: unless-stopped + image: ghcr.io/medizininformatik-initiative/torch:1.0.0-alpha.1 + ports: + - ${TORCH_PORT:-127.0.0.1:8086}:8080 + environment: + TORCH_FHIR_URL: ${TORCH_FHIR_URL:-http://torch-data-store:8080/fhir} + TORCH_FLARE_URL: ${TORCH_FLARE_URL:-http://torch-flare:8080} + TORCH_MAPPING_CONSENT: /app/mappings/consent-mappings_fhir.json + TORCH_MAPPING_CONSENT_TO_PROFILE: /app/mappings/profile_to_consent.json + TORCH_PROFILE_DIR: /app/structureDefinitions + TORCH_RESULTS_PERSISTENCE: ${TORCH_RESULTS_PERSISTENCE:-PT12H30M5S} + LOG_LEVEL: ${TORCH_LOG_LEVEL:-INFO} + NGINX_FILELOCATION: ${TORCH_NGINX_FILELOCATION:-http://localhost:80} + TORCH_BATCHSIZE: ${TORCH_BATCHSIZE:-100} + TORCH_MAXCONCURRENCY: ${TORCH_MAXCONCURRENCY:-4} + TORCH_USE_CQL: ${TORCH_USE_CQL:-true} + TORCH_FHIR_USER: ${TORCH_FHIR_USER:-""} + TORCH_FHIR_PASSWORD: ${TORCH_FHIR_PASSWORD:-""} + TORCH_FHIR_OAUTH_ISSUER_URI: ${TORCH_FHIR_OAUTH_ISSUER_URI:-https://auth.localhost:444/realms/blaze} + TORCH_FHIR_OAUTH_CLIENT_ID: ${TORCH_FHIR_OAUTH_CLIENT_ID:-account} + TORCH_FHIR_OAUTH_CLIENT_SECRET: ${TORCH_FHIR_OAUTH_CLIENT_SECRET:-insecure} + volumes: + - "triangle-torch-data-store:/app/output" + - ../auth:/app/certs + extra_hosts: + - "auth.localhost:host-gateway" + +volumes: + triangle-torch-data-store: diff --git a/feasibility-triangle/torch/example-crtdl.json b/feasibility-triangle/torch/example-crtdl.json new file mode 100644 index 00000000..009f2ee2 --- /dev/null +++ b/feasibility-triangle/torch/example-crtdl.json @@ -0,0 +1,56 @@ +{ + "display": "", + "version": "http://json-schema.org/to-be-done/schema#", + "cohortDefinition": { + "version": "http://to_be_decided.com/draft-1/schema#", + "display": "Ausgewählte Merkmale", + "inclusionCriteria": [ + [ + { + "termCodes": [ + { + "code": "263495000", + "display": "Geschlecht", + "system": "http://snomed.info/sct", + "version": "" + } + ], + "context": { + "code": "Patient", + "display": "Patient", + "system": "fdpg.mii.cds", + "version": "1.0.0" + }, + "valueFilter": { + "selectedConcepts": [ + { + "code": "male", + "display": "Male", + "system": "http://hl7.org/fhir/administrative-gender", + "version": "2099" + } + ], + "type": "concept" + } + } + ] + ] + }, + "dataExtraction": { + "attributeGroups": [ + { + "groupReference": "https://www.medizininformatik-initiative.de/fhir/core/modul-diagnose/StructureDefinition/Diagnose", + "attributes": [ + { + "attributeRef": "Condition.code", + "mustHave": false + }, + { + "attributeRef": "Condition.recordedDate", + "mustHave": false + } + ] + } + ] + } +} \ No newline at end of file diff --git a/feasibility-triangle/torch/execute-crtdl.sh b/feasibility-triangle/torch/execute-crtdl.sh new file mode 100755 index 00000000..550c3873 --- /dev/null +++ b/feasibility-triangle/torch/execute-crtdl.sh @@ -0,0 +1,45 @@ +#!/usr/bin/env sh + +TORCH_BASE_URL=${TORCH_BASE_URL:-https://torch.localhost:444} +TORCH_USERNAME=${TORCH_USERNAME:-"test"} +TORCH_PASSWORD=${TORCH_PASSWORD:-"tast"} +CURL_INSECURE=${CURL_INSECURE:-false} + +CURL_OPTIONS="" +if [ "$CURL_INSECURE" = "true" ]; then + CURL_OPTIONS="-k" +fi + +TORCH_AUTHORIZATION=$(printf "%s:%s" "$TORCH_USERNAME" "$TORCH_PASSWORD" | base64) + +if [ "$#" -ne 1 ]; then + printf "Usage: %s \n" "$0" + exit 1 +fi + +json_file="$1" +json_string=$(cat "$json_file") + +base64_encoded=$(printf "%s" "$json_string" | base64) + +response=$(curl --location -i $CURL_OPTIONS "${TORCH_BASE_URL}/fhir/\$extract-data" \ +--header 'Content-Type: application/fhir+json' \ +--header "Authorization: Basic ${TORCH_AUTHORIZATION}" \ +--data '{ + "resourceType": "Parameters", + "id": "example6", + "parameter": [ + { + "name": "crtdl", + "valueBase64Binary": "'"$base64_encoded"'" + } + ] +}') + +content_location=$(printf "%s" "$response" | grep -i 'Content-Location:' | awk '{print $2}' | tr -d '\r') + +if [ -n "$content_location" ]; then + printf "Extraction submitted, find your extraction under: %s$content_location\n" "$TORCH_BASE_URL" +else + printf "Content-Location header not found in the response.\n" +fi diff --git a/feasibility-triangle/torch/torch.nginx.conf b/feasibility-triangle/torch/torch.nginx.conf new file mode 100644 index 00000000..33a10d62 --- /dev/null +++ b/feasibility-triangle/torch/torch.nginx.conf @@ -0,0 +1,27 @@ +events{} +pid /tmp/nginx.pid; + +http { + server { + listen 8080; + server_name localhost; + + root /app/output; + + index index.html; + + location / { + try_files $uri $uri/ =404; + autoindex off; + } + + access_log /var/log/nginx/access.log; + error_log /var/log/nginx/error.log; + + error_page 404 /404.html; + location = /404.html { + internal; + } + } +} +