diff --git a/.gitignore b/.gitignore index 9c2a94050926..3ea363fbc4a8 100644 --- a/.gitignore +++ b/.gitignore @@ -10,7 +10,7 @@ dspace/src/main/docker/nginx/certs/* # Ignore database dump directory to be used with docker-compose for initializing postgres-init/* !postgres-init/README.md -!postgres-init/pg_restore_drum.sh +!postgres-init/pg_restore.sh ## End UMD Customization ## Ignore the MVN compiled output directories from version tracking diff --git a/README-DRUM.md b/README-DRUM.md index 488903782fb6..9caf34d243d0 100644 --- a/README-DRUM.md +++ b/README-DRUM.md @@ -9,7 +9,7 @@ The original DSpace documentation is in the "README.md" file. ## Development Environment Instructions for building and running drum locally can be found in -[dspace/docs/Drum7DockerDevelopmentEnvironment.md](/dspace/docs/Drum7DockerDevelopmentEnvironment.md) +[dspace/docs/DockerDevelopmentEnvironment.md](/dspace/docs/DockerDevelopmentEnvironment.md) ## Building Images for K8s Deployment diff --git a/dspace/config/local.cfg.EXAMPLE b/dspace/config/local.cfg.EXAMPLE index 1ce1f75317a1..dd8f1dfced4f 100644 --- a/dspace/config/local.cfg.EXAMPLE +++ b/dspace/config/local.cfg.EXAMPLE @@ -354,12 +354,6 @@ loglevel.dspace = INFO ############################ webui.user.assumelogin = true -# Creative Commons settings -cc.api.rooturl = https://api.creativecommons.org/rest/1.5 -# A list of license classes that should be excluded from selection process -# Excluding "publicdomain" because it is now a duplicate of the CC0 license -cc.license.classfilter = recombo, mark, publicdomain - # Browse configuration webui.browse.index.1 = dateissued:item:dateissued webui.browse.index.2 = author:metadata:dc.contributor.author:text diff --git a/dspace/docs/Drum7DockerDevelopmentEnvironment.md b/dspace/docs/DockerDevelopmentEnvironment.md similarity index 94% rename from dspace/docs/Drum7DockerDevelopmentEnvironment.md rename to dspace/docs/DockerDevelopmentEnvironment.md index 9ba256e551e2..deac22e53d84 100644 --- a/dspace/docs/Drum7DockerDevelopmentEnvironment.md +++ b/dspace/docs/DockerDevelopmentEnvironment.md @@ -1,4 +1,4 @@ -# DRUM 7 Docker Development Environment +# Docker Development Environment This document contains instructions for building a local development instance of a DSpace 7-based DRUM using Docker. @@ -51,8 +51,8 @@ and add the following lines: ## Development Setup -This repository uses the "GitHub Flow" branching model, with "drum-main" as the -main branch for DRUM development. +This repository uses the "GitHub Flow" branching model, with "drum-main" as +the main branch for DRUM development. 1) Clone the Git repository and switch to the directory: @@ -79,23 +79,24 @@ main branch for DRUM development. 4) Edit the local configuration file: - ```zsh - $ vi dspace/config/local.cfg - ``` + ```zsh + $ vi dspace/config/local.cfg + ``` - and enter values for the following properties: + and enter values for the following properties: - * drum.ldap.bind.auth - * drum.ldap.bind.password + * drum.ldap.bind.auth + * drum.ldap.bind.password - The appropriate values can be found in LastPass. + The appropriate values can be found in LastPass. - **Note:** The "drum.ldap.bind.auth" value typically contains commas (for - example "uid=foo,cn=bar,ou=baz,dc=quuz,dc=zot"), which must be escaped. So - the actual value added to the file would be similar to - `uid=foo\,cn=bar\,ou=baz\,dc=quuz\,dc=zot`. + **Note:** The "drum.ldap.bind.auth" value typically contains commas (for + example "uid=foo,cn=bar,ou=baz,dc=quuz,dc=zot"), which must be escaped. So + the actual value added to the file would be similar to + `uid=foo\,cn=bar\,ou=baz\,dc=quuz\,dc=zot`. -5) Follow the instructions at [dspace/docs/DrumDBRestore.md](DrumDBRestore.md) +5) Follow the instructions at + [dspace/docs/DrumDBRestore.md](DrumDBRestore.md) to populate the Postgres database with a DSpace 7 database dump from Kubernetes. @@ -116,7 +117,6 @@ main branch for DRUM development. ```zsh # Build the dspace image $ docker compose -f docker-compose.yml build - ``` 8) Start all the containers @@ -155,6 +155,9 @@ $ docker compose -p d7 up -d dspacedb dspacesolr nginx $ docker compose -p d7 up dspace ``` +Once the REST API starts, it should be accessible at + + ## Visual Studio Code IDE Setup The following is the suggested setup for Visual Studio Code for DSpace diff --git a/dspace/docs/DrumConfigurationCustomization.md b/dspace/docs/DrumConfigurationCustomization.md index 7f6e0da1eaf4..05e16f18d2e7 100644 --- a/dspace/docs/DrumConfigurationCustomization.md +++ b/dspace/docs/DrumConfigurationCustomization.md @@ -21,7 +21,7 @@ In general, changes *should not* be made to this file. DRUM customization should be added to the "dspace/config/local.cfg.EXAMPLE" file, which, when deployed, should be copied to "dspace/config/local.cfg" (see -the [DRUM 7 Docker Development Environment](Drum7DockerDevelopmentEnvironment.md). +the [Docker Development Environment](DockerDevelopmentEnvironment.md). ## XML File Changes diff --git a/dspace/docs/DrumDBRestore.md b/dspace/docs/DrumDBRestore.md index 0d01acd96e6c..8737abc0ac22 100644 --- a/dspace/docs/DrumDBRestore.md +++ b/dspace/docs/DrumDBRestore.md @@ -19,11 +19,11 @@ with DRUM running DSpace 7. ``` **Note:** The output file MUST use a ".dump" extension, in order for the - "pg_restore_drum.sh" script to process it, see + "pg_restore.sh" script to process it, see "[postgres-init/README.md](../../postgres-init/README.md)" 3) (Optional) This step can be skipped, if you are following the instructions in - [dspace/docs/Drum7DockerDevelopmentEnvironment.md](Drum7DockerDevelopmentEnvironment.md). + [dspace/docs/DockerDevelopmentEnvironment.md](DockerDevelopmentEnvironment.md). Start the "dspacedb" container and wait for the restore to complete. diff --git a/postgres-init/README.md b/postgres-init/README.md index b51b44095ce1..c098f149023f 100644 --- a/postgres-init/README.md +++ b/postgres-init/README.md @@ -1,9 +1,9 @@ # Database initialization for docker-compose Place your database dump file in this directory, and the "dspacedb" container -will use it to initialize the database, via the "pg_restore_drum.sh" script. +will use it to initialize the database, via the "pg_restore.sh" script. -## pg_restore_drum.sh and CVE-2023-2454 +## pg_restore.sh and CVE-2023-2454 Prior to DSpace 7.6.2, it was possible to use "pg_dump" to create a "plain" SQL file (with a ".sql" extension), and the "postgres" Docker image @@ -16,12 +16,12 @@ process running as the "postgres" user, and the use of the "pgcrypto" extension). Note that plain SQL dumps *will* work when run from the Docker container's command-line. -The workaround was to create the "postgres-init/pg_restore_drum.sh", script +The workaround was to create the "postgres-init/pg_restore.sh", script which uses "pg_restore" to populate the database with a Postgres "custom" dump file. This is used in preference to a plain SQL file, to keep the automatic Docker initialization from interfering with the process. -The "postgres-init/pg_restore_drum.sh" will run automatically when the Docker +The "postgres-init/pg_restore.sh" will run automatically when the Docker container starts, *if* a database doesn't already exist. ## Creating the Postgres dump @@ -33,7 +33,34 @@ $ kubectl exec drum-db-0 -- pg_dump -Fc -C -O -U drum -d drum > postgres-init/dr ``` **Note:** The output file MUST use a ".dump" extension, in order for the -"pg_restore_drum.sh" script to process it. +"pg_restore.sh" script to process it. + +## (Optional) Verifying the Postgres dump + +Use of the custom dump format is slightly risky because the Kubernetes +connection might have terminated before completing the dump. + +Assuming that Postgres is installed on the local workstation, one way to +verify that the dump is complete, is to convert the database dump into a +plain SQL file (such as "verify-db.sql") by running: + +```zsh +$ pg_restore -f verify-db.sql postgres-init/drum-db.dump +``` + +and then running the "tail" command against the plain SQL file: + +```zsh +$ tail verify-db.sql +``` + +If the dump is complete, the last lines will be: + +```text +-- +-- PostgreSQL database dump complete +-- +``` ## Delete local database data diff --git a/postgres-init/pg_restore_drum.sh b/postgres-init/pg_restore.sh similarity index 86% rename from postgres-init/pg_restore_drum.sh rename to postgres-init/pg_restore.sh index 2e1b553405de..2f023b7941d7 100755 --- a/postgres-init/pg_restore_drum.sh +++ b/postgres-init/pg_restore.sh @@ -8,4 +8,4 @@ SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) cd $SCRIPT_DIR # Run pg_restore against any file in the directory with a ".dump" extension -find . -name '*.dump' -exec pg_restore --username="$POSTGRES_USER" --no-owner --dbname=drum --verbose {} \; +find . -name '*.dump' -exec pg_restore --username="$POSTGRES_USER" --no-owner --dbname="$POSTGRES_DB" --verbose {} \;