Skip to content

Latest commit

 

History

History
115 lines (92 loc) · 3.54 KB

README.md

File metadata and controls

115 lines (92 loc) · 3.54 KB

circulation-import

Set of tools facilitating the process of importing book collections into SimplyE's Circulation Manager by using SFTP protocol.

Architecture

circulation-import consists of two parts:

  • client responsible for uploading content to the SFTP server, waiting for a report, downloading it and converting it to CSV format
  • server responsible for watching for new book collections, importing them into CM using its directory_import script and uploading a report to the SFTP server

Picture below illustrates the architecture of the solution: circulation-import architecture

Another picture below contains a sequence diagram Import workflow

Usage

  1. Update all the submodules:
git submodule update --remote --recursive
cd circulation-lcp-test
git submodule update --remote --recursive
cd ..
  1. Run the LCP testbed:
docker-compose \
    --file circulation-lcp-test/docker-compose.yml \
    --env-file circulation-lcp-test/.env \
    up -d

  1. Follow the instructions in LCP testbed's README.md file to set it up

  2. Initialize required environment variables:

export CURRENT_USER_UID=$(id -u)
export CURRENT_USER_GID=$(id -g)
  1. Build the circulation-import server:
docker-compose \
    --file circulation-lcp-test/docker-compose.yml \
    --file docker-compose.yml \
    --env-file circulation-lcp-test/.env \
    build circulation-import-server
  1. Run the server:
docker-compose \
    --file circulation-lcp-test/docker-compose.yml \
    --file docker-compose.yml \
    --env-file circulation-lcp-test/.env \
    up -d
  1. Create and activate a virtual environment:
python -m venv .venv
source .venv/bin/activate
  1. Install circulation-import from PyPi:
pip install circulation-import
  1. Run the client:

⚠️ Please make sure that the books you're about to import were not already imported into Circulation Manager and have different identifiers. Otherwise, you might run into errors

python -m circulation-import client import \
    --collection-name=lcp \
    --data-source-name=data_source_1 \
    --books-directory=./circulation-lcp-test/lcp-collection/collection \
    --covers-directory=./circulation-lcp-test/lcp-collection/collection \
    --reports-directory=./reports \
    --metadata-file=./circulation-lcp-test/lcp-collection/collection/onix.xml \
    --metadata-format=onix \
    --configuration-file=./configuration/client-configuration.yml \
    --logging-configuration-file=./configuration/logging.yml
  1. Go to reports folder and find a report in CSV format

Troubleshooting

If you don't see the books you imported in Circulation Manager's dashboard you need to do the following:

  1. Update Elasticsearch index:
docker-compose \
    --file circulation-lcp-test/docker-compose.yml \
    --file docker-compose.yml \
    --env-file circulation-lcp-test/.env \
    exec circulation-import-server bash
    
source /var/www/circulation/env/bin/activate && /var/www/circulation/bin/search_index_refresh
  1. Delete cached feeds in Circulation Manager's database:
docker-compose \
    --file circulation-lcp-test/docker-compose.yml \
    --env-file circulation-lcp-test/.env \
    exec postgres bash
    
psql -Usimplified simplified_circulation_dev  # Please use correct credentials set up in the LCP testbed's .env file

truncate cachedfeeds cascade;