Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send PIPs to Archivematica as BagIt bags #1007

Merged
merged 2 commits into from
Aug 22, 2024
Merged

Conversation

djjuhasz
Copy link
Collaborator

Fixes #805

  • Remove the bundle activity from the AM processing workflow
  • Change the package type to "zipped bag" when starting a transfer via
    the Archivematica API
  • Bag the PIP before sending it to Archivematica (if it's not already a
    bag)
  • Add a "TransferSourcePath" config value to specify the API path to the
    Transfer Source directory where PIPs are uploaded

@djjuhasz djjuhasz force-pushed the dev/issue-805-send-bag-to-am branch from 6985b40 to 0c97e80 Compare August 16, 2024 21:23
Copy link

codecov bot commented Aug 16, 2024

Codecov Report

Attention: Patch coverage is 85.00000% with 12 lines in your changes missing coverage. Please review.

Project coverage is 53.15%. Comparing base (17ae315) to head (6471106).
Report is 65 commits behind head on main.

Files with missing lines Patch % Lines
internal/workflow/processing.go 87.09% 4 Missing and 4 partials ⚠️
cmd/enduro-am-worker/main.go 0.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1007      +/-   ##
==========================================
+ Coverage   53.12%   53.15%   +0.03%     
==========================================
  Files         102      102              
  Lines        5835     5856      +21     
==========================================
+ Hits         3100     3113      +13     
- Misses       2478     2484       +6     
- Partials      257      259       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

@djjuhasz djjuhasz requested a review from jraddaoui August 16, 2024 21:30
@djjuhasz djjuhasz force-pushed the dev/issue-805-send-bag-to-am branch from 0c97e80 to f3de850 Compare August 19, 2024 23:03
Copy link
Collaborator

@jraddaoui jraddaoui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @djjuhasz, this looks great!

Some minor comments below and a bigger request after seeing the changes from both PRs (more related to #999 than to this one).

We decided at some point that preprocessing child workflows MUST return a valid Bag. If a preprocessing child workflow has been executed and the package is not identified as a Bag, we should fail the workflow instead of making it a Bag.

Check the following diagram that @sallain wrote a while ago, it doesn't include the identify activity but it tries to represent that, if a child workflow was run, we must have a valid Bag.

#805 (comment)

@djjuhasz djjuhasz requested a review from jraddaoui August 22, 2024 03:32
@djjuhasz
Copy link
Collaborator Author

@jraddaoui okay, I've addressed your comments and the workflow will now stop if preprocessing returns a SIP that is not a bag. I also had to fight with prettier a bit about formatting markdown, and made some changes to the preprocessing_test script to get the preprocessing test working and for my sanity trying to figure out where preservation tasks are being created and completed.

I'll squash all the changes before merging.

Copy link
Collaborator

@jraddaoui jraddaoui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @djjuhasz!

@djjuhasz djjuhasz force-pushed the dev/issue-805-send-bag-to-am branch 2 times, most recently from 849f194 to f249c48 Compare August 22, 2024 18:08
Fixes #805

- Move the bundle activity to the a3m branch of the processing workflow
- Change the package type to "zipped bag" when starting a transfer via
  the Archivematica API
- Bag the PIP before sending it to Archivematica (if it's not already a
  bag)
- Add a "TransferSourcePath" config value to specify the API path to the
  Transfer Source directory where PIPs are uploaded
After configuring prettier to use an indent of four spaces for Markdown
files it is adding a four space indent after the "-" character in
bulleted lists (see prettier/prettier#5019).

For now configure prettier to ignore Markdown files altogether to avoid
incorrect formatting changes.
@djjuhasz djjuhasz force-pushed the dev/issue-805-send-bag-to-am branch from f249c48 to 6471106 Compare August 22, 2024 18:12
@djjuhasz djjuhasz merged commit 0104a91 into main Aug 22, 2024
15 checks passed
@djjuhasz djjuhasz deleted the dev/issue-805-send-bag-to-am branch August 22, 2024 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature: send a BagIt bag to Archivematica for preservation
2 participants