DR2-1956 ADR for replacing datasync #386
Conversation
There are now 3 PRs with a 0021 ADR. We'll have to change them based on the order they're merged.
MancunianSam
changed the title
MancunianSam
changed the title
| We will replace the use of AWS DataSync with a mechanism where each Lambda function directly transfers files to Preservica’s S3 bucket. This will involve: | ||
|
|
||
| 1. **Updating Lambda Roles**: | ||
| Each Lambda will continue to use its own IAM role with scoped-down permissions based on its function. |
There was a problem hiding this comment.
You put here "Each Lambda will continue to use its own IAM role" but previously, you mentioned there were all using a "single IAM role"; should we remove "continue to"?
There was a problem hiding this comment.
The single IAM role was for Datasync, not for the lambdas. They each had their own role originally because they didn't need to access Preservica's bucket.
| Each Lambda will continue to use its own IAM role with scoped-down permissions based on its function. | ||
|
|
||
| 2. **Role Assumption for Cross-Account Access**: | ||
| To access Preservica’s bucket, each Lambda will assume a designated IAM role (shared across all Lambdas) that has the necessary `s3:PutObject` permissions on Preservica’s bucket. This role exists solely for the purpose of cross-account transfers. |
There was a problem hiding this comment.
In some cases in this ADR, "lambda" (by that, I mean the object that is run, not the service) is upper case and sometimes it's lower case.
There was a problem hiding this comment.
Good point. I've changed all of them to lower case except when they're part of Lambda-to-Preservica, in a title or at the beginning of a sentence.
|
|
||
| ### Assumptions | ||
| - Preservica’s bucket will continue to allow access via the shared IAM role. | ||
| - Lambda execution time and resource limits will not pose a problem for the largest files transferred. |
There was a problem hiding this comment.
Maybe "for the large files transfers" or "for the larger file transfers"?
There was a problem hiding this comment.
"for the large files transfers" is good.
|
Also, should the file name of the ADR be a bit more descriptive, e.g. |
…com/nationalarchives/dr2-ingest into DR2-1956-adr-for-replacing-datasync
No description provided.