Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Issue #3553] Modify local s3 setup in factories to be easier to use #3574

Merged
merged 6 commits into from
Jan 21, 2025

Conversation

chouinar
Copy link
Collaborator

Summary

Fixes #3553

Time to review: 10 mins

Changes proposed

Modified our attachment factory to create files on s3 automatically

Context for reviewers

We had a lot of tests and local setup assume files would already be pre-generated that the opportunity attachment table would point to. Those often were wrong and broke.

This fixes things by instead making the factories create s3 files for you. It does this by taking the path you gave it, and making a file at that location. This does make assumptions that your s3 bucket exists, but I piggybacked on the enable_factory_create pytest fixture we have that already requires you call it to call any DB operation with create, so that shouldn't be too much of an issue. I've set it up so the factory by default now creates no attachments on an opportunity, but can be told to add a few because the performance degradation on running tests was pretty bad.

Additional information

Running make db-seed-local works. You can see the files it created with aws --endpoint-url http://localhost:4566 s3 ls local-mock-public-bucket --recursive

The tests all work with just a few minor fixes.

@chouinar chouinar requested a review from mdragon as a code owner January 17, 2025 18:53
@chouinar chouinar requested review from mikehgrantsgov and babebe and removed request for mdragon January 17, 2025 18:53
@@ -373,7 +349,7 @@ def mock_s3(reset_aws_env_vars):

@pytest.fixture
def mock_s3_bucket_resource(mock_s3):
bucket = mock_s3.Bucket("test_bucket")
bucket = mock_s3.Bucket("local-mock-public-bucket")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

description here for when to use one vs the other would help.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're free to use either however you see fit - there's not any real specific guidance. I can add a note to the other one to say that it exists because in some cases we want to test across buckets.

Copy link
Collaborator

@babebe babebe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. This will definitely make it easier to create attachments!!! Also, much easier to get s3 session to work locally. We can call it directly when signing the url as well.

Copy link
Collaborator

@babebe babebe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chouinar chouinar merged commit 58c8a2a into main Jan 21, 2025
2 checks passed
@chouinar chouinar deleted the chouinar/fix-attachment-locally branch January 21, 2025 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate local s3 setup to see if there is a way for the factories to create the files they work with
2 participants