AO3-5578 ActiveStorage copy script performance fixes #5015
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
brianjaustin
added
the
Priority: High - Broken on Test
Bilka2
reviewed
Jan 7, 2025
lib/tasks/after_tasks.rake
Outdated
| end | ||
|
|
||
| puts "Finished up to ID #{collection.id}" if collection.id.modulo(100).zero? | ||
| puts "Finished batch #{index + 1}" |
There was a problem hiding this comment.
I think because you sleep right after, it would be good to have a $stdout.flush so that this definitely shows up in the output immediately.
And if you want to be fancy you can do the math for the total batch number that the other find_in_batches tasks do, but it's definitely not necessary.
There was a problem hiding this comment.
I would not want to do the count for Pseud, it's large enough that I want to touch the table as little as possible (even things that are relatively safe)
redsummernight
approved these changes
Jan 7, 2025
slavalamp
pushed a commit
to slavalamp/otwarchive
that referenced
this pull request
Jan 12, 2025
* AO3-5578 ActiveStorage copy script performance fixes * Flush $stdout to force message visibility
brianjaustin
added a commit
that referenced
this pull request
Jan 22, 2025
) * AO3-5502 correct page title on adult content warning chapters * AO3-5502 add Cucumber test * AO3-5502 satisfy Rubocop * AO3-5578 ActiveStorage copy script performance fixes (#5015) * AO3-5578 ActiveStorage copy script performance fixes * Flush $stdout to force message visibility * AO3-6873 Upgrade Elasticsearch for dev and automated tests (#5016) * AO3-5578 Avoid unnecessary DB writes when copying icons (#5017) * AO3-5578 Avoid unnecessary DB writes when copying icons * Rubocop :fistshake: * AO3-5578 Add a transaction to Psued image migration (#5018) Add a transaction * AO3-5578 Try to be nicer to the database when copying icons (#5019) * Test listing in s3 * Other tasks & fixes * Experiment with delayed upload * Fixes * Upload after txn * Rubocop things * Fixes * Avoid duplicate attachments * Revert "Avoid duplicate attachments" This reverts commit 476bd02. * AO3-5502 satisfy Rubocop again...? * AO3-5502 satisfy more Rubocop * AO3-5502 formatting improvements * AO3-5502 make sure it's the adult notice page in the test * AO3-5502 add another indent i forgot --------- Co-authored-by: Brian Austin <[email protected]> Co-authored-by: Bilka <[email protected]> Co-authored-by: james_ <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Checklist
Issue
https://otwarchive.atlassian.net/browse/AO3-5578
Purpose
Update the icon copy tasks to try to be nicer to the prod database (in particular the pseuds table).