Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 3589 - Split file reference store interface into queries and updates #3590

Merged
merged 1 commit into from
Nov 5, 2024

Conversation

patchwork01
Copy link
Collaborator

@patchwork01 patchwork01 commented Oct 31, 2024

Make sure you have checked all steps below.

Issue

Tests

  • My PR adds the following tests OR does not need testing for this extremely good reason:
    • Covered by existing tests

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it, or I have linked to a
    separate issue for that below.

@patchwork01 patchwork01 added one-review-required pr-stacked-top A stacked pull request that we don't want to merge into its target until the target PR is merged labels Oct 31, 2024
Base automatically changed from 3587-snapshots-package to develop November 1, 2024 10:25
@patchwork01 patchwork01 removed the pr-stacked-top A stacked pull request that we don't want to merge into its target until the target PR is merged label Nov 1, 2024
* @return a list of {@link FileReference}s on the partition
* @throws StateStoreException if query fails
*/
default boolean isPartitionFilesAssignedToJob(String partitionId, List<String> filenames, String jobId) throws StateStoreException {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trivial point as copied, doesn't read as good English for method name due to plural in files.

Suggestion: arePartitionFilesAssignedToJob

Change not necessary, just raised for discussion as understand why starts with is as well

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other options would be partitionFilesAreAssignedToJob, or isEachPartitionFileAssignedToJob, or replace the parameters with an object which describes the condition and make the method name shorter. All the options seem better than how it is now, now that you mention it!

Copy link
Collaborator

@rtjd6554 rtjd6554 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes acceptable, small suggestion but purely aesthetic

@gaffer01
Copy link
Member

gaffer01 commented Nov 4, 2024

I'm not sure this change adds any value. Any implementation of file reference store needs to support both updates and queries. The hard part is making sure that updates and queries can happen at the same time whilst giving consistent results. Even if we had a state store where queries ran against a different store, that was effectively a cache of the main store, which was updated from the other store that was used for updates, the hard part is still making that correct, and splitting the functionality up so that it is in separate classes implementing separate interfaces seems to me to not be helpful.

@patchwork01
Copy link
Collaborator Author

I'm not sure this change adds any value. Any implementation of file reference store needs to support both updates and queries. The hard part is making sure that updates and queries can happen at the same time whilst giving consistent results. Even if we had a state store where queries ran against a different store, that was effectively a cache of the main store, which was updated from the other store that was used for updates, the hard part is still making that correct, and splitting the functionality up so that it is in separate classes implementing separate interfaces seems to me to not be helpful.

This isn't about splitting up the implementation, although I do personally think that would be more flexible. The implementations still extend StateStore and FileReferenceStore, rather than the new interfaces.

This is just for readability. A number of times I've wanted to look at the queries that are implemented against files in the system, and it's been quite difficult to find all of them with any confidence.

@gaffer01 gaffer01 assigned patchwork01 and unassigned gaffer01 Nov 5, 2024
@patchwork01 patchwork01 merged commit a5e90da into develop Nov 5, 2024
8 checks passed
@patchwork01 patchwork01 deleted the 3586-file-reference-table-from-transactions branch November 5, 2024 14:26
@patchwork01 patchwork01 changed the title Issue 3586 - Split file reference store interface into queries and updates Issue 3589 - Split file reference store interface into queries and updates Nov 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Split file reference store interface into queries and updates
4 participants