-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issue 3589 - Split file reference store interface into queries and updates #3590
Issue 3589 - Split file reference store interface into queries and updates #3590
Conversation
* @return a list of {@link FileReference}s on the partition | ||
* @throws StateStoreException if query fails | ||
*/ | ||
default boolean isPartitionFilesAssignedToJob(String partitionId, List<String> filenames, String jobId) throws StateStoreException { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Trivial point as copied, doesn't read as good English for method name due to plural in files.
Suggestion: arePartitionFilesAssignedToJob
Change not necessary, just raised for discussion as understand why starts with is as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other options would be partitionFilesAreAssignedToJob, or isEachPartitionFileAssignedToJob, or replace the parameters with an object which describes the condition and make the method name shorter. All the options seem better than how it is now, now that you mention it!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes acceptable, small suggestion but purely aesthetic
I'm not sure this change adds any value. Any implementation of file reference store needs to support both updates and queries. The hard part is making sure that updates and queries can happen at the same time whilst giving consistent results. Even if we had a state store where queries ran against a different store, that was effectively a cache of the main store, which was updated from the other store that was used for updates, the hard part is still making that correct, and splitting the functionality up so that it is in separate classes implementing separate interfaces seems to me to not be helpful. |
This isn't about splitting up the implementation, although I do personally think that would be more flexible. The implementations still extend StateStore and FileReferenceStore, rather than the new interfaces. This is just for readability. A number of times I've wanted to look at the queries that are implemented against files in the system, and it's been quite difficult to find all of them with any confidence. |
Make sure you have checked all steps below.
Issue
PR"
Tests
Documentation
separate issue for that below.