-
-
Notifications
You must be signed in to change notification settings - Fork 224
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TASK: Use individual transactions in subscription engine and use session lock via GET_LOCK
#5383
Conversation
using many small transactions has two advantages: - on a replay with many events, we dont build up a huge transaction that the database has to possible extract into files at some point. Otherwise, we would need to introduce batching and need to require the for update lock - to ensure integrity - that the projection got exactly that many events how it is in the subscription store - each applied event per projection has its own transaction. If something goes really wrong (php crashes) the transaction will not be committed. This is important for the case of using a different connection or a different database for a projection, as otherwise a crash of another projection would lead to a external projection being up-to-date while the subscription update is fully rolled back. later we can introduce a concept of batching possibly - even though it must be patched per subscriber and not mixed to ensure integrity in fatal errors.
…tion is persisted. Even in error case.
we will implement a proper wait mechanism later. Previously that test worked in the subscription branch as we used a FOR UPDATE lock which waits by default instead of using NOWAIT. To not put the load on the database in `GET_LOCK` we call it with no wait (0 timeout)
7da6d1f
to
22c3d3e
Compare
GET_LOCK
GET_LOCK
Initially christian and me wanted to reimplement the current 9.0 behaviour just with subscriptions: #5321 (comment) The idea was to do not do any crazy behaviour changes as that would allow us to get the change in faster and also measure performance later if we want to change anything. That turned out more complex (again :D). So the idea was get - without lock - the lowest position. Fetch the event stream from there, use a lot of for update transactions for each projection and each event like we do currently. Problems with that approach:
Thus this pr now is a compromise. It uses central locking via |
CASE: $this->dbal->isTransactionActive() === false
After discussing with Bastian again we opted against this change. Instead it will be replaced by #5392 Main motivation - as will be further outlined in the main pr - is that the |
Adjustments for #5321
1.) Introduce dedicated locking and use small transactions
using many small transactions has two advantages:
later we can introduce a concept of batching possibly - even though it must be patched per subscriber and not mixed to ensure integrity in fatal errors.
2.) Ensure
onAfterCatchUp
is always executed after the projection is persisted. Even in error case.3.) Deactivate
ParallelWritingInWorkspacesTest
test for nowwe will implement a proper wait mechanism later -> see #5376
Previously that test worked in the subscription branch as we used a FOR UPDATE lock which waits by default instead of using NOWAIT.
To not put the load on the database in
GET_LOCK
we call it with no wait (0 timeout)Upgrade instructions
Review instructions
Checklist
FEATURE|TASK|BUGFIX
!!!
and have upgrade-instructions