-
Notifications
You must be signed in to change notification settings - Fork 135
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Notifications] Fix for ConcurrentModificationException while handling notification #12646
[Notifications] Fix for ConcurrentModificationException while handling notification #12646
Conversation
…Exception when ACTIVE_NOTIFICATIONS_MAP is handled by multiple threads.
…mSystemsBar method so only one thread has access to it at a time. The test pass now
Generated by 🚫 Danger |
📲 You can test the changes from this Pull Request in WooCommerce-Wear Android by scanning the QR code below to install the corresponding build.
|
📲 You can test the changes from this Pull Request in WooCommerce Android by scanning the QR code below to install the corresponding build.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## trunk #12646 +/- ##
============================================
+ Coverage 40.63% 40.64% +0.01%
- Complexity 5670 5673 +3
============================================
Files 1230 1230
Lines 69165 69165
Branches 9579 9579
============================================
+ Hits 28104 28114 +10
+ Misses 38479 38466 -13
- Partials 2582 2585 +3 ☔ View full report in Codecov by Sentry. |
@@ -237,6 +240,7 @@ class NotificationMessageHandler @Inject constructor( | |||
updateNotificationsState() | |||
} | |||
|
|||
@Synchronized | |||
fun removeNotificationByNotificationIdFromSystemsBar(localPushId: Int) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could use ConcurrentHashMap
instead of MutableMap
and remove the @Synchronized
methods, and that would work too. However, given that concurrent access to notifications is infrequent and only happens during specific notification-related events, I opted to use the @Synchronized
annotation for simplicity and minimal overhead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the extra info! I agree, additional argument for using synchronized is that we used to use it before this PR which likely introduced this crash.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would suggest to go with the thread-safe map:
- That's probably faster as we don't lock the whole object, but this is not really important
- More important, I think it is harder to break, because those @synchronized here are not really clear why we have them here, and I think that's why they were removed. Alos, removing just 1 synchronized from any public method will return the crash, or the worst, adding a new public method that touches the map will also cause the same issue again
Wdyt?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@AnirudhBhat also kudos for the unit test!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kidinov Thanks for the input and you raise a valid point here. My train of though was that
- Adding @synchronized annotation would make it explicit that we are dealing with concurrency here and need to be careful of the modifications.
- While
ConcurrentHashMap
is thread safe in general. Some operation do require careful consideration before usage - especially operations that checks existing key and then update its value as these are not atomic and need to be refactored to use other methods likeputIfAbsent
.
Overall. I agree with you that with this approach, it's easy to make a mistake.I don't have a strong preference, and I can replace this implementation with ConcurrentHashMap
in a separate PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I looked up the code again, and actually, we already have public methods that don't have synchronised on them and that modify the map
fun removeAllNotificationsFromSystemsBar() {
clearNotifications()
notificationBuilder.cancelAllNotifications()
}
fun markNotificationsOfTypeTapped(type: NotificationChannelType) {
ACTIVE_NOTIFICATIONS_MAP.asSequence()
.filter { it.value.channelType == type }
.forEach { row ->
analyticsTracker.trackNotificationAnalytics(PUSH_NOTIFICATION_TAPPED, row.value)
analyticsTracker.flush()
}
}
So maybe the crash still can happen even now
Also, interestingly, even during writing the original code, it was considered that concurrent modification is possible, but no generic solution was applied
// Using a copy of the map to avoid concurrency problems
ACTIVE_NOTIFICATIONS_MAP.toMap().asSequence().forEach { row ->
ah, and ACTIVE_NOTIFICATIONS_MAP
is not a constant, so it doesn't follow the naming convention
So overall yes, I'd suggest:
- Use concurrent map
- Rename it to follow the convention
- Remove unnecessary copies of the map
- Maybe extend the test to call all the public methods to validate the solution
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @AnirudhBhat for working on this and providing detailed description! Great job reproducing it in a unit test, that gives me enough confidence these changes actually fix the issue.
I've smoke tested push notifications and they work as expected.
Closes: #12636
Description
This PR addresses a crash peaMlT-TT-p2 related to notifications. While I couldn't reproduce the crash directly, I was able to analyze the stacktrace, identify the root cause, and reproduce the issue through a unit test.
The crash occurred because
ACTIVE_NOTIFICATIONS_MAP
was being modified concurrently by multiple threads, leading to aConcurrentModificationException
.For example - In
ReviewDetailViewModel
, while loading product reviews, the map is modified on a background thread whenmarkReviewAsSeen(remoteReviewId, it)
is called. Simultaneously, if a new notification arrives or the user dismisses an existing notification, we attempt to modify the sameACTIVE_NOTIFICATIONS_MAP
, which could lead to aConcurrentModificationException
.The issue is resolved by adding the
@Synchronized
annotation, ensuring that only one thread can modifyACTIVE_NOTIFICATIONS_MAP
at a time. After applying this fix, the unit test passes successfully.Steps to reproduce
@Synchronized
annotation fromremoveNotificationByNotificationIdFromSystemsBar
methodremove notifications concurrently without throwing ConcurrentModificationException
unit testConcurrentModificationException
Testing information
I couldn't test this with real notifications, but the fact that the unit test passes after the fix provides enough confidence to assert that the
ConcurrentModificationException
crash has been resolved for this particular scenario.RELEASE-NOTES.txt
if necessary. Use the "[Internal]" label for non-user-facing changes.Reviewer (or Author, in the case of optional code reviews):
Please make sure these conditions are met before approving the PR, or request changes if the PR needs improvement: