LiveTseverSet may not be handling watch removal #5157
Labels
Milestone
Comments
keith-turner
added
blocker
|
From what I can tell LiveTServerSet implements Watcher so that it can be passed to ZooCache and so ZooCache can call |
dlmarion
added a commit
to dlmarion/accumulo
that referenced
this issue
Dec 13, 2024
This commit removes most of the places where ZooCache
instances were being created in favor of re-using the
ZooCache from the ClientContext. Additionally, this
commit does not place a Watcher on each node that is
cached and instead places a single persistent
recursive Watcher at the paths in which the caching
is taking place.
This change roughly reduces the Watchers reported in
WatchTheWatchCountIT by 50%. While reducing the number
of Watchers, this commit could reduce ZooKeeper server
performance in two ways:
1. There is a note in the ZooKeeper javadoc for the
AddWatchMode enum that states there is a small
performance decrease when using recursive watchers
as all of the segments of ZNode paths need to be
checked for watch triggering.
2. Because a Watcher is not set on each node this
commit modified the ZooCache.ZCacheWatcher to
remove the parent of the triggered node, the
triggered node, and all of its siblings from the
cache. This overmatching may mean increased
lookups in ZooKeeper.
Related to apache#5134
Closes apache#5154, apache#5157
dlmarion
added a commit
to dlmarion/accumulo
that referenced
this issue
Jan 14, 2025
Reused ZooCache from client / server context where possible. Removed overloaded ZooCache constructor to make it easier to find where new instances are constructed. Removed watcher that was being placed in each call to the underlying ZooKeeper in favor of long-lived persistent recursive watchers set on specific paths which will fire when any child under that path is modified. Related to apache#5134 Closes apache#5154, apache#5157
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
The changes in #5143 introduced new behavior of removing watches in zoocache. ZooCache support passing in external watcher on construction. The only class that seems to utilize this feature is LiveTserverSet. LiveTseverSet may not be handling these changes to remove watches correctly.
Expected behavior
All code that passes a watcher to the zoocache constructor correctly handles watches being removed. More generally may want to evaluate if external code should pass a watcher to zoocache.
The text was updated successfully, but these errors were encountered: