[improve][pip] PIP-380: Support setting up specific namespaces to skipping the load-shedding

[Demogorgon314]

No description provided.

[lhotari]

It seems fine. Just thinking that there could be similar requirements for user defined namespaces and therefore having a property in the namespace policy to control this would be useful. Instead of a single binary value, perhaps it could be some type of value that would be used for ordering the namespace bundles that are picked as candidates for load shedding.
A system namespace could use a value that would result in all other namespace bundles being picked before it becomes a candidate for load shedding.

[heesung-sn]

I wonder if we can limit this blacklisting to the TransferShedder under loadBalanceTransferShedderExcludedNamespaces config .

[Demogorgon314]

@heesung-sn I would like to use the loadBalancerSheddingExcludedNamespaces config to support ExtensibleLoadManager and ModularLoadManager

[heesung-sn]

I think we probably want to explore namespace policy like this work.

#23120

[Demogorgon314]

Adding a namespace policy is a good idea, let me update the PIP.

[Demogorgon314]

@lhotari Are you suggesting adding a loadSheddingPriority to the namespace police? I'm wondering which case will use the priority for load shedding. And when using priority for each namespace will make the load-shedding strategy let bit complex, since when we select the hot bundle to unload we have to consider the namespace priority. I prefer to use loadSheddingEnabled, then for the protocol's system topic and some user's benchmark topic we can make the ownership stable.

[Demogorgon314]

ping @lhotari

[lhotari]

PIP-379 is already taken, I increased the number to PIP-380. The way to find the next available number is to check the dev mailing list and open PRs. The first one to open a discussion for a specific PIP number takes it. This is the PIP-379 discussion: https://lists.apache.org/thread/l5zjq0fb2dscys3rsn6kfl7505tbndlx . I somehow missed your PR when I made the PIP number assignment. I'm sorry about that.

[lhotari]

LGTM

[lhotari]

@lhotari Are you suggesting adding a loadSheddingPriority to the namespace police? I'm wondering which case will use the priority for load shedding. And when using priority for each namespace will make the load-shedding strategy let bit complex, since when we select the hot bundle to unload we have to consider the namespace priority. I prefer to use loadSheddingEnabled, then for the protocol's system topic and some user's benchmark topic we can make the ownership stable.

@Demogorgon314 It was something to consider for the future.