Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: validate messages for individual filter nodes & perform renewals #2057

Merged
merged 13 commits into from
Jul 17, 2024

Conversation

danisharora099
Copy link
Collaborator

Problem

We (may) subscribe to multiple peers with the Filter protocol, over the same content topics, in order to increase the reliability/probability of the message being received by our node. However, the nodes that aren't sending the desired messages aren't being acted upon, and they continue to stay in the connection pool.

Solution

Renew the nodes that aren't sending us the messages, while the other nodes do.

  • run a recurring interval to validate all the messages received yet, against messages that each node send
  • renew, and resubscribe, with the new peer that did not send us a message, while others did

Notes

Contribution checklist:

  • covered by unit tests;
  • covered by e2e test;
  • add ! in title if breaks public API;

@danisharora099 danisharora099 requested a review from a team as a code owner July 10, 2024 11:16
Copy link

github-actions bot commented Jul 10, 2024

size-limit report 📦

Path Size Loading time (3g) Running time (snapdragon) Total time
Waku node 183.73 KB (+0.18% 🔺) 3.7 s (+0.18% 🔺) 3.2 s (-19.53% 🔽) 6.9 s
Waku Simple Light Node 183.91 KB (+0.21% 🔺) 3.7 s (+0.21% 🔺) 3.7 s (-16.55% 🔽) 7.3 s
ECIES encryption 23.12 KB (0%) 463 ms (0%) 911 ms (-46.05% 🔽) 1.4 s
Symmetric encryption 22.58 KB (0%) 452 ms (0%) 1.2 s (+21.83% 🔺) 1.7 s
DNS discovery 72.49 KB (0%) 1.5 s (0%) 3.5 s (+20.94% 🔺) 4.9 s
Peer Exchange discovery 74.15 KB (0%) 1.5 s (0%) 2.4 s (-32.83% 🔽) 3.9 s
Local Peer Cache Discovery 67.68 KB (0%) 1.4 s (0%) 3.2 s (+42.5% 🔺) 4.5 s
Privacy preserving protocols 38.87 KB (0%) 778 ms (0%) 1.9 s (+27.52% 🔺) 2.7 s
Waku Filter 113.3 KB (+0.37% 🔺) 2.3 s (+0.37% 🔺) 4.2 s (+39.81% 🔺) 6.5 s
Waku LightPush 111.3 KB (+0.09% 🔺) 2.3 s (+0.09% 🔺) 3.1 s (+22.05% 🔺) 5.3 s
History retrieval protocols 111.78 KB (+0.05% 🔺) 2.3 s (+0.05% 🔺) 3.4 s (-26.97% 🔽) 5.6 s
Deterministic Message Hashing 7.29 KB (0%) 146 ms (0%) 786 ms (-18.74% 🔽) 931 ms

@@ -326,7 +427,7 @@ class FilterSDK extends BaseProtocolSDK implements IFilterSDK {
) {
super(
new FilterCore(
async (pubsubTopic: PubsubTopic, wakuMessage: WakuMessage) => {
async (pubsubTopic, wakuMessage, peerIdStr) => {
const subscription = this.getActiveSubscription(pubsubTopic);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, I just noticed - we keep only one subscription per pubsubTopic.
That mean that if another one will be created in the application - first one would be dropped.

Also we probably still have this one #1678

Let's follow up if I am not wrong

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No no. We have the ability to have subscriptions with multiple pubsub topics -- each pubsub topic is represented by a new SubscriptionManager object.

The line of code you are reading is actually a callback function:

handleIncomingMessage: (pubsubTopic: PubsubTopic, wakuMessage: WakuMessage, peerIdStr: string) => Promise<void>

TL;DR: We can indeed subscribe to multiple pubsub topics

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh right, there is a fork here -

const subscription =

if a subscription exists - then we return it which will be used for subscribe request

return [...this.receivedMessagesHashes.all];
}

addHash(hash: string, peerIdStr?: string): void {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: let's not forget access modifiers

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. Added to this PR part of 128dd58

I believe we should introduce it into eslint config as well so it's not skipped

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened a PR adding the rule to eslint: #2068

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool!

@@ -35,7 +35,8 @@ export class FilterCore extends BaseProtocol implements IBaseProtocolCore {
constructor(
private handleIncomingMessage: (
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we make it `private handleIncomingMessage: (options: IncomingMessageOptions) => Promise

@danisharora099 danisharora099 requested a review from a team July 17, 2024 10:10
@danisharora099 danisharora099 merged commit 9b0f1e8 into master Jul 17, 2024
10 of 11 checks passed
@danisharora099 danisharora099 deleted the feat/filter-validation-renewal branch July 17, 2024 19:11
@weboko weboko mentioned this pull request Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants