Skip to content
This repository has been archived by the owner on Nov 25, 2024. It is now read-only.

syncserver: double-check backfills for duplicates #932

Open
kegsay opened this issue Mar 20, 2020 · 0 comments
Open

syncserver: double-check backfills for duplicates #932

kegsay opened this issue Mar 20, 2020 · 0 comments
Labels
C-Sync-API T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues.

Comments

@kegsay
Copy link
Member

kegsay commented Mar 20, 2020

When a client hits /messages, we try to get earlier messages from the DB. If we can't get enough, we backfill from other servers. Whenever we hit other servers, we need to make sure we protect ourselves from their bugs/problems.

If a remote server sends a duplicate event (same event ID/depth) for whatever reason, we insert the event into the syncapi_output_room_events_topology table, which correctly de-dupes based on ON CONFLICT (topological_position, room_id) DO UPDATE SET event_id = $1, but the request that initiated the backfill receives the duplicate event.

When we know we need to backfill, we should do so and then re-run the /messages request to pull from the DB, rather than return whatever events the remote server gave us, as the DB has extra checks in place like the aforementioned de-dupe logic.

This pattern of "hit remote servers to get data then re-run our internal logic which we trust" is probably something we should do in more places too!

@kegsay kegsay added T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. and removed bug labels Dec 5, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
C-Sync-API T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues.
Projects
None yet
Development

No branches or pull requests

1 participant