Implement filtering

[matrixbot]

This issue was originally created by @babolivier at matrix-org/dendrite#587.

This issue is tracking the progress of the implementation of filtering on the CS API. AFAIK @CromFr has already been working on implementing that on /sync. Will also need to be implemented on /messages when that endpoint itself gets implemented.

[matrixbot]

This comment was originally posted by @CromFr at matrix-org/dendrite#587 (comment).

Yay filters ! 🎉

My current filter-related PRs:

• Store & retrieve filters as structs rather than []byte #436 implements the use of filter objects when storing / retrieving them in the database
• State events filtering database api #438 and Account data events filtering database API #513 implements the filtering logic inside the event query functions (but uses default filters inside IncrementalSync/CompleteSync so the /sync behaviour is unchanged), respectively for state and account data events
• Filter /sync implementation #388 (closed) was my first attempt at implementing the filtering logic, all in one huge and messy PR.

I should have in my local repo some branches for other event types, and the final IncrementalSync/CompleteSync implementation, but requires some additional work to be sane & clean.

[matrixbot]

This comment was originally posted by @babolivier at matrix-org/dendrite#587 (comment).

Thanks for the info!
For the record, and for maximum clarity, this issue being created doesn't mean that filtering will be addressed immediately (it basically means that I needed an open issue to make a TODO comment that's referring to it in a PR I'm currently working on).
Hopefully there'll soon be enough bandwidth to deal with it, though.

[matrixbot]

This comment was originally posted by @kegsay at matrix-org/dendrite#587 (comment).

Sytests:

    × Can request federation format via the filter
    × Check creating invalid filters returns 4xx

[matrixbot]

This comment was originally posted by @kegsay at matrix-org/dendrite#587 (comment).

@neilalexander did you do this in matrix-org/dendrite#1721 ?

[matrixbot]

This comment was originally posted by @neilalexander at matrix-org/dendrite#587 (comment).

We have filtering in /sync but I don't think we have it in /messages yet.

[matrixbot]

This comment was originally posted by @spaetz at matrix-org/dendrite#587 (comment).

So, was that implemented by matrix-org/dendrite#2347 and can be closed then?

[matrixbot]

This comment was originally posted by @kegsay at matrix-org/dendrite#587 (comment).

There are still 2 outstanding tests which are failing.

[matrixbot]

This comment was originally posted by @kegsay at matrix-org/dendrite#587 (comment).

Still failing.