✨Collaboration long polling fallback

[AntoLC]

Purpose

Some users have their websockets blocked, so they cannot collaborate.
If they are connected with other collaborators at the same time, it will create constant conflict in the document.

Proposal

We have succeeded to propose an experience almost as good as with websocket.

• We will use a http fallback when the websocket is not able to connect.
• We are still using the Hocus Pocus mechanism, so push and pull are trigger by the Hocus Pocus provider and server.
• By using the Hocus Pocus mechanism, we are still using y-protocols/sync making our request very light (a few bytes).
• We are using the SSE (https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events)
  to pull data, to minimize the requests amount and keep as much as possible our documents sync between each others.

Cases we solved:

• connect even without websockets users altogether
• keep rights (can edit / can view) by using the same mechanism as with the WS
• keep the awareness (cursor), sync and doc update
• keep our requests light
• add a nginx auth cache system - query the backend 1 time every 30 seconds
• test what I could

Architecture

flowchart TD
    title1[WebSocket Success]-->Client1(Client)<--->|WebSocket Success|WS1(Websocket) --> Nginx1(Ngnix) <--> Auth1("Auth Sub Request (Django)") --->|With the good right|YServer1("Hocus Pocus Server")
  YServer1 --> WS1
  YServer1 <--> clients(Dispatch to clients)
  title2[WebSocket Fails - Push data]-->Client2(Client)---|WebSocket fails|HTTP2(HTTP) --> Nginx2(Ngnix) <--> Auth2("Auth Sub Request (Django)")--->|With the good right|Express2(Express) --> YServer2("Hocus Pocus Server") --> clients(Dispatch to clients)
  title3[WebSocket Fails - Pull data]-->Client3(Client)<--->|WebSocket fails|SSE(SSE) --> Nginx3(Ngnix) <--> Auth3("Auth Sub Request (Django)") --->|With the good right|Express3(Express) --> YServer3("Listen Hocus Pocus Server")
  YServer3("Listen Hocus Pocus Server") --> SSE
  YServer3("Listen Hocus Pocus Server") <--> clients(Data from clients)

Loading

[virgile-dev]

Great job @AntoLC !

[YousefED]

@AntoLC Nice you got this working. If I see it correctly, you created a new endpoint over which you're always syncing the entire Y.Doc to and from the server.

If I'm not mistaken, normally the Y.js sync protocol is more efficient than this and syncs the exact updates required. What's the reason you went for this approach (new endpoint, syncing entire doc) instead of the proxy approach? I think the proxy approach has some potential advantages:

• We can keep the same sync protocol, but just switch to a different transport (more efficient and awareness would still work)
• The HocusPocus side can stay the same, our "fix" would be isolated to a separate layer) (less code complexity and smaller chance of bugs or security issues)

I might be missing some advantages of your current approach, but my concern is mainly that it adds more "custom code" that's another surface we need to test, maintain and secure. The proxy approach would isolate / limit this more I think

[AntoLC]

You can test this PR before it is merged on https://docs-ia.beta.numerique.gouv.fr/.
To deactivate the websocket add the query param withoutWS=true

Example public doc: https://docs-ia.beta.numerique.gouv.fr/docs/481a9933-3514-4aeb-9877-c21be1388877/?withoutWS=true