docs: add a guide on Batching in Tailcall #97
Conversation
tusharmath
force-pushed
the
docs/batching
branch
from
ad5ded3 to
3c27dff
Compare
|
@tusharmath @amitksingh1490 when you have a moment, please review. |
ayewo
force-pushed
the
docs/batching
branch
2 times, most recently
from
7332821 to
c04fbea
Compare
ayewo
force-pushed
the
docs/batching
branch
2 times, most recently
from
75b1d84 to
5fd123f
Compare
|
Hey @meskill Would appreciate if you could review and approve this PR. It’s been open for about a month now, thank you. |
|
|
||
| Batching when configured correctly, can significantly reduce strain on high-traffic REST backends. You only need to add a handful of operators to your GraphQL schema (i.e. custom directives) for Tailcall to do most of the heavy lifting for you[^1]. | ||
|
|
||
| ## Scenario |
There was a problem hiding this comment.
we need some introduction before describing the scenario because it feels like steep turn for this here and hard to catch what is this about
There was a problem hiding this comment.
Thanks for your remarks. What would you suggest here?
The original prompt by Tushar was:
Write a guide about the batching capabilities of Tailcall. Consider a food delivery application where a lot of users are requesting the same data. Batching could be implemented based on GEO Location / City / Locality etc. Emphasize on the impact it has on performance and how it can help scale. Consider the backend to be in REST.
There was a problem hiding this comment.
- REduce the drama about "perks" and "remote work" and "slack bot"
- Talk about traffic surge during lunch hours. This is not just about orders, but also users in the same GEO location requesting for the same data again and again and the upstreams services serving literraly the same data to everyone. Instead having an orchestrator that could sit in between the frontend and the upstream services could very efficiently batch these request, reducing the load massively.
There was a problem hiding this comment.
@tusharmath if upstream is repeatedly serving the same data to difference users, then the obvious solution here is caching and not batching.
This is why I didn’t dwell on concurrents users but on a single user performing multiple requests.
To my knowledge, there’s no way for Tailcall to batch requests from multiple users (read multiple simultaneous requests) into a single upstream request.
There was a problem hiding this comment.
if upstream is repeatedly serving the same data to difference users, then the obvious solution here is caching and not batching.
Not necessary, caching is relevant when the data is changing. With batching enabled, we can serve multiple people the same data without requesting it multiple times upstream and can do it even if the data isn't cachable.
To my knowledge, there’s no way for Tailcall to batch requests from multiple users (read multiple simultaneous requests) into a single upstream request.
Check @server.batchRequest it allows tailcall to batch multiple requests.
There was a problem hiding this comment.
@tusharmath
I feel like we are talking past each other here. Tailcall has two notions of batching but based on your feedback, it is not really clear which one you are asking for.
Inbound Batching
My understanding of batching in this context is similar to the concept of multiplexing in HTTP/2 as illustrated by the diagram in this SO answer:
As an example, an end user sends 5 different graphql requests in quick succession to the Tailcall server (this might happen on app load when trying to hydrate multiple view components). Because the 5 inbound requests were received within a few millis of each other, the Tailcall server can decide to queue all 5 responses so it can send only 1 response to the end user, if the Tailcall server was started with batchRequests option set to true.
This type of batching reduces the network round-trip from 5 responses down to only 1 response.
Outbound Batching
The Tailcall server needs to make a multiple outbound requests to another server using a combination of the @upstream operator and the @http operation. Instead of making 50 or 100 requests, one-by-one, Tailcall can batch them into a single outbound request. This type of batching is what I understood that the original article was asking for as can be seen in the diagram, examples and description.
The only way to use geo-location to improve response times for millions of end users trying to use a service is to either use geo-routing or introduce a CDN that is geo-aware to cache identical requests. It was hard to work either of those solutions into the article which is why I focused on the Slack command example.
In fact your last sentence in the original issue was: "Consider the backend to be in REST" and I asked clarifying questions under the assumption this would be an article about outbound batching on the original issue but never got any response.
Essentially, this would mean a complete rewrite of an article that took 2 months just to get feedback ...
There was a problem hiding this comment.
@tusharmath no response ?
There was a problem hiding this comment.
Hey @tusharmath
your response is needed to this question as it affects all of the feedback provided.
|
@meskill just a friendly reminder to re-review, thanks. |
ayewo
force-pushed
the
docs/batching
branch
5 times, most recently
from
106bb1f to
0ae02b3
Compare
|
@meskill @tusharmath bump. |
|
|
||
| ### Multiple Vendors | ||
|
|
||
|  |
There was a problem hiding this comment.
use excalidraw to create images.
| ### Constraints | ||
|
|
||
| The nature of such a service could cause traffic spikes for every upstream vendor needed to fulfil each individual order, so some care has to be taken when designing the meal app's API. | ||
| For instance, if thousands of managers across the world use the command at the same time (just before lunch break) to place team orders, the sudden traffic spike will spread to upstream vendors, leading to delayed or failed orders. |
There was a problem hiding this comment.
| For instance, if thousands of managers across the world use the command at the same time (just before lunch break) to place team orders, the sudden traffic spike will spread to upstream vendors, leading to delayed or failed orders. | |
| For instance, if millions of people across the world use the command at the same time (just before lunch break) to place team orders, the sudden traffic spike will spread to upstream vendors, leading to delayed or failed orders. |
| - performing the same operation on different entities of the same type (e.g. _Update_ the team's order so meals are shipped to employees with the following ids `1`, `4` & `7`) or; | ||
| - grouping together different operations in one request (e.g. _Create_ `Jain` as a new employee, _Update_ an employee's meal preferences and _Delete_ meals above a certain price from the menu). | ||
|
|
||
| ### Real-world Examples of Batching |
There was a problem hiding this comment.
Table isn't readable because we have too many columns. Flatten it into multiple paras.
| | | `POST` | 5. `/batch/submitJob` (async) | Request body | `application/json` | [arcgis.com](https://developers.arcgis.com/rest/services-reference/enterprise/batch-geocode.htm) | | ||
| | | `POST` | 6. `/batch/` | Request body | `multipart/mixed` | [google.com](https://cloud.google.com/storage/docs/batch) | | ||
|
|
||
| ### Batching: Sync or Async |
There was a problem hiding this comment.
Some context about error handling should also be added to this guide.
|
|
||
| --- | ||
|
|
||
| ## Batching in Tailcall |
There was a problem hiding this comment.
I think the head of the document is generally talking about REST & Batching. The jump to Tailcall is quite sudden. I would split this doc into two separate ones. The tailcall specific bits should be in the guide, the remaining stuff can be on our blogs.
amitksingh1490
force-pushed
the
docs/batching
branch
from
d5531ba to
8f15d73
Compare
Co-authored-by: Kiryl Mialeshka <[email protected]>
|
/tip 50$ |
|
🎉🎈 @ayewo has been awarded $50! 🎈🎊 |
Fixes tailcallhq/tailcall#882
/claim tailcallhq/tailcall#882