From e0896568d0974457dd1d61b5dbf92735429d361d Mon Sep 17 00:00:00 2001 From: meskill <8974488+meskill@users.noreply.github.com> Date: Fri, 26 Jan 2024 17:49:24 +0000 Subject: [PATCH 01/12] feat: grpc guide --- docs/guides/grpc.md | 277 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 277 insertions(+) create mode 100644 docs/guides/grpc.md diff --git a/docs/guides/grpc.md b/docs/guides/grpc.md new file mode 100644 index 0000000000..f265768c34 --- /dev/null +++ b/docs/guides/grpc.md @@ -0,0 +1,277 @@ +--- +title: Integration with gRPC service +--- + +In this guide we will setup simple gRPC service and use it inside Tailcall's config to fetch some of the data provided by the service. This way Tailcall can provide single GraphQL interface wrapping any number of gRPC services. + +## What is gRPC? + +This guide assumes a basic familiarity with gRPC. + +**gRPC** is a high-performance framework created by Google for remote procedure calls (RPCs). Its key features include: + +- **HTTP/2 Transport:** Ensures efficient and fast data transfer. +- **Protocol Buffers (Protobuf):** Serves as a powerful interface description language. +- **Efficiency:** Offers binary serialization, reduced latency, and supports data streaming. + +This combination of features makes gRPC ideal for microservices and distributed systems. + +If you need a more detailed understanding or are new to gRPC, we recommend visiting the [official gRPC website](https://grpc.io/) for comprehensive documentation and resources. + +Now, let's explore how gRPC can be integrated into our proxy gateway to enhance communication and data exchange in distributed systems. + +## gRPC upstream + +We need some gRPC service available to be able to execute requests from Tailcall gateway. For the pure example purposes, we will build some simple gRPC service. + +### Protobuf definition + +First, we need to create example protobuf file that will define the structure of data we want to transmit using gRPC. + +Here are the definition of `NewsService` that implements CRUD operations on news data that we'll put into `news.proto` file. + +```protobuf +syntax = "proto3"; + +import "google/protobuf/empty.proto"; + +// Define message type for News with all its fields +message News { + int32 id = 1; + string title = 2; + string body = 3; + string postImage = 4; +} + +// Message with just id of single news +message NewsId { + int32 id = 1; +} + +// List of ids of news to get multiple response +message MultipleNewsId { + repeated NewsId ids = 1; +} + +// List of all news +message NewsList { + repeated News news = 1; +} + +// NewsService defines read and write operations for news items +service NewsService { + // GetAllNews retrieves all news items without any arguments + rpc GetAllNews (google.protobuf.Empty) returns (NewsList) {} + + // GetNews fetches a single news item by its ID + rpc GetNews (NewsId) returns (News) {} + + // GetMultipleNews retrieves multiple news items based on their IDs + rpc GetMultipleNews (MultipleNewsId) returns (NewsList) {} +} +``` + +### Implement gRPC service + +Now having the protobuf file you can write a server that implements `NewsService` at any language you want that supports gRPC. + +Tailcall organization has sample node.js service inside [this repo](https://github.com/tailcallhq/node-grpc) that you can pull to your local machine + +To spin up the sample service just run inside repo + +```sh +npm i +node server +``` + +and wait for logs about service is running. + +## Tailcall config + +Now when we have a running gRPC service we're going to write Tailcall's config to make the integration. + +To do this we need to specify GraphQL types corresponding to gRPC types we have defined in protobuf file. Let's create new file `grpc.graphql` file with following content: + +```graphql +# The GraphQL representation for News message type +type News { + id: Int + title: String + body: String + postImage: String +} + +# Input type that is used to fetch news data by its id +input NewsInput { + id: Int +} + +# Resolves multiple news entries +type NewsData { + news: [News]! +} +``` + +Now when we have corresponding types in schema we want to define GraphQL Query that specifies the operation we can execute onto news. We can extend our config with next Query: + +```graphql +type Query { + # Get all news i.e. NewsService.GetAllNews + news: NewsData! + # Get single news by id i.e. NewsService.GetNews + newsById(news: NewsInput!): News! +} +``` + +It not working example yet because we haven't specified how to use connect to server. To do this you may want to explore more about [`@grpc` operator](../operators/grpc.md). It's usage is pretty straightforward and require you to specify some required options like path to protobuf file itself, name of the service and method that should be used to make a call. + +If you need to provide any input to gRPC method call you can specify it with the `body` option that allows to specify a Mustache template and therefore it could use any input data like `args` and `value` to construct the body request. The body value is specified in the JSON format if you need to create the input manually and cannot just use `args` input. + +```graphql +type Query { + news: NewsData! + @grpc( + service: "NewsService" + method: "GetAllNews" + protoPath: "news.proto" + ) + newsById(news: NewsInput!): News! + @grpc( + service: "NewsService" + method: "GetNews" + body: "{{args.news}}" + protoPath: "news.proto" + ) +} +``` + +And it's only left to specify options of Tailcall's ingress and egress in the beginning of the config using [`@server`](../operators/server.md) and [`@upstrem`](../operators/upstream.md) operators. Wrapping up the whole result config that may look like this: + +```graphql +# highlight-start +schema @server(port: 8000, graphiql: true) @upstream(baseURL: "http://localhost:50051", httpCache: true) { + query: Query +} + +type Query { + news: NewsData! + @grpc( + service: "NewsService" + method: "GetAllNews" + protoPath: "src/grpc/tests/news.proto" + ) + newsById(news: NewsInput!): News! + @grpc( + service: "NewsService" + method: "GetNews" + body: "{{args.news}}" + protoPath: "src/grpc/tests/news.proto" + ) +} +# highlight-end + +type News { + id: Int + title: String + body: String + postImage: String +} + +input NewsInput { + id: Int +} + +type NewsData { + news: [News]! +} +``` + +And now you can go to the page `http://127.0.0.1:8000/graphql` and execute some GraphQL queries e.g.: + +```graphql +{ + news { + news { + id + title + body + } + } +} +``` + +Or + +```graphql +{ + newsById(news: {id: 2}) { + id + title + body + } +} +``` + +### Batching + +Another important feature of `@grpc` operator is that it allows you to implement request batching for remote data almost effortlessly as soon as you have gRPC methods that implements resolves multiple responses for multiple input in single request. + +In out protobuf example file we have such a method called `GetMultipleNews` that we can use. To enable batching we need to enable [`@upstream.batch` option](../operators/upstream.md#batch) first and specify `groupBy` option for the `@grpc` operator. + +```graphql +schema @server(port: 8000, graphiql: true) @upstream(baseURL: "http://localhost:50051", httpCache: true, batch: {delay: 10}) { + query: Query +} + +type Query { + newsById(news: NewsInput!): News! + @grpc( + service: "NewsService" + method: "GetNews" + body: "{{args.news}}" + protoPath: "src/grpc/tests/news.proto" + # highlight-next-line + groupBy: ["news", "id"] + ) +} +``` + +Restart the Tailcall server and make the query with multiple news separately, e.g.: + +```graphql +{ + n1: newsById(news: {id: 1}) { + id + title + body + } + n2: newsById(news: {id: 2}) { + id + title + body + } +} +``` + +Those 2 requests will be executed inside single request to gRPC method `GetMultipleNews` + +## Conclusion + +Well done on integrating a gRPC service with the Tailcall gateway! This tutorial has demonstrated the straightforward and efficient process, showcasing Tailcall's compatibility with advanced communication protocols like gRPC. + +You can find this working example and test it by yourself by next links: + +- [node-grpc](https://github.com/tailcallhq/node-grpc) - example implementation for gRPC service in node.js +- [gRPC example config](https://github.com/tailcallhq/tailcall/blob/main/examples/grpc.graphql) - Tailcall's config to integrate with gRPC service above + +### Key Takeaways + +- **Simplicity of Integration:** The integration of gRPC with Tailcall is not only seamless but also enhances the overall capability of your system to handle high-performance and efficient data composition. +- **Scalability and Performance:** By leveraging the power of gRPC along with Tailcall, we've laid a foundation for building scalable and high-performing distributed systems. + +### Next Steps + +With the basics in place, we encourage you to explore further: + +- **Dive Deeper:** Tailcall gateway offers a lot of other features and configurations that you can utilize. Dive deeper into our documentation to explore more advanced settings and customization options. +- **Explore Other Guides:** Our documentation is filled with various guides and tutorials that can help you leverage the full potential of Tailcall in different scenarios. Whether it's adding security layers, load balancing, or detailed logging, there's a lot more to explore. From 893037e20c2775458c82298f5f787ef2e2231a71 Mon Sep 17 00:00:00 2001 From: meskill <8974488+meskill@users.noreply.github.com> Date: Fri, 26 Jan 2024 17:53:29 +0000 Subject: [PATCH 02/12] fix spelling --- docs/guides/grpc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/grpc.md b/docs/guides/grpc.md index f265768c34..54454beb10 100644 --- a/docs/guides/grpc.md +++ b/docs/guides/grpc.md @@ -145,7 +145,7 @@ type Query { } ``` -And it's only left to specify options of Tailcall's ingress and egress in the beginning of the config using [`@server`](../operators/server.md) and [`@upstrem`](../operators/upstream.md) operators. Wrapping up the whole result config that may look like this: +And it's only left to specify options of Tailcall's ingress and egress in the beginning of the config using [`@server`](../operators/server.md) and [`@upstream`](../operators/upstream.md) operators. Wrapping up the whole result config that may look like this: ```graphql # highlight-start From edf6f1183714ed0848288bc71d8a017540841da6 Mon Sep 17 00:00:00 2001 From: meskill <8974488+meskill@users.noreply.github.com> Date: Fri, 26 Jan 2024 18:08:25 +0000 Subject: [PATCH 03/12] fix prettier --- docs/guides/grpc.md | 32 +++++++------------------------- 1 file changed, 7 insertions(+), 25 deletions(-) diff --git a/docs/guides/grpc.md b/docs/guides/grpc.md index 54454beb10..75c0725f19 100644 --- a/docs/guides/grpc.md +++ b/docs/guides/grpc.md @@ -129,19 +129,9 @@ If you need to provide any input to gRPC method call you can specify it with the ```graphql type Query { - news: NewsData! - @grpc( - service: "NewsService" - method: "GetAllNews" - protoPath: "news.proto" - ) + news: NewsData! @grpc(service: "NewsService", method: "GetAllNews", protoPath: "news.proto") newsById(news: NewsInput!): News! - @grpc( - service: "NewsService" - method: "GetNews" - body: "{{args.news}}" - protoPath: "news.proto" - ) + @grpc(service: "NewsService", method: "GetNews", body: "{{args.news}}", protoPath: "news.proto") } ``` @@ -154,19 +144,9 @@ schema @server(port: 8000, graphiql: true) @upstream(baseURL: "http://localhost: } type Query { - news: NewsData! - @grpc( - service: "NewsService" - method: "GetAllNews" - protoPath: "src/grpc/tests/news.proto" - ) + news: NewsData! @grpc(service: "NewsService", method: "GetAllNews", protoPath: "src/grpc/tests/news.proto") newsById(news: NewsInput!): News! - @grpc( - service: "NewsService" - method: "GetNews" - body: "{{args.news}}" - protoPath: "src/grpc/tests/news.proto" - ) + @grpc(service: "NewsService", method: "GetNews", body: "{{args.news}}", protoPath: "src/grpc/tests/news.proto") } # highlight-end @@ -219,7 +199,9 @@ Another important feature of `@grpc` operator is that it allows you to implement In out protobuf example file we have such a method called `GetMultipleNews` that we can use. To enable batching we need to enable [`@upstream.batch` option](../operators/upstream.md#batch) first and specify `groupBy` option for the `@grpc` operator. ```graphql -schema @server(port: 8000, graphiql: true) @upstream(baseURL: "http://localhost:50051", httpCache: true, batch: {delay: 10}) { +schema + @server(port: 8000, graphiql: true) + @upstream(baseURL: "http://localhost:50051", httpCache: true, batch: {delay: 10}) { query: Query } From f14232489c0108876d4417d8cefad863d90ac7bf Mon Sep 17 00:00:00 2001 From: meskill <8974488+meskill@users.noreply.github.com> Date: Wed, 14 Feb 2024 17:33:48 +0000 Subject: [PATCH 04/12] update docs with link usage --- docs/guides/grpc.md | 46 +++++++++++++++++++++-------- docs/operators/grpc.md | 67 +++++++++++++----------------------------- 2 files changed, 53 insertions(+), 60 deletions(-) diff --git a/docs/guides/grpc.md b/docs/guides/grpc.md index 75c0725f19..1d291ece5a 100644 --- a/docs/guides/grpc.md +++ b/docs/guides/grpc.md @@ -35,6 +35,8 @@ syntax = "proto3"; import "google/protobuf/empty.proto"; +package news; + // Define message type for News with all its fields message News { int32 id = 1; @@ -123,30 +125,49 @@ type Query { } ``` -It not working example yet because we haven't specified how to use connect to server. To do this you may want to explore more about [`@grpc` operator](../operators/grpc.md). It's usage is pretty straightforward and require you to specify some required options like path to protobuf file itself, name of the service and method that should be used to make a call. +Also let's specify options of Tailcall's ingress and egress in the beginning of the config using [`@server`](../operators/server.md) and [`@upstream`](../operators/upstream.md) operators. + +```graphql +schema + @server(port: 8000, graphiql: true) + @upstream(baseURL: "http://localhost:50051", httpCache: true) +{ + query: Query +} +``` + +To specify protobuf file to read types from, use `@link` operator with type `Protobuf` on schema. `id` is important part of the definition that will be used by `@grpc` operator later + +```graphql +schema @link(id: "news", src: "./news.proto", type: Protobuf) +``` + +Now you can connect graphql types to grpc types. To do this you may want to explore more about [`@grpc` operator](../operators/grpc.md). It's usage is pretty straightforward and require you to specify path to method that should be used to make a call. The method name will start with the `id` of file we were linked before, package name, service name and the method name, all separated by `.` symbol. If you need to provide any input to gRPC method call you can specify it with the `body` option that allows to specify a Mustache template and therefore it could use any input data like `args` and `value` to construct the body request. The body value is specified in the JSON format if you need to create the input manually and cannot just use `args` input. ```graphql type Query { - news: NewsData! @grpc(service: "NewsService", method: "GetAllNews", protoPath: "news.proto") - newsById(news: NewsInput!): News! - @grpc(service: "NewsService", method: "GetNews", body: "{{args.news}}", protoPath: "news.proto") + news: NewsData! @grpc(method: "news.news.NewsService.GetAllNews") + newsById(news: NewsInput!): News! @grpc(service: "news.news.NewsService.GetNews", body: "{{args.news}}") } ``` -And it's only left to specify options of Tailcall's ingress and egress in the beginning of the config using [`@server`](../operators/server.md) and [`@upstream`](../operators/upstream.md) operators. Wrapping up the whole result config that may look like this: +Wrapping up the whole result config that may look like this: ```graphql # highlight-start -schema @server(port: 8000, graphiql: true) @upstream(baseURL: "http://localhost:50051", httpCache: true) { +schema + @server(port: 8000, graphiql: true) + @upstream(baseURL: "http://localhost:50051", httpCache: true) + @link(id: "news", src: "./news.proto", type: Protobuf) +{ query: Query } type Query { - news: NewsData! @grpc(service: "NewsService", method: "GetAllNews", protoPath: "src/grpc/tests/news.proto") - newsById(news: NewsInput!): News! - @grpc(service: "NewsService", method: "GetNews", body: "{{args.news}}", protoPath: "src/grpc/tests/news.proto") + news: NewsData! @grpc(method: "news.news.NewsService.GetAllNews") + newsById(news: NewsInput!): News! @grpc(method: "news.news.NewsService.GetNews", body: "{{args.news}}") } # highlight-end @@ -201,17 +222,16 @@ In out protobuf example file we have such a method called `GetMultipleNews` that ```graphql schema @server(port: 8000, graphiql: true) - @upstream(baseURL: "http://localhost:50051", httpCache: true, batch: {delay: 10}) { + @upstream(baseURL: "http://localhost:50051", httpCache: true, batch: {delay: 10}) + @link(id: "news", src: "./news.proto", type: Protobuf) { query: Query } type Query { newsById(news: NewsInput!): News! @grpc( - service: "NewsService" - method: "GetNews" + method: "news.news.NewsService.GetNews" body: "{{args.news}}" - protoPath: "src/grpc/tests/news.proto" # highlight-next-line groupBy: ["news", "id"] ) diff --git a/docs/operators/grpc.md b/docs/operators/grpc.md index c19c018a2c..98b88697a4 100644 --- a/docs/operators/grpc.md +++ b/docs/operators/grpc.md @@ -9,12 +9,16 @@ The **@grpc** operator is an essential GraphQL custom directive designed to inte The **@grpc** operator allows GraphQL fields to be resolved using gRPC services. Here's an example demonstrating its usage in a GraphQL schema: ```graphql showLineNumbers +schema @link(id: "proto", src: "./users.proto", type: Protobuf) { + query: Query +} + type Query { - users: [User] @grpc(service: "UserService", method: "ListUsers", protoPath: "./proto/user_service.proto") + users: [User] @grpc(method: "proto.users.UserService.ListUsers") } ``` -In this example, when the `users` field is queried, the GraphQL server will make a gRPC request to the `ListUsers` method of the `UserService`. +In this example, when the `users` field is queried, the GraphQL server will make a gRPC request to the `users.ListUsers` method of the `UserService`. ### Sample proto File @@ -23,6 +27,8 @@ The `.proto` file defines the structure of the gRPC service and its methods. Her ```proto showLineNumbers syntax = "proto3"; +package users; + service UserService { rpc ListUsers (UserListRequest) returns (UserListReply) {} rpc GetUser (UserGetRequest) returns (UserGetReply) {} @@ -45,52 +51,26 @@ message UserGetReply { } ``` -### service - -Indicates the gRPC service to be called. This should match the service name as defined in the `.proto` file. +To connect that file to Tailcall use `@link` operator: ```graphql showLineNumbers -type Query { - users: [User] - @grpc( - # highlight-start - service: "UserService" - # highlight-end - method: "ListUsers" - protoPath: "./proto/user_service.proto" - ) +schema @link(id: "proto", src: "./users.proto", type: Protobuf) { + query: Query } ``` -### method - -Indicates the specific gRPC method to be invoked within the specified service. This should match the method name as defined in the `.proto` file. +Later use provided `id` as the first part of `method` option for `@grpc` operator to connect the definition to this specific proto file. -```graphql showLineNumbers -type Query { - users: [User] - @grpc( - service: "UserService" - # highlight-start - method: "ListUsers" - # highlight-end - protoPath: "./proto/user_service.proto" - ) -} -``` - -### protoPath +### method -Path to the `.proto` file, containing service and method definitions for request/response encoding and decoding. The path can be relative or absolute. If the path is relative, it is resolved relative to the directory where the tailcall command is run from. +Indicates the gRPC service and method to be called. This should match the service name as defined in the `.proto` file with package path. ```graphql showLineNumbers type Query { users: [User] @grpc( - service: "UserService" - method: "ListUsers" # highlight-start - protoPath: "./proto/user_service.proto" + method: "proto.users.UserService.ListUsers" # highlight-end ) } @@ -104,9 +84,7 @@ Indicates the base URL for the gRPC API. If omitted, the default URL defined in type Query { users: [User] @grpc( - service: "UserService" - method: "ListUsers" - protoPath: "./proto/user_service.proto" + method: "proto.users.UserService.ListUsers" # highlight-start baseURL: "https://grpc-server.example.com" # highlight-end @@ -126,9 +104,8 @@ type UserInput { type Query { user(id: UserInput!): User @grpc( - service: "UserService" - method: "GetUser" - protoPath: "./proto/user_service.proto" + + method: "proto.users.UserService.GetUser" # highlight-start body: "{{args.id}}" # highlight-end @@ -144,9 +121,7 @@ Custom headers for the gRPC request can be specified using the `headers` argumen type Query { users: [User] @grpc( - service: "UserService" - method: "ListUsers" - protoPath: "./proto/user_service.proto" + method: "proto.users.UserService.ListUsers" baseURL: "https://grpc-server.example.com" # highlight-start headers: [{key: "X-CUSTOM-HEADER", value: "custom-value"}] @@ -165,9 +140,7 @@ For using the groupBy capability, the response type of the gRPC method should be type Query { users(id: UserInput!): User @grpc( - service: "UserService" - method: "ListUsers" - protoPath: "./proto/user_service.proto" + method: "proto.users.UserService.ListUsers" baseURL: "https://grpc-server.example.com" # highlight-start groupBy: ["id"] From 9b0ed6b3d3f23856c5f6878e701e4725fd992ced Mon Sep 17 00:00:00 2001 From: meskill <8974488+meskill@users.noreply.github.com> Date: Wed, 14 Feb 2024 17:45:36 +0000 Subject: [PATCH 05/12] fix prettier --- docs/guides/grpc.md | 8 ++------ docs/operators/grpc.md | 1 - 2 files changed, 2 insertions(+), 7 deletions(-) diff --git a/docs/guides/grpc.md b/docs/guides/grpc.md index 1d291ece5a..81ec3942ea 100644 --- a/docs/guides/grpc.md +++ b/docs/guides/grpc.md @@ -128,10 +128,7 @@ type Query { Also let's specify options of Tailcall's ingress and egress in the beginning of the config using [`@server`](../operators/server.md) and [`@upstream`](../operators/upstream.md) operators. ```graphql -schema - @server(port: 8000, graphiql: true) - @upstream(baseURL: "http://localhost:50051", httpCache: true) -{ +schema @server(port: 8000, graphiql: true) @upstream(baseURL: "http://localhost:50051", httpCache: true) { query: Query } ``` @@ -160,8 +157,7 @@ Wrapping up the whole result config that may look like this: schema @server(port: 8000, graphiql: true) @upstream(baseURL: "http://localhost:50051", httpCache: true) - @link(id: "news", src: "./news.proto", type: Protobuf) -{ + @link(id: "news", src: "./news.proto", type: Protobuf) { query: Query } diff --git a/docs/operators/grpc.md b/docs/operators/grpc.md index 98b88697a4..b805c0cf36 100644 --- a/docs/operators/grpc.md +++ b/docs/operators/grpc.md @@ -104,7 +104,6 @@ type UserInput { type Query { user(id: UserInput!): User @grpc( - method: "proto.users.UserService.GetUser" # highlight-start body: "{{args.id}}" From 940093ad0e73a4ac6c9670a31c4342c792d562d9 Mon Sep 17 00:00:00 2001 From: Tushar Mathur Date: Tue, 27 Feb 2024 17:08:54 -0500 Subject: [PATCH 06/12] chore: update docs a little bit --- docs/operators/grpc.md | 117 ++++++++++++++++------------------------- src/css/custom.css | 2 +- 2 files changed, 45 insertions(+), 74 deletions(-) diff --git a/docs/operators/grpc.md b/docs/operators/grpc.md index 80392ad8ca..f49d92cc82 100644 --- a/docs/operators/grpc.md +++ b/docs/operators/grpc.md @@ -2,29 +2,25 @@ title: "@grpc" --- -The **@grpc** operator, a crucial GraphQL custom directive, interfaces with gRPC services. It directly invokes gRPC services through GraphQL queries, bridging two powerful technologies. This directive proves useful in integrating GraphQL with microservices exposing gRPC endpoints. +The `@grpc` directive enables the resolution of GraphQL fields via gRPC services. Below is an illustrative example of how to apply this directive within a GraphQL schema: -### Using the `@grpc` Operator - -The **@grpc** operator resolves GraphQL fields using gRPC services. For example, querying the `users` field triggers a gRPC request to the `ListUsers` method of the `UserService`. - -```graphql showLineNumbers -schema @link(id: "proto", src: "./users.proto", type: Protobuf) { +```graphql +schema @link(src: "./users.proto", type: Protobuf) { query: Query } type Query { - users: [User] @grpc(method: "proto.users.UserService.ListUsers") + users: [User] @grpc(method: "users.UserService.ListUsers") } ``` -In this example, querying the `users` field prompts the GraphQL server to make a gRPC request to the `ListUsers` method of the `users.UserService`. +This schema snippet demonstrates the directive's application, where a query for `users` triggers a gRPC request to the `UserService`'s `ListUsers` method, thereby fetching the user data. -### Sample proto File +## Understanding the Proto File Structure -The `.proto` file outlines the gRPC service structure and its methods. For instance: +The `.proto` file delineates the structure and methods of the gRPC service. A simplified example of such a file is as follows: -```proto showLineNumbers +```proto syntax = "proto3"; package users; @@ -35,117 +31,92 @@ service UserService { } message UserListRequest { - // Request parameters + // Definitions of request parameters } message UserListReply { - // Reply structure + // Structure of the reply } message UserGetRequest { - // Reply structure + // Definitions of request parameters } message UserGetReply { - // Reply structure + // Structure of the reply } ``` -To connect that file to Tailcall use `@link` operator: +Linking this file within a GraphQL schema is facilitated by the `@link` directive, as shown below: -```graphql showLineNumbers -schema @link(id: "proto", src: "./users.proto", type: Protobuf) { +```graphql +schema @link(src: "./users.proto", type: Protobuf) { query: Query } ``` -Later use provided `id` as the first part of `method` option for `@grpc` operator to connect the definition to this specific proto file. +Tailcall automatically resolves the protobuf file for any methods referenced in the `@grpc` directive. -### method +### Directive Parameters -Specifies the gRPC method for invocation within the service. It must match the method name in the `.proto` file including package path. +#### `method` -```graphql showLineNumbers +This parameter specifies the gRPC service and method to be invoked, formatted as `..`: + +```graphql type Query { - users: [User] - @grpc( - # highlight-start - method: "proto.users.UserService.ListUsers" - # highlight-end - ) + users: [User] @grpc(method: "proto.users.UserService.ListUsers") } ``` -### baseURL +#### `baseURL` -Indicates the base URL for the gRPC API. If omitted, it defaults to the URL defined in the `@upstream` operator. +Defines the base URL for the gRPC API. If not specified, the URL set in the `@upstream` directive is used by default: -```graphql showLineNumbers +```graphql type Query { - users: [User] - @grpc( - method: "proto.users.UserService.ListUsers" - # highlight-start - baseURL: "https://grpc-server.example.com" - # highlight-end - ) + users: [User] @grpc(baseURL: "https://grpc-server.example.com", method: "proto.users.UserService.ListUsers") } ``` -### body +#### `body` -Outlines the arguments for the gRPC call. The `body` field specifies the arguments for the gRPC call, either static or dynamic. +This parameter outlines the arguments for the gRPC call, allowing for both static and dynamic inputs: -```graphql showLineNumbers +```graphql type UserInput { id: ID } type Query { - user(id: UserInput!): User - @grpc( - method: "proto.users.UserService.GetUser" - # highlight-start - body: "{{args.id}}" - # highlight-end - ) + user(id: UserInput!): User @grpc(body: "{{args.id}}", method: "proto.users.UserService.GetUser") } ``` -### headers +#### `headers` -Custom headers for the gRPC request can specify using the `headers` argument. This proves useful for passing authentication tokens or other contextual information. +Custom headers for the gRPC request can be defined, facilitating the transmission of authentication tokens or other contextual data: -```graphql showLineNumbers +```graphql type Query { users: [User] - @grpc( - method: "proto.users.UserService.ListUsers" - baseURL: "https://grpc-server.example.com" - # highlight-start - headers: [{key: "X-CUSTOM-HEADER", value: "custom-value"}] - # highlight-end - ) + @grpc(headers: [{key: "X-CUSTOM-HEADER", value: "custom-value"}], method: "proto.users.UserService.ListUsers") } ``` -### groupBy +#### `groupBy` -The `groupBy` argument optimizes batch requests by grouping them based on specified response keys, enhancing performance in scenarios with similar requests. +This argument is employed to optimize batch requests by grouping them based on specified response keys, enhancing performance in scenarios requiring multiple, similar requests: -By understanding and using its fields, developers can create efficient, streamlined APIs that leverage the strengths of both GraphQL and gRPC. - -```graphql showLineNumbers +```graphql type Query { - users(id: UserInput!): User - @grpc( - method: "proto.users.UserService.ListUsers" - baseURL: "https://grpc-server.example.com" - # highlight-start - groupBy: ["id"] - # highlight-end - ) + users(id: UserInput!): [User] + @grpc(groupBy: ["id"], method: "proto.users.UserService.ListUsers", baseURL: "https://grpc-server.example.com") } ``` -The **@grpc** operator is a powerful tool for GraphQL developers, enabling seamless integration with gRPC services. By understanding and utilizing its specific fields, developers can create efficient, streamlined APIs that leverage the strengths of both GraphQL and gRPC. +:::info +Read about **[n + 1]** to learn how to use the `groupBy` setting. +::: + +[n + 1]: /docs/guides/n+1/ diff --git a/src/css/custom.css b/src/css/custom.css index 6ec377a980..2bfbf26753 100644 --- a/src/css/custom.css +++ b/src/css/custom.css @@ -48,7 +48,7 @@ --ifm-code-font-size: 100%; --ifm-code-background: rgb(246, 247, 248); --ifm-code-border-radius: var(--ifm-global-radius); - --ifm-code-font-size: 0.9rem; + --ifm-code-font-size: 90%; --ifm-code-padding-horizontal: 0.2rem; --ifm-code-padding-vertical: 0.2rem; --ifm-pre-background: var(--ifm-code-background); From 2729de9c6be6047e5f9b5ef05cc83edb0cbf6f79 Mon Sep 17 00:00:00 2001 From: Tushar Mathur Date: Tue, 27 Feb 2024 17:10:31 -0500 Subject: [PATCH 07/12] chore: update operator doc --- docs/operators/grpc.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/operators/grpc.md b/docs/operators/grpc.md index f49d92cc82..524aa564a8 100644 --- a/docs/operators/grpc.md +++ b/docs/operators/grpc.md @@ -47,6 +47,10 @@ message UserGetReply { } ``` +:::important +It is mandatory to have a package name in a protobuf file. +::: + Linking this file within a GraphQL schema is facilitated by the `@link` directive, as shown below: ```graphql From 8c5c1914e105ef6a8ad6533effb105facc58f2b3 Mon Sep 17 00:00:00 2001 From: Tushar Mathur Date: Tue, 27 Feb 2024 17:42:44 -0500 Subject: [PATCH 08/12] update grpc doc --- docs/guides/grpc.md | 45 +++++++++++++++++++++++---------------------- 1 file changed, 23 insertions(+), 22 deletions(-) diff --git a/docs/guides/grpc.md b/docs/guides/grpc.md index 81ec3942ea..90dff21972 100644 --- a/docs/guides/grpc.md +++ b/docs/guides/grpc.md @@ -1,8 +1,9 @@ --- -title: Integration with gRPC service +title: GraphQL on gRPC +description: Building a GraphQL API on top of gRPC endpoints. --- -In this guide we will setup simple gRPC service and use it inside Tailcall's config to fetch some of the data provided by the service. This way Tailcall can provide single GraphQL interface wrapping any number of gRPC services. +In this guide, we will set up a simple gRPC service and use it inside Tailcall's config to fetch some of the data provided by the service. This way Tailcall can provide a single GraphQL interface wrapping any number of gRPC services. ## What is gRPC? @@ -12,7 +13,7 @@ This guide assumes a basic familiarity with gRPC. - **HTTP/2 Transport:** Ensures efficient and fast data transfer. - **Protocol Buffers (Protobuf):** Serves as a powerful interface description language. -- **Efficiency:** Offers binary serialization, reduced latency, and supports data streaming. +- **Efficiency:** Offers binary serialization, reduces latency, and supports data streaming. This combination of features makes gRPC ideal for microservices and distributed systems. @@ -22,13 +23,13 @@ Now, let's explore how gRPC can be integrated into our proxy gateway to enhance ## gRPC upstream -We need some gRPC service available to be able to execute requests from Tailcall gateway. For the pure example purposes, we will build some simple gRPC service. +We need some gRPC service available to be able to execute requests from the Tailcall gateway. For pure example purposes, we will build some simple gRPC services. ### Protobuf definition -First, we need to create example protobuf file that will define the structure of data we want to transmit using gRPC. +First, we need to create an example protobuf file that will define the structure of the data we want to transmit using gRPC. -Here are the definition of `NewsService` that implements CRUD operations on news data that we'll put into `news.proto` file. +Here is the definition of `NewsService` that implements CRUD operations on news data that we'll put into the `news.proto` file. ```protobuf syntax = "proto3"; @@ -45,12 +46,12 @@ message News { string postImage = 4; } -// Message with just id of single news +// Message with just the id of a single news message NewsId { int32 id = 1; } -// List of ids of news to get multiple response +// List of IDs of news to get multiple responses message MultipleNewsId { repeated NewsId ids = 1; } @@ -77,22 +78,22 @@ service NewsService { Now having the protobuf file you can write a server that implements `NewsService` at any language you want that supports gRPC. -Tailcall organization has sample node.js service inside [this repo](https://github.com/tailcallhq/node-grpc) that you can pull to your local machine +Tailcall organization has a sample node.js service inside [this repo](https://github.com/tailcallhq/node-grpc) that you can pull to your local machine -To spin up the sample service just run inside repo +To spin up the sample service just run inside the repo ```sh npm i node server ``` -and wait for logs about service is running. +and wait for logs about the service running. ## Tailcall config Now when we have a running gRPC service we're going to write Tailcall's config to make the integration. -To do this we need to specify GraphQL types corresponding to gRPC types we have defined in protobuf file. Let's create new file `grpc.graphql` file with following content: +To do this we need to specify GraphQL types corresponding to gRPC types we have defined in the protobuf file. Let's create a new file `grpc.graphql` file with the following content: ```graphql # The GraphQL representation for News message type @@ -114,7 +115,7 @@ type NewsData { } ``` -Now when we have corresponding types in schema we want to define GraphQL Query that specifies the operation we can execute onto news. We can extend our config with next Query: +Now when we have corresponding types in schema we want to define GraphQL Query that specifies the operation we can execute onto news. We can extend our config with the next Query: ```graphql type Query { @@ -125,7 +126,7 @@ type Query { } ``` -Also let's specify options of Tailcall's ingress and egress in the beginning of the config using [`@server`](../operators/server.md) and [`@upstream`](../operators/upstream.md) operators. +Also, let's specify options for Tailcall's ingress and egress at the beginning of the config using [`@server`](../operators/server.md) and [`@upstream`](../operators/upstream.md) operators. ```graphql schema @server(port: 8000, graphiql: true) @upstream(baseURL: "http://localhost:50051", httpCache: true) { @@ -133,15 +134,15 @@ schema @server(port: 8000, graphiql: true) @upstream(baseURL: "http://localhost: } ``` -To specify protobuf file to read types from, use `@link` operator with type `Protobuf` on schema. `id` is important part of the definition that will be used by `@grpc` operator later +To specify the protobuf file to read types from, use the `@link` operator with the type `Protobuf` on the schema. `id` is an important part of the definition that will be used by the `@grpc` operator later ```graphql schema @link(id: "news", src: "./news.proto", type: Protobuf) ``` -Now you can connect graphql types to grpc types. To do this you may want to explore more about [`@grpc` operator](../operators/grpc.md). It's usage is pretty straightforward and require you to specify path to method that should be used to make a call. The method name will start with the `id` of file we were linked before, package name, service name and the method name, all separated by `.` symbol. +Now you can connect GraphQL types to gRPC types. To do this you may want to explore more about [`@grpc` operator](../operators/grpc.md). Its usage is pretty straightforward and requires you to specify the path to a method that should be used to make a call. The method name will start with the package name, followed by the service name and the method name, all separated by the `.` symbol. -If you need to provide any input to gRPC method call you can specify it with the `body` option that allows to specify a Mustache template and therefore it could use any input data like `args` and `value` to construct the body request. The body value is specified in the JSON format if you need to create the input manually and cannot just use `args` input. +If you need to provide any input to the gRPC method call you can specify it with the `body` option that allows you to specify a Mustache template and therefore it could use any input data like `args` and `value` to construct the body request. The body value is specified in the JSON format if you need to create the input manually and cannot just use `args` input. ```graphql type Query { @@ -211,9 +212,9 @@ Or ### Batching -Another important feature of `@grpc` operator is that it allows you to implement request batching for remote data almost effortlessly as soon as you have gRPC methods that implements resolves multiple responses for multiple input in single request. +Another important feature of the `@grpc` operator is that it allows you to implement request batching for remote data almost effortlessly as soon as you have gRPC methods that resolve multiple responses for multiple inputs in a single request. -In out protobuf example file we have such a method called `GetMultipleNews` that we can use. To enable batching we need to enable [`@upstream.batch` option](../operators/upstream.md#batch) first and specify `groupBy` option for the `@grpc` operator. +In our protobuf example file, we have a method called `GetMultipleNews` that we can use. To enable batching we need to enable [`@upstream.batch` option](../operators/upstream.md#batch) first and specify `groupBy` option for the `@grpc` operator. ```graphql schema @@ -226,7 +227,7 @@ schema type Query { newsById(news: NewsInput!): News! @grpc( - method: "news.news.NewsService.GetNews" + method: "news.NewsService.GetNews" body: "{{args.news}}" # highlight-next-line groupBy: ["news", "id"] @@ -251,13 +252,13 @@ Restart the Tailcall server and make the query with multiple news separately, e. } ``` -Those 2 requests will be executed inside single request to gRPC method `GetMultipleNews` +Those 2 requests will be executed inside a single request to the gRPC method `GetMultipleNews` ## Conclusion Well done on integrating a gRPC service with the Tailcall gateway! This tutorial has demonstrated the straightforward and efficient process, showcasing Tailcall's compatibility with advanced communication protocols like gRPC. -You can find this working example and test it by yourself by next links: +You can find this working example and test it by yourself by the next links: - [node-grpc](https://github.com/tailcallhq/node-grpc) - example implementation for gRPC service in node.js - [gRPC example config](https://github.com/tailcallhq/tailcall/blob/main/examples/grpc.graphql) - Tailcall's config to integrate with gRPC service above From f3ce3c22205d32a4e836a661da74f4d1096b0151 Mon Sep 17 00:00:00 2001 From: Tushar Mathur Date: Tue, 27 Feb 2024 18:04:11 -0500 Subject: [PATCH 09/12] update grpc doc --- docs/guides/grpc.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/docs/guides/grpc.md b/docs/guides/grpc.md index 90dff21972..892db1d6f9 100644 --- a/docs/guides/grpc.md +++ b/docs/guides/grpc.md @@ -7,17 +7,13 @@ In this guide, we will set up a simple gRPC service and use it inside Tailcall's ## What is gRPC? -This guide assumes a basic familiarity with gRPC. - -**gRPC** is a high-performance framework created by Google for remote procedure calls (RPCs). Its key features include: +This guide assumes a basic familiarity with gRPC. It is a high-performance framework created by Google for remote procedure calls (RPCs). Its key features include: - **HTTP/2 Transport:** Ensures efficient and fast data transfer. - **Protocol Buffers (Protobuf):** Serves as a powerful interface description language. - **Efficiency:** Offers binary serialization, reduces latency, and supports data streaming. -This combination of features makes gRPC ideal for microservices and distributed systems. - -If you need a more detailed understanding or are new to gRPC, we recommend visiting the [official gRPC website](https://grpc.io/) for comprehensive documentation and resources. +This combination of features makes gRPC ideal for microservices and distributed systems. If you need a more detailed understanding or are new to gRPC, we recommend visiting the [official gRPC website](https://grpc.io/) for comprehensive documentation and resources. Now, let's explore how gRPC can be integrated into our proxy gateway to enhance communication and data exchange in distributed systems. @@ -154,7 +150,8 @@ type Query { Wrapping up the whole result config that may look like this: ```graphql -# highlight-start +# file: app.graphql + schema @server(port: 8000, graphiql: true) @upstream(baseURL: "http://localhost:50051", httpCache: true) @@ -166,7 +163,6 @@ type Query { news: NewsData! @grpc(method: "news.news.NewsService.GetAllNews") newsById(news: NewsInput!): News! @grpc(method: "news.news.NewsService.GetNews", body: "{{args.news}}") } -# highlight-end type News { id: Int @@ -184,6 +180,12 @@ type NewsData { } ``` +Simply start the server by pointing it to the config. + +``` +tailcall start ./app.graphql +``` + And now you can go to the page `http://127.0.0.1:8000/graphql` and execute some GraphQL queries e.g.: ```graphql From 3368d9a17f6b670979f4c96f5d9b7e1128f237db Mon Sep 17 00:00:00 2001 From: Tushar Mathur Date: Tue, 27 Feb 2024 18:09:12 -0500 Subject: [PATCH 10/12] chore: update styles --- docs/guides/grpc.md | 2 +- src/css/custom.css | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/guides/grpc.md b/docs/guides/grpc.md index 892db1d6f9..9b49ab98e3 100644 --- a/docs/guides/grpc.md +++ b/docs/guides/grpc.md @@ -212,7 +212,7 @@ Or } ``` -### Batching +## Batching Another important feature of the `@grpc` operator is that it allows you to implement request batching for remote data almost effortlessly as soon as you have gRPC methods that resolve multiple responses for multiple inputs in a single request. diff --git a/src/css/custom.css b/src/css/custom.css index 2bfbf26753..99436b8554 100644 --- a/src/css/custom.css +++ b/src/css/custom.css @@ -56,6 +56,7 @@ code { border: none; + letter-spacing: 0.00001px; vertical-align: baseline; } From 1035a0e9a0a307825996a77a327d64352c5c0f1c Mon Sep 17 00:00:00 2001 From: Tushar Mathur Date: Tue, 27 Feb 2024 18:14:57 -0500 Subject: [PATCH 11/12] whitespace changes --- docs/guides/grpc.md | 18 ++++-------------- docs/guides/n+1.md | 4 +--- 2 files changed, 5 insertions(+), 17 deletions(-) diff --git a/docs/guides/grpc.md b/docs/guides/grpc.md index 9b49ab98e3..86d7bb6dca 100644 --- a/docs/guides/grpc.md +++ b/docs/guides/grpc.md @@ -23,9 +23,7 @@ We need some gRPC service available to be able to execute requests from the Tail ### Protobuf definition -First, we need to create an example protobuf file that will define the structure of the data we want to transmit using gRPC. - -Here is the definition of `NewsService` that implements CRUD operations on news data that we'll put into the `news.proto` file. +First, we need to create an example protobuf file that will define the structure of the data we want to transmit using gRPC. Here is the definition of `NewsService` that implements CRUD operations on news data that we'll put into the `news.proto` file. ```protobuf syntax = "proto3"; @@ -72,24 +70,16 @@ service NewsService { ### Implement gRPC service -Now having the protobuf file you can write a server that implements `NewsService` at any language you want that supports gRPC. - -Tailcall organization has a sample node.js service inside [this repo](https://github.com/tailcallhq/node-grpc) that you can pull to your local machine - -To spin up the sample service just run inside the repo +Now having the protobuf file you can write a server that implements `NewsService` at any language you want that supports gRPC. Tailcall organization has a sample node.js service inside [this repo](https://github.com/tailcallhq/node-grpc) that you can pull to your local machine. To spin up the sample service just run inside the repo and wait for logs about the service running. ```sh npm i -node server +npm start ``` -and wait for logs about the service running. - ## Tailcall config -Now when we have a running gRPC service we're going to write Tailcall's config to make the integration. - -To do this we need to specify GraphQL types corresponding to gRPC types we have defined in the protobuf file. Let's create a new file `grpc.graphql` file with the following content: +Now when we have a running gRPC service we're going to write Tailcall's config to make the integration. To do this we need to specify GraphQL types corresponding to gRPC types we have defined in the protobuf file. Let's create a new file `grpc.graphql` file with the following content: ```graphql # The GraphQL representation for News message type diff --git a/docs/guides/n+1.md b/docs/guides/n+1.md index dea190f927..83cad886f2 100644 --- a/docs/guides/n+1.md +++ b/docs/guides/n+1.md @@ -122,13 +122,11 @@ type Post { # highlight-start query: [{key: "id", value: "{{value.userId}}"}] groupBy: ["id"] + # highlight-end ) - # highlight-end } ``` -### Understanding the Update - The described changes introduce significant tweaks to the `@http` directive and incorporate the `@groupBy` operator: - `query: [{key: "id", value: "{{value.userId}}"}]`: In this configuration, the TailCall CLI generates a URL that aligns the user id with the `userId` from the parent `Post`. For a batch of posts, the CLI compiles a single URL, such as `/users?id=1&id=2&id=3...id=10`, consolidating the requests into one. From e3db33c7a9d9238c2761bc6412098fae579ad02d Mon Sep 17 00:00:00 2001 From: Tushar Mathur Date: Tue, 27 Feb 2024 18:25:00 -0500 Subject: [PATCH 12/12] chore lint fixes --- .github/workflows/grammar-check.yml | 2 +- docs/guides/grpc.md | 12 ++++++------ 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/.github/workflows/grammar-check.yml b/.github/workflows/grammar-check.yml index e5cdd99b4b..274cd8f67e 100644 --- a/.github/workflows/grammar-check.yml +++ b/.github/workflows/grammar-check.yml @@ -23,4 +23,4 @@ jobs: run: npm install write-good - name: Check grammar - run: npx write-good docs/**/*.md --no-tooWordy + run: npx write-good docs/**/*.md --no-tooWordy --no-passive --no-illusion diff --git a/docs/guides/grpc.md b/docs/guides/grpc.md index 86d7bb6dca..074852814f 100644 --- a/docs/guides/grpc.md +++ b/docs/guides/grpc.md @@ -40,7 +40,7 @@ message News { string postImage = 4; } -// Message with just the id of a single news +// Message with the id of a single news message NewsId { int32 id = 1; } @@ -70,7 +70,7 @@ service NewsService { ### Implement gRPC service -Now having the protobuf file you can write a server that implements `NewsService` at any language you want that supports gRPC. Tailcall organization has a sample node.js service inside [this repo](https://github.com/tailcallhq/node-grpc) that you can pull to your local machine. To spin up the sample service just run inside the repo and wait for logs about the service running. +Now having the protobuf file you can write a server that implements `NewsService` at any language you want that supports gRPC. Tailcall organization has a sample node.js service inside [this repo](https://github.com/tailcallhq/node-grpc) that you can pull to your local machine. To spin up the sample service run inside the repo and wait for logs about the service running. ```sh npm i @@ -128,7 +128,7 @@ schema @link(id: "news", src: "./news.proto", type: Protobuf) Now you can connect GraphQL types to gRPC types. To do this you may want to explore more about [`@grpc` operator](../operators/grpc.md). Its usage is pretty straightforward and requires you to specify the path to a method that should be used to make a call. The method name will start with the package name, followed by the service name and the method name, all separated by the `.` symbol. -If you need to provide any input to the gRPC method call you can specify it with the `body` option that allows you to specify a Mustache template and therefore it could use any input data like `args` and `value` to construct the body request. The body value is specified in the JSON format if you need to create the input manually and cannot just use `args` input. +If you need to provide any input to the gRPC method call you can specify it with the `body` option that allows you to specify a Mustache template and therefore it could use any input data like `args` and `value` to construct the body request. The body value is specified in the JSON format if you need to create the input manually and cannot use `args` input. ```graphql type Query { @@ -170,7 +170,7 @@ type NewsData { } ``` -Simply start the server by pointing it to the config. +Start the server by pointing it to the config. ``` tailcall start ./app.graphql @@ -257,7 +257,7 @@ You can find this working example and test it by yourself by the next links: ### Key Takeaways -- **Simplicity of Integration:** The integration of gRPC with Tailcall is not only seamless but also enhances the overall capability of your system to handle high-performance and efficient data composition. +- **Simplicity of Integration:** The integration of gRPC with Tailcall seamlessly enhances the overall capability of your system to handle high-performance and efficient data composition. - **Scalability and Performance:** By leveraging the power of gRPC along with Tailcall, we've laid a foundation for building scalable and high-performing distributed systems. ### Next Steps @@ -265,4 +265,4 @@ You can find this working example and test it by yourself by the next links: With the basics in place, we encourage you to explore further: - **Dive Deeper:** Tailcall gateway offers a lot of other features and configurations that you can utilize. Dive deeper into our documentation to explore more advanced settings and customization options. -- **Explore Other Guides:** Our documentation is filled with various guides and tutorials that can help you leverage the full potential of Tailcall in different scenarios. Whether it's adding security layers, load balancing, or detailed logging, there's a lot more to explore. +- **Explore Other Guides:** Our documentation includes a variety of guides and tutorials that can help you leverage the full potential of Tailcall in different scenarios. Whether it's adding security layers, load balancing, or detailed logging, there's a lot more to explore.