Support API mocking via MSW

[timostamm]

Is your feature request related to a problem? Please describe.
When writing a web application that makes many API calls, it is often challenging to get good code coverage without mocking the API.

Describe the solution you'd like
The Mock Service Worker library can intercept requests on the network level and mock responses in a framework-agnostic way, and it runs on Node.js too.

MSW can already be used with Connect, but it requires mocking on the HTTP level, without any benefits that the schema provides: For example, it is easy to misspell a header name or request path - even though both are already well-defined by the schema or protocol.

It would be fantastic if there was a type-safe integration for MSW with Connect to remove the boilerplate, for example:

import { setupWorker } from 'msw'
import { service } from '@connectrpc/connect-msw'

const worker = setupWorker(
  service(ElizaService, {
    say(req) {
      return { sentence: `you said ${req.sentence}` }
    }
  }),
)

worker.start()

Describe alternatives you've considered
@connectrpc/connect-playwright provides API mocking with Playwright, but that means all tests must be written in Playwright, limiting the options.

Additional context

• Prior effort for gRPC-web: mswjs/msw#238 and a blog post.
• Ongoing effort to use fetch API primities in MSW: mswjs/msw#1436

[paul-sachs]

I like the basic API, the one thing I'd like to figure out is how to apply a baseUrl. Ideally we'd only have to define it once (in the transport) but there's no real good way to extract that. Actually that would probably be a bad idea anyways since services could have diff baseUrls anyways. So I think the second arg might need to be an options object:

const worker = setupWorker(
  ...service(ElizaService, { baseUrl: "/api" }, {
    say(req) {
      return { sentence: `you said ${req.sentence}` }
    }
  }),
)

Though it's not quite as attractive an api, and it gets worse with multiple services:

const worker = setupWorker(
  ...service(ElizaService, { baseUrl: "/api" }, {
    say(req) {
      return { sentence: `you said ${req.sentence}` }
    }
  }),
  ...service(BigIntService, { baseUrl: "/api" }, {
    add() {
      return { result: 2n };
    }
  )
)

Though maybe that's fine. I thought of pushing the options arg to the last, but the service impl will always be the most important (and largest) argument so I feel like it shouldn't be stuck in the center of argument list.

[timostamm]

I should have been more clear that the code snippet was really just a pseudo-code example to illustrate that users don't have to hand-write requests paths and serialization code - a pretty nice benefit from the schema.

We'll certainly need to accept several options (the baseUrl you mention, but also serialization options, and likely also server side interceptors from #527). I'm sure that a reasonable API will manifest, even if it might need a couple of iterations 🙂

[nguyenyou]

MSW can already be used with Connect, but it requires mocking on the HTTP level

Hi @timostamm, do you have any examples using this approach? Thank you very much!

[timostamm]

@nguyenyou, the Connect protocol is very simple, especially for unary and JSON. This should mock the rpc Say of the demo service Eliza:

import { http, HttpResponse } from 'msw'
 
export const handlers = [
  http.post('connectrpc.eliza.v1.ElizaService/Say', async (request) => {
    const requestJson = await request.json();
    return HttpResponse.json({
      sentence: `you said: ${requestJson.sentence}`
    })
  }),
]

(Based on the current examples on mswjs.io)

[martines3000]

@timostamm
Hi. I have a question regarding the proposed solution you provided.

I tried it with my code, and MSW is not intercepting the requests.
Could this maybe be related to me using [email protected] ?

This is how I call my RPC method:

const transport = createConnectTransport({
  baseUrl: `http://127.0.0.1:3003/grpc`,
  httpVersion: '1.1',
});
const client = createPromiseClient(ExampleService, transport);
return client.hello({ name: 'bob' });

This is how I tried mocking it (I tried all possible URL combinations, because I thought the problem was there, but now I'm not sure anymore):

const mswServer = setupServer();
mswServer.listen({ onUnhandledRequest: 'error' });

const handler = http.post(
  'http://127.0.0.1:3003/grpc/martines3000.example.v1.ExampleService/Hello',
  // eslint-disable-next-line @typescript-eslint/require-await
  async (request) => {
    console.log('request', request);
    return HttpResponse.arrayBuffer(
      new HelloResponse({ message: 'Does not work' }).toBinary(),
      { status: 200 },
    );
  },
);

mswServer.use(handler);

The handler code is a little bit different, but the problem is that it never gets to this part (MSW never intercepts the request).

Do you maybe have any ideas where I should look ?

The PR #830 implementation will probably solve all of this issues and making mocking much easier, right ?

Thanks for the help!

[boan-anbo]

@timostamm Hi. I have a question regarding the proposed solution you provided.

@martines3000

I got it working with MSW 2.0.3 using the #830 PR's util method like this in Jest test set up.

import { setupServer } from 'msw/node'

export const handlersTest = service(
   UnitKeywordService,
   {
   	baseUrl: 'http://127.0.0.1:23012',
   },
   {
   	querySomething: () => {
   		return {
   			something: new SampleSomething(),
   		}
   	},
   }
)

export const mockServer = setupServer(...handlersTest)

beforeAll(() => mockServer.listen())

You can find the service method here:

connect-es/packages/connect-msw/src/service.ts

Line 9 in 9c384c0

export function service<T extends ServiceType>(

[timostamm]

@martines3000, yes, the PR will make this much easier and less error-prone. We'll pick it up again soon.

You're using createConnectTransport from @connectrpc/connect-node, which is using the Node.js built-in modules http and http2. msw can only intercept requests that use fetch() for HTTP requests.

You have two options in this case:

1. Use createConnectTransport from @connectrpc/connect-web instead for your clients. It will use fetch() for HTTP requests, and that works very well on recent versions of Node.js.
2. Continue to use createConnectTransport from @connectrpc/connect-node, but use the router transport for testing instead of msw. It's documented here.

I think this is actually an important point. We have to be very clear that MWS only intercepts fetch(). That's not self-explanatory just from reading the MSW docs or this issue description.

[boan-anbo]

@timostamm Hi. I have a question regarding the proposed solution you provided.

@martines3000

I got it working with MSW 2.0.3 using the #830 PR's util method like this in Jest test set up.

import { setupServer } from 'msw/node'

export const handlersTest = service(
   UnitKeywordService,
   {
   	baseUrl: 'http://127.0.0.1:23012',
   },
   {
   	querySomething: () => {
   		return {
   			something: new SampleSomething(),
   		}
   	},
   }
)

export const mockServer = setupServer(...handlersTest)

beforeAll(() => mockServer.listen())

You can find the service method here:

connect-es/packages/connect-msw/src/service.ts

Line 9 in 9c384c0

export function service<T extends ServiceType>(

However, I do have a related issue using the above method, which completely confuses me.

When I use the exact same setup in Storybook with its msw add-on

mswjs/msw-storybook-addon#121 (comment)

The mocked call errs with a ConnectError","message":"[deadline_exceeded] the operation timed out by the browser worker.

And if compare the request received by Jest version of createResponseResolver and the one received by storybook msw-addon setupWorker createResponseResolver, the timeout header are different:

• With default timeout ms = 12345m

export const grpcWebTransport = createGrpcWebTransport({
	baseUrl: 'http://127.0.0.1:23012',
	useBinaryFormat: true,
	defaultTimeoutMs: 12345,
})

const headerGrpcTimeout = request.headers.get('grpc-timeout')
console.log('headerGrpcTimeout', headerGrpcTimeout)

the Jest resolver receives:

12345m

while the storybook msw-addon setup receives:

() => date.getTime() - Date.now()m`. 

// or

() => void m

which causes the timeout error (?)

I don't know why this happens. Cause MSW add-on doesn't seem to do anything different, and both are using the exact same client and unary method call. But somehow one's GrpcTiemout header is incorrect.

[martines3000]

Thanks @timostamm for the quick reply and the solution. It works now 💯 .
Also thanks @boan-anbo for the suggestion, will try it out.