Document how to write a Fastify/Node.js tunnel that handles replay data

[GeekCornerGH]

Is there an existing issue for this?

• I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• I have reviewed the documentation https://docs.sentry.io/
• I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/react

SDK Version

7.69.0

Framework Version

React 18.2.0, Vite 4.4.9

Link to Sentry event

No response

SDK Setup

import * as Sentry from "@sentry/react";

Sentry.init({
    dsn: import.meta.env.VITE_SENTRY_DSN,
    integrations: [new Sentry.BrowserTracing(), new Sentry.Replay({
        maskAllText: false,
        blockAllMedia: true,
    })],
    release: 1.0.0,
    environment: import.meta.env.MODE,
    // Performance Monitoring
    tracesSampleRate: 1.0, // Capture 100% of the transactions, reduce in production!
    // Session Replay
    replaysSessionSampleRate: 0, // This sets the sample rate at 10%. You may want to change it to 100% while in development and then sample at a lower rate in production.
    replaysOnErrorSampleRate: 1.0, // If you're not already sampling the entire session, change the sample rate to 100% when sampling sessions where errors occur.
    tunnel: "http://127.0.0.1:3001/tunnel"
});

Steps to Reproduce

1. Create a Fastify app
2. Create a route which should proceed data. No need to actually do anything with it, just parse body. Example:

fastify.post("/tunnel", async (request, reply) => {
        try {
            const envelope = request.body as string;
            const piece = envelope.split("\n")[0];
            const header = JSON.parse(piece);
        }
        catch {
            return reply.code(204).send();
        }
        return reply.code(204).send();
    });

3. Open a react project with tunnel endpoint, and open it in your browser.

Expected Result

Fastify should return a 204

Actual Result

{"level":30,"time":1695121324894,"pid":1984,"hostname":"redacted","reqId":"req-4","res":{"statusCode":500},"responseTime":2.3399999737739563,"msg":"request completed"}```

[Lms24]

Hi @GeekCornerGH thanks for writing in!

Is the resulting 500 error one you return from your fastify endpoint or is this returned by Sentry?

Are you able to identify what kind of envelope causes this errors? i.e. one for an error, transaction or replay?
If have a hunch that this might be related to Replay sending compressed data but not sure yet.

[GeekCornerGH]

Hi @Lms24,
According to my devtools, the envelope causing issues seems to be a replay_event type.
Hope it helps!

[Lms24]

To make sure this is it, can you try setting the useCompression option to false in the Replay constructor options?

// from your Sentry.init:

new Sentry.Replay({
    maskAllText: false,
    blockAllMedia: true,
    useCompression: false,
})

Curious if this changes something

[GeekCornerGH]

It indeed fixes the issue. This is a bug that should be fixed in a future release, right?

[Lms24]

It's not directly a bug because usually we actually want to send replay data compressed (which is why it's enabled by default).

We should provide a better example how to handle this in the tunnel endpoint. cc @billyvg I think this came up before but I can't find the respective issue/discussion. Do you by chance remember it? Curious if there's a specific workaround/adjustment we recommend for tunnel endpoints.

@GeekCornerGH our of curiousity, which guide did you follow to set up your fastify endpoint?

[GeekCornerGH]

Python example, and adapted it to typescript myself

[GeekCornerGH]

Figured out I've done fastify/help#158 (comment) for stripe, maybe we need to do the same for replays? (Asking because I can't test it right now)

[mydea]

We'd have to dig into this - I think it would be nice to have an example of how to do this in plain JS, ideally one that works in fastify but also express etc. I looked into this superficially and it seems it is sadly not that straightforward to achieve this, as it seems harder than it should be to parse the binary encoded body being sent & forward it correctly 😬 I'll put this into our backlog, PRs would be very welcome on this, I'd be happy to review a contribution! But I'd also accept an example that works in fastify only.

[GeekCornerGH]

I’ll pr both once I get fastify working. Edit: Express should work with my current code, but I'll have to take a look regarding requests made to fastify

[GeekCornerGH]

it seems it is sadly not that straightforward to achieve this, as it seems harder than it should be to parse the binary encoded body being sent & forward it correctly

Didn't tried to send to sentry right now, but on Fastify, it seems to be as easy as adding a * content type header

[GeekCornerGH]

Aaaaand there we go! Before I PR this, any plans to put the examples repo in hacktoberfest?

[mydea]

Aaaaand there we go! Before I PR this, any plans to put the examples repo in hacktoberfest?

Awesome! Yes, absolutely - we added the hacktoberfest topic to the example repo, great callout 🎉

[GeekCornerGH]

Awesome! Yes, absolutely - we added the hacktoberfest topic to the example repo, great callout 🎉

You're welcome. Hopefully my pr is fine, I had to remove the TypeScript stuff since you asked for JS examples

[GeekCornerGH]

There we go. Again, thank you for the help, and for providing a such tool for free! Genuinely appreciated.

[mydea]

thanks to @GeekCornerGH we now have tunneling examples for both fastify and express! thank you!

[GeekCornerGH]

Glad I could help

[darioackermann]

I hope this is seen by anyone.
I just wanted to point out this should make it to the docs. It cost us hours to realize that although sentry seemed to have accepted our requests, what we sent was empty because we were missing

	fastifyAdapter.getInstance().addContentTypeParser("*", { parseAs: "buffer" }, function (req, body, done) {
		done(null, body);
	});

in our codebase.

Wouldn't it be possible for sentry to throw a 400, let's say, if an empty payload is transmitted?
Currently it returns 200 OK with {} which is super unintuitive (I figured that would be fine.)

[mydea]

So our example for fastify does include this now: https://github.com/getsentry/examples/blob/master/tunneling/fastify/index.js

We do not have node examples in docs (yet), but I opened a PR to do so here: getsentry/sentry-docs#8144

The reason I have not merged this yet is because I am not sure the express example actually works - I haven't found time to really try it out yet. But for now, it's best to refer to the example implementations like the one I linked above!

[darioackermann]

I have opened issue #9582 to investigate what I'm currently seeing: The tunnel does not seem to successfully transmit replay events