Fastify / NodeJS tunnel does not seem to tunnel replays correctly

[darioackermann]

Is there an existing issue for this?

• I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• I have reviewed the documentation https://docs.sentry.io/
• I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/browser

SDK Version

7.80.0

Framework Version

Node + Fastify (backend), @sentry/nextjs

Link to Sentry event

No response

SDK Setup

Sentry.init({
  dsn: "dsn_url"

  // Adjust this value in production, or use tracesSampler for greater control
  tracesSampleRate: 1,

  // Setting this option to true will print useful information to the console while you're setting up Sentry.
  debug: true,

  // Set tunnel
  tunnel: `${process.env.NEXT_PUBLIC_API_HOST}/my-fancy-tunnel`,

  replaysOnErrorSampleRate: 1.0,

  // This sets the sample rate to be 10%. You may want this to be 100% while
  // in development and sample at a lower rate in production
  replaysSessionSampleRate: 1.0,

  // You can remove this option if you're not planning to use the Sentry Session Replay feature:
  integrations: [
    new Sentry.Replay({
      // Additional Replay configuration goes in here, for example:
      maskAllText: true,
      blockAllMedia: true,
    }),
    new HttpClient(),
    /*new CaptureConsole({
      levels: ['warn', 'error', 'assert']
    }),*/
    new ReportingObserver(),
  ],

  release: `mypackage@${process.env.NEXT_PUBLIC_VERSION}`,

  environment: process.env.NODE_ENV || "development",
})

Backend setup

Fastify Setup

export async function bootstrap() {
   // ....
	let fastifyAdapter = new FastifyAdapter({
		// ...
	});

	// Needed for sentry.
	fastifyAdapter.getInstance().addContentTypeParser("*", { parseAs: "buffer" }, function (req, body, done) {
		done(null, body);
	});
}

Sentry Controller (Nest)

@Controller("my-fancy-tunnel/")
export class SentryController {
	@Public()
	@Post()
	@ApiOperation({
		summary: "Tunnels a request to Sentry",
		tags: ["sentry"],
	})
	@ApiResponse({ status: 200 })
	async tunnel(@Req() req, @Res({ passthrough: true }) res): Promise<{message: string}> {
		try {
			const envelope = req.body;
			const pieces = envelope.toString().split("\n");
			const header = JSON.parse(pieces[0]);
			const { host, pathname, username } = new URL(header.dsn);
			const projectId = pathname.slice(1);
			const url = `https://${host}/api/${projectId}/envelope/`;

			console.log(url)

			const res = await fetch(url, {
				method: "POST",
				body: envelope,
				headers: {
					"Content-Type": "application/x-sentry-envelope",
				},
			});
			return await res.json();
		} catch (e) {
			const error = e?.response || e?.message;
			throw new BadRequestException(error);
		}
	}
}

Steps to Reproduce

1. Use sentry without tunnel option
2. Verify that issues and replays work
3. Use sentry with tunnel option and setup backend according to https://github.com/getsentry/examples/blob/master/tunneling/fastify/index.js
4. Issues are still transmitted to sentry over the tunnel, but replays are not

Expected Result

Replays arrive at Sentry when using a tunnel

Actual Result

Replays do not arrive at Sentry when using a tunnel

Additional Information

• I have already tried turning compression off.
• The transmission seems to work at least partially, as sentry returns {id: "sessionhash/eventhash"}

Sample request to tunnel capture from network tab

fetch("https://my-api.com/my-fancy-tunnel", {
  "headers": {
    //...
  },
  "body": "{\"event_id\":\"df5efc38dfa844b1b12cd5100ce643ee\",\"sent_at\":\"2023-11-16T09:49:30.228Z\",\"sdk\":{\"name\":\"sentry.javascript.nextjs\",\"version\":\"7.80.0\"},\"dsn\":\"https://[email protected]/6484952\"}\n{\"type\":\"replay_event\"}\n{\"type\":\"replay_event\",\"replay_start_timestamp\":1700128107.113,\"timestamp\":1700128170.225,\"error_ids\":[],\"trace_ids\":[\"d1cdfdd971a845a1a0e745889c635ed1\"],\"urls\":[],\"replay_id\":\"df5efc38dfa844b1b12cd5100ce643ee\",\"segment_id\":5,\"replay_type\":\"session\",\"request\":{\"url\":\"https://local.internal.virtualalpha.io:8080/\",\"headers\":{\"User-Agent\":\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36\"}},\"event_id\":\"df5efc38dfa844b1b12cd5100ce643ee\",\"environment\":\"development\",\"release\":\"[email protected]\",\"sdk\":{\"integrations\":[\"InboundFilters\",\"FunctionToString\",\"TryCatch\",\"Breadcrumbs\",\"GlobalHandlers\",\"LinkedErrors\",\"Dedupe\",\"HttpContext\",\"Replay\",\"HttpClient\",\"ReportingObserver\",\"RewriteFrames\",\"BrowserTracing\"],\"name\":\"sentry.javascript.nextjs\",\"version\":\"7.80.0\"},\"tags\":{\"runtime\":\"browser\"},\"user\":{\"id\":\"d01c0224-1b5d-496c-8cd6-112e20706b68\",\"username\":\"name\",\"email\":\"email\"},\"contexts\":{\"state\":{\"state\":{\"type\":\"redux\",\"value\":{\"api\":\"redacted\",\"games\":{\"717008ab-4503-496a-a38c-4f0eb25ab6a6\":{\"id\":\"717008ab-4503-496a-a38c-4f0eb25ab6a6\",\"is_finished\":false,\"is_started\":true}},\"flags\":{\"isServiceUnavailable\":false},\"layout\":{\"desktopSidebarState\":\"expanded\",\"mobileSidebarState\":\"closed\"},\"documents\":{\"admin\":[],\"player\":[]},\"preferences\":{\"theme\":\"light\",\"locale\":\"DE\"},\"me\":\"redacted\",\"_persist\":{\"version\":-1,\"rehydrated\":true}}}}},\"platform\":\"javascript\"}\n{\"type\":\"replay_recording\",\"length\":338}\n{\"segment_id\":5}\nx���ÝjÃ0\f�ßE×Y\u0017Ûùõ\u0013�1ØEw7Jq\u0013§x«cÏv YÉ»OiZÚÀÂ�¯�t¤ïHz?Aè­\u0004Î\"¨E\u0010ÀOàMç*\f�\b¬ñ*(Ózà¨<b,fY\u0004=ð�$\u0011¨\u001axÆð\u0013��¯Mãe\u0000\u001e\u000f�a�ø ´Å�<�\t-HV\u0014¤\u0018¢�\u00123�B\u0004\u0012�^�Y>\u0007>$é2³d�Ý�Øâ\u0017Q\u001e¯(½¹\tb\u000f\u001c¬t�qZ´�\\[Ñ\u0002Z\u0013ýÁ\btp\u0002�Õ ¥6®ÇD-}å�\u001dmß��âÂÛ\"N¶õròêåÒ\f�\u001fþI\n»VßòEi�c'´Lò8%)Ê�\tâð¼¾*�SVäxª2� ó²�¥(IRV&Ù0¾?WCéÝ¡¦Õì�\u0014uå:½�oeiµØ`b�j\u001b�5�\br��\u000b�¬»ãJTçåÝ�&ývëÂç×c�eÕ8\u0007��7?\u001c Ö_",
  "method": "POST"
});

Response:

{"id":"df5efc38dfa844b1b12cd5100ce643ee"}

[mydea]

Hey,

In next.js, you can use the provided tunnelRoute option to automatically set up a tunnel for you:

https://docs.sentry.io/platforms/javascript/guides/nextjs/manual-setup/#configure-tunneling-to-avoid-ad-blockers

Did you just not find this option, or do you specifically want to use a custom tunnel instead?

(This is not documented ideally everywhere in the docs yet, we'll look into this)

In regard to your issue concretely, to be clear, is it also not working if you disable compression? That would be weird, because then nothing should be compressed and this should be straightforward enough... 🤔

Also just a note, you should definitely check the projectId if it is valid (like in the example), to prevent misuse of this!

TBH I don't 100% know how your fastify + nest setup works together. Can it be that the content parser is not properly applied there, or something like this?

[darioackermann]

In next.js, you can use the provided tunnelRoute option to automatically set up a tunnel for you:
https://docs.sentry.io/platforms/javascript/guides/nextjs/manual-setup/#configure-tunneling-to-avoid-ad-blockers

We cannot use this option as I presume this will add a route to the next server, we however do a static export of our project and there will be no server present

In regard to your issue concretely, to be clear, is it also not working if you disable compression? That would be weird, because then nothing should be compressed and this should be straightforward enough... 🤔

Good news: After deleting all build caches, it seems to work when useCompression: false.

We have now upgraded our Nest Version to the ^10 range and the issue seems to be gone even for the compressed case. Potentially there was some bug in an interface between nest and fastify.

Also just a note, you should definitely check the projectId if it is valid (like in the example), to prevent misuse of this!

Definitely - I've stripped down the code to eliminate potential show-stoppers to debug this issue.

I'm closing this and I hope somebody else will find it to be useful :)

[mydea]

Ah, makes sense - thank you for the investigation work! Maybe somebody else will also run into this in the future 😅