Is it better to consume not needed body or abort the request?

[Dzieni]

Let's say I want to add some basic validation to fetch to make my code DRY. Here is the naive implementation:

const fetchAndValidate = async (resource, options) => {
	const res = await fetch(resource, { ...options, signal: ac.signal });
	if (!res.ok) throw new Error(`HTTP ${res.status} - ${res.statusText}`);
	return res;
};

It's naive because we never consume the body. Now if I understand correctly, there are two solutions:

1. Consume the body before throwing:

const fetchAndValidate = async (resource, options) => {
	const res = await fetch(resource, options);
	if (!res.ok) {
		for await (const _ of res.body) {
		}
		throw new Error(`HTTP ${res.status} - ${res.statusText}`);
	}
	return res;
};

2. Abort request before throwing:

const fetchAndValidate = async (resource, options) => {
	const ac = new AbortController();
	const res = await fetch(resource, { ...options, signal: ac.signal });
	if (!res.ok) {
		ac.abort()
		throw new Error(`HTTP ${res.status} - ${res.statusText}`);
	}
	return res;
};

The second one seems more reasonable to me as we don't download data we don't really need, but I wanna make sure it is equally safe to use in terms of garbage collection, connection re-use etc.

So, which solution would you choose and why?

[metcoder95]

AFAIA, both have a similar impact on the way the whole fetch process is aborted, but one has a different impact over the other on the user's code.

Meanwhile, the first option requires you to iterate and wait for each chunk to be consumed (using an AsyncIterator); the second option internally makes the body be discarded immediately to can be GC'ed.

Given their effects, both are somehow interchangeable. And I'd have a preference for the latter.

[Dzieni]

And what about the connection re-use? If you abort the request, do you drop the TCP connection or what?

There is a guy on StackOverflow having the same doubts as myself, unfortunately he didn't get an answer :(

[metcoder95]

And what about the connection re-use? If you abort the request, do you drop the TCP connection or what?

Noup, that's different. The request body is thrown away but the connection (if keep-alive hasn't been disabled) will remain open for the origin.
If you want to also customize the behaviour of the TCP settings, you can provide a custom agent to fetch. e.g.

import { fetch, Agent } from 'undici'

const res = await fetch('https://example.com', {
  // Mocks are also supported
  dispatcher: new Agent({
    keepAliveTimeout: 10,
    keepAliveMaxTimeout: 10
  })
})
const json = await res.json()
console.log(json)

[metcoder95]

For the StackOverflow question, I'm not sure about the intrinsics of the Browsers, but I believe that beyond the GC that is defined in the fetch spec, each implementation is somehow "free" to manage the connections settings at their own

[ronag]

I would recommend the following:

async function dump(body) {
  try {
    let limit = 1e5;
    for await (const buf of body) {
       limit -= buf.byteLength;
       if (limit < 0) { 
         return;
       }
    }
  } catch {
    // Do nothing...
  }
}

const fetchAndValidate = async (resource, options) => {
	const res = await fetch(resource, options);
	if (!res.ok) {
		dump(res.body)
		throw new Error(`HTTP ${res.status} - ${res.statusText}`);
	}
	return res;
};

That way you don't lose keep-alive connections for small replies.

[Dzieni]

And what happens if the body length is higher than 1e5? From my understanding, this code would leave the body only partially consumed?

[ronag]

No, the for await loop finalizer will destroy the body.

[Dzieni]

I'm having a hard time understanding what happens under the hood in those scenarios. Please confirm I get it correctly:

• The HTTP 1.1 standard does not offer a way to say "you know what, you can throw that response away" other than closing the TCP connection.
• Aborting a request means that we have to stop downloading and say to the server we're fine with not getting the response. Since there is no other way than closing on TCP level, Undici does this as soon as the response body is destroyed.
• That's why you propose a compromise - consuming the first 100 KB. Thanks to this most of the response bodies will be fully consumed (since it's not common that you get a >100 KB of an error message) and there will be no need to close TCP.

[jdalton]

Is this dump() helper equiv to res.body?.dump()?
https://github.com/nodejs/undici/pull/1118/files

[metcoder95]

You use either res.body.dump or the dumpInterceptor to discard the request's body