Skip to content

Commit

Permalink
feat: Drop strict mode in favor of leniency flags. (#234)
Browse files Browse the repository at this point in the history 
* feat: Drop strict mode in favor of leniency flags.

* chore: Simplified build script.

* fix: Removed leftover.

* chore: Updated benchmark script.

* chore: Updated docs.
  • Loading branch information
@ShogunPanda
ShogunPanda authored Jul 31, 2023
1 parent 78af631 commit 6d04465
Show file tree
Hide file tree
Showing 23 changed files with 786 additions and 538 deletions.
54 changes: 47 additions & 7 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,7 @@ protocol support to highly non-compliant clients/server.
No `HPE_INVALID_HEADER_TOKEN` will be raised for incorrect header values when
lenient parsing is "on".
**USE AT YOUR OWN RISK!**
**Enabling this flag can pose a security issue since you will be exposed to request smuggling attacks. USE WITH CAUTION!**
### `void llhttp_set_lenient_chunked_length(llhttp_t* parser, int enabled)`
Expand All @@ -300,23 +300,22 @@ conjunction with `Content-Length`.
This error is important to prevent HTTP request smuggling, but may be less desirable
for small number of cases involving legacy servers.
**USE AT YOUR OWN RISK!**
**Enabling this flag can pose a security issue since you will be exposed to request smuggling attacks. USE WITH CAUTION!**
### `void llhttp_set_lenient_keep_alive(llhttp_t* parser, int enabled)`
Enables/disables lenient handling of `Connection: close` and HTTP/1.0
requests responses.
Normally `llhttp` would error on (in strict mode) or discard (in loose mode)
the HTTP request/response after the request/response with `Connection: close`
and `Content-Length`.
Normally `llhttp` would error the HTTP request/response
after the request/response with `Connection: close` and `Content-Length`.
This is important to prevent cache poisoning attacks,
but might interact badly with outdated and insecure clients.
With this flag the extra request/response will be parsed normally.
**USE AT YOUR OWN RISK!**
**Enabling this flag can pose a security issue since you will be exposed to poisoning attacks. USE WITH CAUTION!**
### `void llhttp_set_lenient_transfer_encoding(llhttp_t* parser, int enabled)`
Expand All @@ -331,7 +330,48 @@ avoid request smuggling.
With this flag the extra value will be parsed normally.
**USE AT YOUR OWN RISK!**
**Enabling this flag can pose a security issue since you will be exposed to request smuggling attacks. USE WITH CAUTION!**
### `void llhttp_set_lenient_version(llhttp_t* parser, int enabled)`
Enables/disables lenient handling of HTTP version.
Normally `llhttp` would error when the HTTP version in the request or status line
is not `0.9`, `1.0`, `1.1` or `2.0`.
With this flag the extra value will be parsed normally.
**Enabling this flag can pose a security issue since you will allow unsupported HTTP versions. USE WITH CAUTION!**
### `void llhttp_set_lenient_data_after_close(llhttp_t* parser, int enabled)`
Enables/disables lenient handling of additional data received after a message ends
and keep-alive is disabled.
Normally `llhttp` would error when additional unexpected data is received if the message
contains the `Connection` header with `close` value.
With this flag the extra data will discarded without throwing an error.
**Enabling this flag can pose a security issue since you will be exposed to poisoning attacks. USE WITH CAUTION!**
### `void llhttp_set_lenient_optional_lf_after_cr(llhttp_t* parser, int enabled)`
Enables/disables lenient handling of incomplete CRLF sequences.
Normally `llhttp` would error when a CR is not followed by LF when terminating the
request line, the status line, the headers or a chunk header.
With this flag only a CR is required to terminate such sections.
**Enabling this flag can pose a security issue since you will be exposed to request smuggling attacks. USE WITH CAUTION!**
### `void llhttp_set_lenient_optional_crlf_after_chunk(llhttp_t* parser, int enabled)`
Enables/disables lenient handling of chunks not separated via CRLF.
Normally `llhttp` would error when after a chunk data a CRLF is missing before
starting a new chunk.
With this flag the new chunk can start immediately after the previous one.
**Enabling this flag can pose a security issue since you will be exposed to request smuggling attacks. USE WITH CAUTION!**
## Build Instructions
Expand Down
132 changes: 57 additions & 75 deletions bench/index.ts
View file
Original file line number Diff line number Diff line change
@@ -1,89 +1,71 @@
// NOTE: run `npm test` to build `./test/tmp/http-loose-request`
import * as assert from "assert";
import { spawnSync } from "child_process";
import { existsSync } from "fs";
import { resolve } from "path";

import * as assert from 'assert';
import { spawnSync } from 'child_process';
import { existsSync } from 'fs';

const isURL = !process.argv[2] || process.argv[2] === 'url';
const isHTTP = !process.argv[2] || process.argv[2] === 'http';
function request(tpl: TemplateStringsArray): string {
return tpl.raw[0].replace(/^\s+/gm, '').replace(/\n/gm, '').replace(/\\r/gm, '\r').replace(/\\n/gm, '\n')
}

const requests: Map<string, string> = new Map();
const urlExecutable = resolve(__dirname, "../test/tmp/url-url-c");
const httpExecutable = resolve(__dirname, "../test/tmp/http-request-c");

const httpRequests: Record<string, string> = {
"seanmonstar/httparse": request`
GET /wp-content/uploads/2010/03/hello-kitty-darth-vader-pink.jpg HTTP/1.1\r\n
Host: www.kittyhell.com\r\n
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; ja-JP-mac; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 Pathtraq/0.9\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n
Accept-Language: ja,en-us;q=0.7,en;q=0.3\r\n
Accept-Encoding: gzip,deflate\r\n
Accept-Charset: Shift_JIS,utf-8;q=0.7,*;q=0.7\r\n
Keep-Alive: 115\r\n
Connection: keep-alive\r\n
Cookie: wp_ozh_wsa_visits=2; wp_ozh_wsa_visit_lasttime=xxxxxxxxxx; __utma=xxxxxxxxx.xxxxxxxxxx.xxxxxxxxxx.xxxxxxxxxx.xxxxxxxxxx.x; __utmz=xxxxxxxxx.xxxxxxxxxx.x.x.utmccn=(referral)|utmcsr=reader.livedoor.com|utmcct=/reader/|utmcmd=referral\r\n\r\n
`,
"nodejs/http-parser": request`
POST /joyent/http-parser HTTP/1.1\r\n
Host: github.com\r\n
DNT: 1\r\n
Accept-Encoding: gzip, deflate, sdch\r\n
Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4\r\n
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1)
AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/39.0.2171.65 Safari/537.36\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,
image/webp,*/*;q=0.8\r\n
Referer: https://github.com/joyent/http-parser\r\n
Connection: keep-alive\r\n
Transfer-Encoding: chunked\r\n
Cache-Control: max-age=0\r\n\r\nb\r\nhello world\r\n0\r\n\r\n
`
}
const urlRequest = "http://example.com/path/to/file?query=value#fragment";

if (!existsSync('./test/tmp/http-loose-request.c')) {
console.error('Run npm test to build ./test/tmp/http-loose-request');
if (!existsSync(urlExecutable) || !existsSync(urlExecutable)) {
console.error(
"\x1b[31m\x1b[1mPlease run npm test in order to create required executables."
);
process.exit(1);
}

requests.set('seanmonstar/httparse',
'GET /wp-content/uploads/2010/03/hello-kitty-darth-vader-pink.jpg HTTP/1.1\r\n' +
'Host: www.kittyhell.com\r\n' +
'User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; ja-JP-mac; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 Pathtraq/0.9\r\n' +
'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n' +
'Accept-Language: ja,en-us;q=0.7,en;q=0.3\r\n' +
'Accept-Encoding: gzip,deflate\r\n' +
'Accept-Charset: Shift_JIS,utf-8;q=0.7,*;q=0.7\r\n' +
'Keep-Alive: 115\r\n' +
'Connection: keep-alive\r\n' +
'Cookie: wp_ozh_wsa_visits=2; wp_ozh_wsa_visit_lasttime=xxxxxxxxxx; __utma=xxxxxxxxx.xxxxxxxxxx.xxxxxxxxxx.xxxxxxxxxx.xxxxxxxxxx.x; __utmz=xxxxxxxxx.xxxxxxxxxx.x.x.utmccn=(referral)|utmcsr=reader.livedoor.com|utmcct=/reader/|utmcmd=referral\r\n\r\n');

requests.set('nodejs/http-parser',
'POST /joyent/http-parser HTTP/1.1\r\n' +
'Host: github.com\r\n' +
'DNT: 1\r\n' +
'Accept-Encoding: gzip, deflate, sdch\r\n' +
'Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4\r\n' +
'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) ' +
'AppleWebKit/537.36 (KHTML, like Gecko) ' +
'Chrome/39.0.2171.65 Safari/537.36\r\n' +
'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,' +
'image/webp,*/*;q=0.8\r\n' +
'Referer: https://github.com/joyent/http-parser\r\n' +
'Connection: keep-alive\r\n' +
'Transfer-Encoding: chunked\r\n' +
'Cache-Control: max-age=0\r\n\r\nb\r\nhello world\r\n0\r\n\r\n');

if (process.argv[2] === 'loop') {
if (process.argv[2] === "loop") {
const reqName = process.argv[3];
assert(requests.has(reqName), `Unknown request name: "${reqName}"`);

const request = requests.get(reqName)!;
spawnSync('./test/tmp/http-loose-request', [
'loop',
request
], { stdio: 'inherit' });
const request = httpRequests[reqName]!;

assert(request, `Unknown request name: "${reqName}"`);
spawnSync(httpExecutable, ["loop", request], { stdio: "inherit" });
process.exit(0);
}

if (isURL) {
console.log('url loose (C)');

spawnSync('./test/tmp/url-loose-url-c', [
'bench',
'http://example.com/path/to/file?query=value#fragment'
], { stdio: 'inherit' });

console.log('url strict (C)');

spawnSync('./test/tmp/url-strict-url-c', [
'bench',
'http://example.com/path/to/file?query=value#fragment'
], { stdio: 'inherit' });
if (!process.argv[2] || process.argv[2] === "url") {
console.log("url (C)");
spawnSync(urlExecutable, ["bench", urlRequest], { stdio: "inherit" });
}

if (isHTTP) {
for(const [name, request] of requests) {
console.log('http loose: "%s" (C)', name);

spawnSync('./test/tmp/http-loose-request-c', [
'bench',
request
], { stdio: 'inherit' });

console.log('http strict: "%s" (C)', name);

spawnSync('./test/tmp/http-strict-request-c', [
'bench',
request
], { stdio: 'inherit' });
if (!process.argv[2] || process.argv[2] === "http") {
for (const [name, request] of Object.entries(httpRequests)) {
console.log('http: "%s" (C)', name);
spawnSync(httpExecutable, ["bench", request], { stdio: "inherit" });
}
}
124 changes: 35 additions & 89 deletions bin/generate.ts
View file
Original file line number Diff line number Diff line change
@@ -1,101 +1,47 @@
#!/usr/bin/env -S npx ts-node
import * as fs from 'fs';
import { LLParse } from 'llparse';
import * as path from 'path';
import * as semver from 'semver';

import * as llhttp from '../src/llhttp';

const pkgFile = path.join(__dirname, '..', 'package.json');
const pkg = JSON.parse(fs.readFileSync(pkgFile).toString());

const BUILD_DIR = path.join(__dirname, '..', 'build');
const C_DIR = path.join(BUILD_DIR, 'c');
const SRC_DIR = path.join(__dirname, '..', 'src');

const C_FILE = path.join(C_DIR, 'llhttp.c');
const HEADER_FILE = path.join(BUILD_DIR, 'llhttp.h');

for (const dir of [ BUILD_DIR, C_DIR ]) {
try {
fs.mkdirSync(dir);
} catch (e) {
// no-op
}
}

function build(mode: 'strict' | 'loose') {
const llparse = new LLParse('llhttp__internal');
const instance = new llhttp.HTTP(llparse, mode);

return llparse.build(instance.build().entry, {
c: {
header: 'llhttp',
},
debug: process.env.LLPARSE_DEBUG ? 'llhttp__debug' : undefined,
headerGuard: 'INCLUDE_LLHTTP_ITSELF_H_',
});
}

function guard(strict: string, loose: string): string {
let out = '';

if (strict === loose) {
return strict;
}

out += '#if LLHTTP_STRICT_MODE\n';
out += '\n';
out += strict + '\n';
out += '\n';
out += '#else /* !LLHTTP_STRICT_MODE */\n';
out += '\n';
out += loose + '\n';
out += '\n';
out += '#endif /* LLHTTP_STRICT_MODE */\n';

return out;
}

const artifacts = {
loose: build('loose'),
strict: build('strict'),
};

let headers = '';

headers += '#ifndef INCLUDE_LLHTTP_H_\n';
headers += '#define INCLUDE_LLHTTP_H_\n';

headers += '\n';

const version = semver.parse(pkg.version)!;
import { mkdirSync, readFileSync, writeFileSync } from 'fs';
import { LLParse } from 'llparse';
import { dirname, resolve } from 'path';
import { parse } from 'semver';
import { CHeaders, HTTP } from '../src/llhttp';

headers += `#define LLHTTP_VERSION_MAJOR ${version.major}\n`;
headers += `#define LLHTTP_VERSION_MINOR ${version.minor}\n`;
headers += `#define LLHTTP_VERSION_PATCH ${version.patch}\n`;
headers += '\n';
const C_FILE = resolve(__dirname, '../build/c/llhttp.c');
const HEADER_FILE = resolve(__dirname, '../build/llhttp.h');

headers += '#ifndef LLHTTP_STRICT_MODE\n';
headers += '# define LLHTTP_STRICT_MODE 0\n';
headers += '#endif\n';
headers += '\n';
const pkg = JSON.parse(
readFileSync(resolve(__dirname, '..', 'package.json')).toString(),
);
const version = parse(pkg.version)!;
const llparse = new LLParse('llhttp__internal');

const cHeaders = new llhttp.CHeaders();
const cHeaders = new CHeaders();
const nativeHeaders = readFileSync(resolve(__dirname, '../src/native/api.h'));
const generated = llparse.build(new HTTP(llparse).build().entry, {
c: {
header: 'llhttp',
},
debug: process.env.LLPARSE_DEBUG ? 'llhttp__debug' : undefined,
headerGuard: 'INCLUDE_LLHTTP_ITSELF_H_',
});

headers += guard(artifacts.strict.header, artifacts.loose.header);
const headers = `
#ifndef INCLUDE_LLHTTP_H_
#define INCLUDE_LLHTTP_H_
headers += '\n';
#define LLHTTP_VERSION_MAJOR ${version.major}
#define LLHTTP_VERSION_MINOR ${version.minor}
#define LLHTTP_VERSION_PATCH ${version.patch}
headers += cHeaders.build();
${generated.header}
headers += '\n';
${cHeaders.build()}
headers += fs.readFileSync(path.join(SRC_DIR, 'native', 'api.h'));
${nativeHeaders}
headers += '\n';
headers += '#endif /* INCLUDE_LLHTTP_H_ */\n';
#endif /* INCLUDE_LLHTTP_H_ */
`;

fs.writeFileSync(C_FILE,
guard(artifacts.strict.c || '', artifacts.loose.c || ''));
fs.writeFileSync(HEADER_FILE, headers);
mkdirSync(dirname(C_FILE), { recursive: true });
writeFileSync(HEADER_FILE, headers);
writeFileSync(C_FILE, generated.c);
1 change: 1 addition & 0 deletions package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
"clean": "rm -rf lib && rm -rf test/tmp",
"prepare": "npm run clean && npm run build-ts",
"lint": "tslint -c tslint.json bin/*.ts src/*.ts src/**/*.ts test/*.ts test/**/*.ts",
"lint-fix": "tslint --fix -c tslint.json bin/*.ts src/*.ts src/**/*.ts test/*.ts test/**/*.ts",
"mocha": "mocha --timeout=10000 -r ts-node/register/type-check --reporter progress test/*-test.ts",
"test": "npm run mocha && npm run lint",
"postversion": "RELEASE=`node -e \"process.stdout.write(require('./package').version)\"` make -B postversion",
Expand Down
3 changes: 0 additions & 3 deletions src/llhttp.ts
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
import * as constants from './llhttp/constants';

import HTTPMode = constants.HTTPMode;

export { constants };
export { HTTPMode };

export { HTTP } from './llhttp/http';
export { URL } from './llhttp/url';
Expand Down
Loading

0 comments on commit 6d04465

Please sign in to comment.