proxy_service web port behind haproxy in http mode with saml

[programmerq]

When running haproxy in http mode (as opposed to tcp mode), it has a limit in the size of the response headers that can be returned from the upstream application server.

In the case of Teleport's web port, this can be a problem if you are using SAML authentication.

When the SAML provider calls the teleport callback URL, it returns a fairly large header with the SAMLResponse. This means that when you log in to the Web UI via SAML, that request response is pretty close to the 16KiB limit that haproxy has by default. When logging in via tsh login --auth=saml, haproxy will return a 502 when the browser gets to the teleport callback URL. This is because in the case of the CLI login, the callback URL must issue a redirect back to the 127.0.0.1 server that the tsh login binary is running locally. This Location header has all the same information that's in the SAMLResponse, but embedded in the Location URL. This pushes the total response header size up to about 22KiB, which triggers the haproxy response header buffer size limit, and haproxy simply returns a 502 to the client.

The limit in question is documented here: https://cbonte.github.io/haproxy-dconv/2.5/configuration.html#tune.bufsize

If an HTTP request is larger than (tune.bufsize - tune.maxrewrite), HAProxy will return HTTP 400 (Bad Request) error. Similarly if an HTTP response is larger than this size, HAProxy will return HTTP 502 (Bad Gateway).

HAProxy may be recompiled to raise this limit as a workaround.

Another alternative is to use haproxy in tcp mode instead of http mode.