Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ngx.fetch() breaks when the HTTP(S) reply contains header names with underscores #856

Open
2 tasks done
fabriziofiorucci opened this issue Feb 13, 2025 · 0 comments
Open
2 tasks done

ngx.fetch() breaks when the HTTP(S) reply contains header names with underscores #856

fabriziofiorucci opened this issue Feb 13, 2025 · 0 comments
Assignees
@xeioex
Labels
bug

Comments

@fabriziofiorucci
Copy link

fabriziofiorucci commented Feb 13, 2025 •
edited
Loading

Describe the bug

If the HTTP(S) URL requested through ngx.fetch returns a reply that contains HTTP headers with underscores in their name, ngx.fetch breaks.

Reproducible with njs 0.8.9

  • The bug is reproducible with the latest version of njs.
  • I minimized the code and NGINX configuration to the smallest possible to reproduce the issue.

To reproduce

Steps to reproduce the behavior:

  1. Define a test "backend" server on NGINX:
server {
    server_name vm-test.ie.ff.lan;
    listen 80;

    location / {
        limit_except GET POST {
            allow all;
        }

        add_header DUMMY_HEADER_WITH_UNDERSCORES "blahblah";
        
        return 200 '{"dummy":123}';
    }
}
  1. Define the "main" NGINX server that uses the njs function with ngx.fetch()
js_import lab from conf.d/lab.js;

server {
  listen 80;
  server_name sms.nginx.lab;

  location /get_token_fetch {
    js_content lab.getAccessTokenFetch;
  }
}
  1. Define the njs function
export default {
  getAccessTokenFetch
};

async function getAccessTokenFetch(r) {
  ngx.log(ngx.INFO, '--- getAccessTokenFetch');
  var accessTokenReply = await ngx.fetch('http://vm-test.ie.ff.lan',
    {
      verify: false,
      method: 'POST',
      headers:
      {
        'Authorization': 'Basic SUtJQTc1MTRDMjg4MENEOTc1QjkwMTA2Mjc1QTRERDM1QjVGOTVBQ0FERDc6MUYyMzhCRDAwRjgwMERERkY3RkY1NEFCNDk5RjZDRUMxMkU2ODU5MQ==',
        'Content-Type': 'application/x-www-form-urlencoded'
      },
      body: 'scope=profile&grant_type=client_credentials'
    });

    var accessTokenPayload = await accessTokenReply.text();
    ngx.log(ngx.INFO,'.. AccessTokenPayload [' + accessTokenPayload + ']');

    var accessTokenJson = JSON.parse(accessTokenPayload);
    var accessTokenStatus = accessTokenReply.status;

    ngx.log(ngx.INFO,'.. AccessToken [' + accessTokenStatus + '] [' + accessTokenPayload + ']');

  r.status = 200;
  r.sendHeader();
  r.send(accessTokenPayload);
  r.finish();
}
  1. Trigger the issue:
$ curl -i sms.nginx.lab/get_token_fetch
HTTP/1.1 500 Internal Server Error
Server: nginx/1.27.2
Date: Wed, 12 Feb 2025 10:52:01 GMT
Content-Type: text/html
Content-Length: 177
Connection: close

<html>
<head><title>500 Internal Server Error</title></head>
<body>
<center><h1>500 Internal Server Error</h1></center>
<hr><center>nginx/1.27.2</center>
</body>
</html>

The NGINX error log reports:

2025/02/12 10:50:53 [info] 380702#380702: *319 client 192.168.2.18 closed keepalive connection
2025/02/12 10:52:01 [info] 380702#380702: *321 js: --- getAccessTokenFetch

Removing the HTTP header with underscores and testing the "working" case:

  • Comment out the line add_header DUMMY_HEADER_WITH_UNDERSCORES "blahblah"; in the "backend" server at step 1 above
  • Reload the NGINX configuration using nginx -s reload
  • Send the request again
curl .i sms.nginx.lab/get_token_fetch

The response is now correct:

HTTP/1.1 200 OK
Server: nginx/1.27.2
Date: Wed, 12 Feb 2025 10:50:53 GMT
Content-Type: application/octet-stream
Transfer-Encoding: chunked
Connection: keep-alive

{"dummy":123}

Expected behavior

When the line add_header DUMMY_HEADER_WITH_UNDERSCORES "blahblah"; is uncommented, the request:

curl .i sms.nginx.lab/get_token_fetch

Should return:

HTTP/1.1 200 OK
Server: nginx/1.27.2
Date: Wed, 12 Feb 2025 10:50:53 GMT
Content-Type: application/octet-stream
Transfer-Encoding: chunked
Connection: keep-alive

{"dummy":123}

Your environment

  • njs 0.8.9
  • NGINX Plus R33-p2
  • the only enabled NGINX module is modules/ngx_http_js_module.so;
  • OS: irrelevant, but tested on Ubuntu 22.04
  • Note: vm-test.ie.ff.lan and sms.nginx.lab both resolve to the IP address of the NGINX instance
xeioex added a commit to xeioex/njs that referenced this issue Feb 22, 2025
@xeioex
Fetch: accepting response headers with underscore characters.
a660ad6 
This fixes nginx#856 on Github.
@xeioex xeioex self-assigned this Feb 22, 2025
xeioex added a commit to xeioex/njs that referenced this issue Feb 22, 2025
@xeioex
Fetch: accepting response headers with underscore characters.
29bd2b8 
This fixes nginx#856 on Github.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@xeioex xeioex

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@xeioex @fabriziofiorucci