Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

408 error after about a minute of working #203

Open
rawvie-tnt opened this issue Aug 23, 2023 · 28 comments
Open

408 error after about a minute of working #203

rawvie-tnt opened this issue Aug 23, 2023 · 28 comments

Comments

@rawvie-tnt
Copy link

I am hosting the local tunnel server in an AWS ec2 instance, after setting up everything when, I try to connect from the client, the tunnel works for about a minute then shows HTTP ERROR 408.

site.conf from nginx/conf.d:

proxy_http_version 1.1;


server {
    listen 0.0.0.0:80;

    location / {
        proxy_pass http://127.0.0.1:3000/;

        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Proto http;
        proxy_set_header X-NginX-Proxy true;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;

        proxy_redirect off;
    }
}

nginx.conf:

user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  off;

    sendfile        on;
    #tcp_nopush     on;


    gzip  on;
    gzip_min_length  1000;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites/*;
}

why does it work just for about a minute. I tried it from both CLI and API.

@j-peace
Copy link

j-peace commented Aug 23, 2023 via email

@rawvie-tnt
Copy link
Author

It’s is your first time hosting at AWS? Did you run local tunnel local and got the same error ?

On Wed, 23 Aug 2023 at 07:22 rawvie-tnt @.> wrote: I am hosting the local tunnel server in an AWS ec2 instance, after setting up everything when, I try to connect from the client, the tunnel works for about a minute then shows HTTP ERROR 408. site.conf from nginx/conf.d: proxy_http_version 1.1; server { listen 0.0.0.0:80; location / { proxy_pass http://127.0.0.1:3000/; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-Proto http; proxy_set_header X-NginX-Proxy true; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_redirect off; } } nginx.conf: user nginx; worker_processes 1; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log off; sendfile on; #tcp_nopush on; gzip on; gzip_min_length 1000; include /etc/nginx/conf.d/.conf; include /etc/nginx/sites/; } why does it work just for about a minute. I tried it from both CLI and API. — Reply to this email directly, view it on GitHub <#203>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATONHPRWSH2C55RIEMEQJ6LXWXKYFANCNFSM6AAAAAA33HT5DA . You are receiving this because you are subscribed to this thread.Message ID: @.**>

yes is its is my first time running local tunnel in AWS, but i have tried with multiple different ec2's with same result, when connecting with the default host server it works as expected, the issue occurs when I try to connect with my server

@j-peace
Copy link

j-peace commented Aug 25, 2023

As it is yout first time running local tunnel server. Can you run it locals first and let me know if you get a similar issue?

Follow this 2 commits to run local: 

Fix esm modules

It will help you to setup the project because there are some differences between node versions.

https://github.com/tiagopazhs/localtunnelServer-raise/commit/425c9733393a13dedd32a363c325a076d4006cbf

Run server in your machine

If you debug your application you will got this problem because localtunnel uses an especific lib that don't handle with local urls.

https://github.com/tiagopazhs/localtunnelServer-raise/commit/518214b9163b4cc80b50f2d9d36686904179f344

@creazy231
Copy link

creazy231 commented Sep 20, 2023

Facing the same issue. After around one minute the assigned domain just returns error 408 on self hosted server. When I use the default hosted one, it works. I'd like to fix this but I don't really know where to look at. Server? Client? But I guess it's a server problem.

@j-peace
Copy link

j-peace commented Sep 20, 2023

@creazy231 can you share error logs ?
Did you already clone localtunnel/client and clone localtunnel/server and running local in your machine ?

@creazy231
Copy link

@tiagopazhs Answering here since I saw j-peace/localTunnel-OpenSource#1 as an alternative.

Yes, I already setup everything and before Node v18+ everything worked fine. Thought it could be the server so I also updated to Node v18 but since then I'm getting a http error code 408 in the browser with Node v16 and Node v18 after exactly one minute (+/- 10 seconds)

I guess you should be able to reproduce using npx lt --host http://lt.boltapi.com --local-host 127.0.0.1 --port 1337 for example. localtunnel/server is hosted on lt.boltapi.com using Node v18 + PM2 with modifications you mentioned here using this commit https://github.com/tiagopazhs/localtunnelServer-raise/commit/425c9733393a13dedd32a363c325a076d4006cbf

also tried to use the docker version on the server - same result.

The only thing works is localtunnel.me - the default host - so I guess there had to be some changes on their side they're using.

@j-peace
Copy link

j-peace commented Sep 21, 2023

@creazy231 so, if you share your repository, I can hel you.
I ran it using node 20.7.0 with out problems.
So, let me know if you follow this step by step.

Fix esm modules

Step 1:
Follow this instructions
https://github.com/tiagopazhs/localtunnelServer-raise/commit/425c9733393a13dedd32a363c325a076d4006cbf
! don't forget to rename the file 'server' to 'server.js'
Step 2:
Set your server port at package.json:
"start": "node ./bin/server.js --port 3043",
Step 3:
npm i
npm run start
Step 4 (optional):

As you debug the application in your machine you can face an issue about the period od grace. For safety reasons you have just 1 second to connect. And if you connect out of this time. You'll be redirected to the landing page: https://localtunnel.github.io/www/
Update the period of grace to degub the application in your client.js file:

image

Now you have a localtunnel server running in your own machine :

image

Run the application that you want to share with the world

I recomend you run a hello world server like this:

const http = require('http');

const server = http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/plain' });
  res.end('Hello, Tobias!\n');
});

const port = process.env.PORT || 7000;

server.listen(port, () => {
  console.log(`Server running at http://localhost:${port}/`);
});

Connect the client to your server

Step 1:
Clone client repository
https://github.com/localtunnel/localtunnel
Step 2:
After install dependencies, run this command:
node bin/lt.js --port 7000 --host http://localhost:3043 --subdomain tobias-28
Explanation: The port 7000 is where the application that you want to share is running; http://localhost:3043 is the server link; subdomain is optional
Step 3 !IMPORTANT!:
The lib tldjs don't recognize local urls. Because of this, you need to edit yout GetClientIdFromHostname function to recognize your subdomain. Its just say yout subdomain. Like this:
    function GetClientIdFromHostname(hostname) {
        hostname = hostname.replace(':3004', '.com.br')
        return myTldjs.getSubdomain(hostname);
    }
Observation: Just in development. In production, you don't need to do it.

@creazy231
Copy link

@tiagopazhs thanks for this awesome setup guide ❤️
I'll follow it with exactly the same settings using npm instead of yarn and also using Node v20 instead of Node 18 and will come back as soon as I'm done. Cross my fingers to be able to make it work

@creazy231
Copy link

creazy231 commented Sep 21, 2023

@tiagopazhs after exact same setup it works for some time and page reloads.. until this happens upon reload:

Client:

  ...
  localtunnel:client establishing tunnel http://localhost:4000 <> lt.boltapi.com:36461 +0ms
  localtunnel:client tunnel open [total: 10] +31ms
  localtunnel:client connecting locally to http://localhost:4000 +64ms
  localtunnel:client connected locally +5ms
  localtunnel:client local connection closed [false] +44s
  localtunnel:client local connection closed [false] +0ms
  localtunnel:client local connection closed [false] +0ms
  localtunnel:client local connection closed [false] +30s
MacBook-Pro-von-Tobias-2:test thiele$

after local connection closed on client side, Server logs:

-- Start getting HTTP ERROR 408 in Browser
  lt:Client[tobias-28] > / +2m
  lt:TunnelAgent[tobias-28] create connection +2m
  lt:TunnelAgent[tobias-28] socket given +1ms
  lt:Client[tobias-28] < / +4ms
  lt:TunnelAgent[tobias-28] closed socket (error: false) +4ms
  lt:TunnelAgent[tobias-28] connected sockets: 3 +0ms
-- Refresh page -> getting HTTP ERROR 408 in Browser
  lt:Client[tobias-28] > / +204ms
  lt:TunnelAgent[tobias-28] create connection +202ms
  lt:TunnelAgent[tobias-28] socket given +1ms
  lt:Client[tobias-28] < / +3ms
  lt:TunnelAgent[tobias-28] closed socket (error: false) +5ms
  lt:TunnelAgent[tobias-28] connected sockets: 2 +0ms
-- Refresh page -> getting HTTP ERROR 408 in Browser
  lt:Client[tobias-28] > / +104ms
  lt:TunnelAgent[tobias-28] create connection +100ms
  lt:TunnelAgent[tobias-28] socket given +1ms
  lt:Client[tobias-28] < / +1ms
  lt:TunnelAgent[tobias-28] closed socket (error: false) +1ms
  lt:TunnelAgent[tobias-28] connected sockets: 1 +0ms
-- Refresh page -> getting HTTP ERROR 408 in Browser
  lt:Client[tobias-28] > / +129ms
  lt:TunnelAgent[tobias-28] create connection +129ms
  lt:TunnelAgent[tobias-28] socket given +0ms
  lt:Client[tobias-28] < / +2ms
-- Refresh page -> getting HTTP ERROR 408 in Browser
  lt:TunnelAgent[tobias-28] closed socket (error: false) +3ms
  lt:TunnelAgent[tobias-28] connected sockets: 0 +0ms
  lt:TunnelAgent[tobias-28] all sockets disconnected +0ms
  lt:Client[tobias-28] client offline tobias-28 +2ms
  lt:Client[tobias-28] > / +113ms
  lt:TunnelAgent[tobias-28] create connection +114ms
  lt:TunnelAgent[tobias-28] waiting connected: 0 +0ms
  lt:TunnelAgent[tobias-28] waiting available: 0 +1ms
  lt:ClientManager removing client: tobias-28 +2m
  lt:TunnelAgent[tobias-28] closed tcp socket +887ms
  lt:TunnelAgent[tobias-28] closed tcp socket +0ms
-- Refresh page -> getting HTTP ERROR 404 in Browser

Maybe you can reproduce this by just idling in the browser for some time without refreshing etc. and then refresh page after around one or two minutes.. at least using lt.boltapi.com as host you should face the same problem.
It's weird because when I constantly refresh the page for like once every second, the connection keeps stable.

Using Ubuntu 22.04, nginx with LetsEncrypt wildcard cert, Cloudflare for DNS without Proxy.

I just don't know what to do anymore at this point to make it work.. even changing back to Node v16 on server and client doesn't change anything.


Edit:
after all that struggle, I finally cloned and setup the whole project locally as you described - same result. After idling for some time - HTTP ERROR 408
ever additional refresh of the page results in an additional socket connection. When it reaches 10 (default max) it drops the connection and closes the tunnel.


Edit:
Running the whole project on localhost using Node v16 - it works

Some Testing:

Process Node Version Server Node Version Client Node Version Works Host
16 16 16 localhost
18 18 18 localhost
20 20 20 localhost
16 20 16 localhost
18 20 18 localhost
18 20 16 localhost
16 20 18 localhost
16 20 20 localhost
18 20 20 localhost
16 20 16 127.0.0.1
18 20 18 127.0.0.1
20 20 20 127.0.0.1

so it points out, that the process has to run on Node v16 to make it work. The problem seems to be not related to localtunnel itself but changes made from Node v16 to Node v18. Maybe nodejs/node#40537 is related to this? But even changing everything to 127.0.0.1 results in the same error

@j-peace
Copy link

j-peace commented Sep 21, 2023

This is a very good analysis! So I didn't know that.

So, I don't know what is releated to this problem.

Maybe to update some dependencies. In the past I used Express instead Koa and it works well but I can't say that this is the problem.

So, using this bellow scenario everything works fine? Can you see your local application running in other machine ?
16 | 20 | 16 | ✅ | 127.0.0.1

In this moment our main goal is to run with this scenario, rigth ? I will try to solve that.

18 18 18 127.0.0.1

@creazy231
Copy link

Exactly, 18 | 18/20 | 18 | 127.0.0.1/locahost is what we wanna get up and running. In fact - every time a user uses Node v18 or higher, localtunnel stops working after about 1 minute. I guess the socket connection just drops or something.

I already tried to update all dependencies to its latest version - with the same result - getting HTTP ERROR 408 after some time and dropping connection when max socket connection is reached.

Also thought about using socket.io for connection but it's not that easy to make it work. Reason therefore is, that this repository uses socket.io (https://github.com/BenMcH/node-grok) and it simply works! But without a subdomain like localtunnel :(

Will try to replace koa with express like you mentioned, maybe this gives me some other responses or ideas.

Were you able to reproduce the error I mentioned with the 408 http error?

@creazy231
Copy link

creazy231 commented Sep 21, 2023

@tiagopazhs quick update after debugging the hell out of locatunnel and your repository of localTunnel-OpenSource:

I debugged the problem down to the net.connect since I figured out, that the local connection just drops with http error 408. The local connection which maps the local application to the client service.

As I thought, the reason is this: nodejs/node#43776

I used your localTunnel-OpenSource to debug since socket.io and express just gives me more details when running it with DEBUG=*.

Now I'll try to implement the changes I've made into localtunnel.. let's see

@creazy231
Copy link

creazy231 commented Sep 22, 2023

After non stop debugging and testing the whole day and night, I (hopefully) found a solution. At least it works locally and has nothing todo with localhost / 127.0.0.1

This simple line in server/TunnelAgent.js makes the magic happen:

     // new socket connection from client for tunneling requests to client
     _onConnection(socket) {
         // no more socket connections allowed
         if (this.connectedSockets >= this.maxTcpSockets) {
             this.debug('no more sockets allowed');
             socket.destroy();
             return false;
         }

+        socket.on("data", () => { /* silence is golden */ });

         socket.once('close', (hadError) => {
             this.debug('closed socket (error: %s)', hadError);
             this.connectedSockets -= 1;

It ensures for some reason (didn't look deeper into it for now) that the connection is getting reestablished after getting dropped. Without this line an important other event (don't know which one for now) is not getting fired which results in broken sockets (9+1 as defined by default).

Will try to implement these changes on my live server as soon as I got some sleep and hopefully it works there too.

Example:
Without this "fix":
CleanShot 2023-09-22 at 04 29 11

With this "fix":
CleanShot 2023-09-22 at 04 35 52


Tested on my VPS with success 🎉
Works with all versions of Node (16,18,20)

@j-peace
Copy link

j-peace commented Sep 22, 2023

Sorry for my latest answer. Really ? Congratiulations 🎉.
You made an incredible work and now we have a localtunnel able to revive.

@j-peace
Copy link

j-peace commented Sep 22, 2023

@creazy231 when I use nove v18 the client is don't found the server.
Is there another thing to do ?
ps: I am already using the magic line: socket.on("data", () => { /* silence is golden */ });

@creazy231
Copy link

@tiagopazhs connecting with node bin/lt.js --port 4000 --host http://localhost:3043 on local and using lt --host http://lt.boltapi.com --local-host 127.0.0.1 --port 3000 --subdomain my-sub-domain connecting to my VPS LocalTunnel. Didn't change anything else on client side.

I guess the important part is the --local-host 127.0.0.1 using Node v18+ because of the default dns change since Node v17. Also make sure your local apps also running on 127.0.0.1

@j-peace
Copy link

j-peace commented Oct 4, 2023

Sorry for the late answer. For me it is not working yet:

My server is running on 18 version:

image

But when I use version 18 in my client. It can't find anything:

image

@rawvie-tnt
Copy link
Author

I used this fork and everything worked.

@j-peace
Copy link

j-peace commented Oct 11, 2023

@rawvie-tnt thanks for this awesome way to use localtunnel. The fork has interesting features and ways to make localtunnel better.

So, about the previous error that I mentioned: #203 (comment)
I am trying to get the answer for why it is happening. Do you know how the author of the fork did to solve this bug in node 18 ?

@yashwanth2714
Copy link

yashwanth2714 commented Dec 19, 2023

@tiagopazhs I followed your instructions but am facing the below issue. The URL is not coming properly. Could you please help me? I have not been able to figure out the problem.

I am using this fork mentioned by @rawvie-tnt

#203 (comment)

your url is: http://tobias-28.undefined

@SlitheryMender
Copy link

@tiagopazhs I followed your instructions but am facing the below issue. The URL is not coming properly. Could you please help me? I have not been able to figure out the problem.

I am using this fork mentioned by @rawvie-tnt

#203 (comment)

your url is: http://tobias-28.undefined

Try starting localtunnel server using --domain <your_domain_name>. Then when you connect the client with subdomain, the client url is shown correctly. It works for me for this fork of localtunnel server.

I was able to connect node v18 client running nodev18 process to node v18 server properly. Hope this helps !

@yashwanth2714
Copy link

@SlitheryMender Thanks for your reply! May I know where did you deploy the localtunnel-server to get the domain? I am trying to deploy it to https://onrender.com/ but it is not working.

@yashwanth2714
Copy link

@rawvie-tnt I am also using the EC2 instance to run the server. Only HTTP URLs are functional. The issue I am encountering is the inability to add HTTPS support. I have attempted to point the DNS records to the ALB domain by attaching an HTTPS listener with the ACM certificate, but it is not working. Did you add HTTPS support?

@creazy231
Copy link

guess your server setup is kinda wrong. I'm running a wildcard cert using certbot and everything works

@yashwanth2714
Copy link

@creazy231 Yes, but I'm not sure where the problem lies. Are you also utilizing the EC2, ALB, and Route 53 services to host the server code?

@creazy231
Copy link

nope.. running on a 3€/mo hetzner cloud server because I don't like the complexity of AWS for simple braindead projects

@mister-teddy
Copy link

It looks like the issue has been around for a year. What I can tell is that this is not a server (only) problem: on the same machine, using a Node 16 client (via Electron) created a stable tunnel, but using a Node 18 client (via Visual Studio Code Extension Host) create a tunnel that broken at some point, with 408 error code.

I will try this fork tomorrow and let you guys know if it works. And I wonder what exactly is modified in the fork? I took a quick glance at the diff, and the only thing I saw was code formatting...

@adearriba
Copy link

Maybe this is the reason.
nodejs/node#35825

Instead of an empty “data” event handled, you should use a socket.resume().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants