[BUG] Jail stops network after a while

[marschro]

Description

Background

• I use Bastille since a few years in this setup without any issue and I like it way tooo much
• zfs enabled
• Using the loopback network approach on Vultr.com (a smaller server, so I don't have all these ipv6 addresses)
• I run a jail called portal (elixir based) that serves a web application on port 443
• haproxy upfront which works fine and healthy (very likely not an upstream issue)
• When the jail is properly working, curl-ing the jail from the host with its ip and port responds with a valid html response
• When the jail is properly working, curl-ing the app from within the jail via loopback interface also responds with a html response

What changed

• I set up a fresh new host system with freebsd 14.2-RELEASE
• I upgraded from Bastille 0.10.x to 0.12.x
• The application did not change and is healthy

The problematic behavior that now came up after the upgrades

• The mentioned jail can be created and started perfectly fine.
• After about one to two hours, the jail can not be reached anymore via network
• curl-ing the app from host or from inside the jail closes with no response in 0ms
• Thus, the app running in the jail can not be reached anymore
• The jail itself is still up, and I can console into it
• There is nothing in the jails log (except of those standard messages)
• ifconfig shows that the correct IP is assigned
• top and htop all show healthy informations
• So basically the jail seems to loose its network reachability - but always after ~1 to 2 hours

[MANDATORY] Bastille and FreeBSD version (paste bastille -v && freebsd-version -kru output)

> sudo bastille -v && sudo freebsd-version -kru -j portal
0.12.20241124
14.2-RELEASE
14.2-RELEASE
14.2-RELEASE
14.2-RELEASE

[MANDATORY] How did you install bastille? (port/pkg/git)

• pkg

[optional] Steps to reproduce?

• This is currently always the case.
• I cannot find a cause when the jail stops being reachable
• It must be something time based... ?
• Also interesting: I run multiple jails on that host. Only this jail has this issue. Others, that run other applications are working perfectly fine. This lead me to the conclusion that it has something to do with the application - but I tracked that down for the last four days and there is no issue with the app.

[optional] Expected behavior

• I expect the jail to never lose network connection and it is always reachable.

[optional] Screenshots

• can be delivered if needed

Additional informations

• What hints into an application related thing is, that I have two jail that are problematic in that way. They both serve elixir phoenix applications. Other jails that serve nginx or node.js apps do not have that problem
• Classic jail
• Thin jail

[tschettervictor]

Thick or thin jails?
VNET or classic? Assuming classic because of the "loopback" mentioned above.

[marschro]

Thick or thin jails? VNET or classic? Assuming classic because of the "loopback" mentioned above.

• classic
• thin

But I have to admit. Maybe totally not a jail or bastille issue. Trying to find the cause for days and posted this here with the hope someone has experienced this or might have an idea.

[bmac2]

since it is ONLY the Elixer Phoenix app I would suspect it. Since the other jails are fine, that points to your setup being right.

[marschro]

since it is ONLY the Elixer Phoenix app I would suspect it. Since the other jails are fine, that points to your setup being right.

Yes, I agree.
Meanwhile I was able to narrow down the cause to the server of the app. OS level works fine. I filed a bug in the Bandit Elixir server. In combination with Haproxy it seems to cause the trouble.
I can pretty much exclude bastille as an issue and will now close that.