Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FTL crash after clean install #2250

Open
DaitiDay opened this issue Feb 22, 2025 · 25 comments
Open

FTL crash after clean install #2250

DaitiDay opened this issue Feb 22, 2025 · 25 comments

Comments

@DaitiDay
Copy link

Versions

  • Pi-hole: v6.0.3
  • AdminLTE: (don't know where to find this)
  • FTL: v6.0.2

Platform

  • OS and version: Ubuntu 20.04 LTS
  • Platform: Raspberry Pi

Expected behavior

Can add/delete CNAME.

Actual behavior / bug

After the clean installation of the latest version of pihole, whenever I try to add/delete a CNAME, the FTL crashes.

Steps to reproduce

Steps to reproduce the behavior:

  1. Go to 'Local DNS'
  2. Add/delete a CNAME
  3. See error

Debug Token

Additional context

During an update I got a DNS resolution problem related to a setting I did not change after a router reset. I quitted the update and got prompted to uninstall pihole and remove al unnecessary packages. I did that, and then reinstalled pihole following method 2 on the official site. Installation was fine, but I couldn't access the web interface, launching pihole status reported [✗] DNS service is NOT running, so I rebooted and everything was working.
As soon as I added the first CNAME (which worked), every subsequent try I encoutered an error, and refreshing the page resulted in the same inaccessible web interface as before. Same output from pihole status. Rebooting fixes the problem, but adding/deleting a CNAME results in FTL crash (checked in FTL.log).

@yubiuser
Copy link
Member

Can you please post some lines from above the !!!!!!!!! header from /var/log/pihole/FTL.log?

@DaitiDay
Copy link
Author

Sorry got an emergency.
I don't know what you exactly wanted, so this is the entire FTL.log

2025-02-22 12:37:08.440 CET [10019M] INFO: Asked to terminate by "/sbin/init fixrtc splash" (PID 1, user root UID 0)
2025-02-22 12:37:08.486 CET [10019/T10032] INFO: Terminating database thread
2025-02-22 12:37:08.502 CET [10019/T10035] INFO: Terminating timer thread
2025-02-22 12:37:08.703 CET [10019M] INFO: Finished final database update
2025-02-22 12:37:08.703 CET [10019M] INFO: Waiting for threads to join
2025-02-22 12:37:08.704 CET [10019M] INFO: Thread housekeeper (1) is idle, terminating it.
2025-02-22 12:37:08.704 CET [10019M] INFO: Thread dns-client (2) is idle, terminating it.
2025-02-22 12:37:08.704 CET [10019M] INFO: Thread ntp-client (4) is idle, terminating it.
2025-02-22 12:37:08.704 CET [10019M] INFO: All threads joined
2025-02-22 12:37:08.705 CET [10019M] INFO: PID file emptied
2025-02-22 12:37:08.750 CET [10019M] INFO: ########## FTL terminated after 39m 28s  (code 0)! ##########
2025-02-22 12:37:17.011 CET [840M] INFO: ########## FTL started on netpi! ##########
2025-02-22 12:37:17.021 CET [840M] INFO: FTL branch: master
2025-02-22 12:37:17.021 CET [840M] INFO: FTL version: v6.0.2
2025-02-22 12:37:17.021 CET [840M] INFO: FTL commit: ac500d5f
2025-02-22 12:37:17.022 CET [840M] INFO: FTL date: 2025-02-21 21:48:20 +0000
2025-02-22 12:37:17.022 CET [840M] INFO: FTL user: pihole
2025-02-22 12:37:17.022 CET [840M] INFO: Compiled for linux/arm64/v8 (compiled on CI) using cc (Alpine 14.2.0) 14.2.0
2025-02-22 12:37:17.028 CET [840M] INFO: Wrote config file:
2025-02-22 12:37:17.028 CET [840M] INFO:  - 152 total entries
2025-02-22 12:37:17.028 CET [840M] INFO:  - 148 entries are default
2025-02-22 12:37:17.028 CET [840M] INFO:  - 4 entries are modified
2025-02-22 12:37:17.028 CET [840M] INFO:  - 0 entries are forced through environment
2025-02-22 12:37:17.098 CET [840M] INFO: Parsed config file /etc/pihole/pihole.toml successfully
2025-02-22 12:37:17.098 CET [840M] INFO: PID file does not exist or not readable
2025-02-22 12:37:17.098 CET [840M] INFO: No other running FTL process found.
2025-02-22 12:37:17.104 CET [840M] INFO: PID of FTL process: 840
2025-02-22 12:37:17.199 CET [840M] INFO: listening on 0.0.0.0 port 53
2025-02-22 12:37:17.199 CET [840M] INFO: listening on :: port 53
2025-02-22 12:37:17.201 CET [840M] INFO: PID of FTL process: 840
2025-02-22 12:37:17.428 CET [840M] INFO: Database version is 21
2025-02-22 12:37:17.429 CET [840M] INFO: Database successfully initialized
2025-02-22 12:37:17.565 CET [840M] INFO: Imported 4084 queries from the on-disk database (it has 4084 rows)
2025-02-22 12:37:17.565 CET [840M] INFO: Parsing queries in database
2025-02-22 12:37:17.609 CET [840M] INFO: Imported 4084 queries from the long-term database
2025-02-22 12:37:17.609 CET [840M] INFO:  -> Total DNS queries: 4084
2025-02-22 12:37:17.610 CET [840M] INFO:  -> Cached DNS queries: 3533
2025-02-22 12:37:17.610 CET [840M] INFO:  -> Forwarded DNS queries: 524
2025-02-22 12:37:17.610 CET [840M] INFO:  -> Blocked DNS queries: 13
2025-02-22 12:37:17.610 CET [840M] INFO:  -> Unknown DNS queries: 0
2025-02-22 12:37:17.610 CET [840M] INFO:  -> Unique domains: 267
2025-02-22 12:37:17.610 CET [840M] INFO:  -> Unique clients: 13
2025-02-22 12:37:17.610 CET [840M] INFO:  -> DNS cache records: 2
2025-02-22 12:37:17.610 CET [840M] INFO:  -> Known forward destinations: 2
2025-02-22 12:37:18.975 CET [840M] INFO: FTL is running as user pihole (UID 997)
2025-02-22 12:37:18.979 CET [840M] INFO: Reading certificate from /etc/pihole/tls.pem ...
2025-02-22 12:37:18.987 CET [840M] INFO: Using SSL/TLS certificate file /etc/pihole/tls.pem
2025-02-22 12:37:18.988 CET [840M] INFO: Web server ports:
2025-02-22 12:37:18.988 CET [840M] INFO:   - 80 (HTTP, IPv4, optional)
2025-02-22 12:37:18.988 CET [840M] INFO:   - 443 (HTTPS, IPv4, optional)
2025-02-22 12:37:18.988 CET [840M] INFO:   - 80 (HTTP, IPv6, optional)
2025-02-22 12:37:18.988 CET [840M] INFO:   - 443 (HTTPS, IPv6, optional)
2025-02-22 12:37:19.017 CET [840M] INFO: Restored 0 API sessions from the database
2025-02-22 12:37:19.054 CET [840M] INFO: Blocking status is enabled
2025-02-22 12:37:19.112 CET [840/T880] INFO: Compiled 0 allow and 0 deny regex for 13 clients in 0.3 msec
2025-02-22 12:37:51.280 CET [840/T879] ERROR: Failed to receive data from NTP server pool.ntp.org (95.110.254.234): Timeout
2025-02-22 12:37:56.400 CET [840/T879] ERROR: Failed to receive data from NTP server pool.ntp.org (95.110.254.234): Timeout
2025-02-22 12:37:56.933 CET [840/T879] INFO: Received 5/8 valid NTP replies from pool.ntp.org
2025-02-22 12:37:56.933 CET [840/T879] INFO: Time offset: 2.444868e+04 ms (excluded 3 outliers)
2025-02-22 12:37:56.933 CET [840/T879] INFO: Round-trip delay: 3.279791e+01 ms (excluded 3 outliers)
2025-02-22 12:38:21.382 CET [840/T879] INFO: Local time is too inaccurate, retrying in 600 seconds before launching NTP server
2025-02-22 12:41:05.941 CET [840/T12706] INFO: Local URI: "/admin/"
2025-02-22 12:41:05.951 CET [840/T12707] INFO: Local URI: "/admin/login"
2025-02-22 12:41:06.057 CET [840/T12706] INFO: Local URI: "/admin/vendor/nprogress/nprogress.min.js"
2025-02-22 12:41:06.060 CET [840/T12716] INFO: Local URI: "/admin/vendor/fonts/SourceSansPro/SourceSansPro.css"
2025-02-22 12:41:06.060 CET [840/T12707] INFO: Local URI: "/admin/vendor/nprogress/nprogress.min.css"
2025-02-22 12:41:06.061 CET [840/T12706] INFO: Local URI: "/admin/vendor/bootstrap/css/bootstrap.min.css"
2025-02-22 12:41:06.063 CET [840/T12716] INFO: Local URI: "/admin/vendor/animate/animate.min.css"
2025-02-22 12:41:06.064 CET [840/T12706] INFO: Local URI: "/admin/vendor/bstreeview/bstreeview.min.css"
2025-02-22 12:41:06.066 CET [840/T12707] INFO: Local URI: "/admin/vendor/select2/select2.min.css"
2025-02-22 12:41:06.068 CET [840/T12716] INFO: Local URI: "/admin/vendor/adminLTE/AdminLTE.min.css"
2025-02-22 12:41:06.070 CET [840/T12716] INFO: Local URI: "/admin/style/pi-hole.css"
2025-02-22 12:41:06.071 CET [840/T12707] INFO: Local URI: "/admin/style/themes/default-auto.css"
2025-02-22 12:41:06.071 CET [840/T12707] INFO: Local URI: "/admin/vendor/jquery/jquery.min.js"
2025-02-22 12:41:06.072 CET [840/T12716] INFO: Local URI: "/admin/vendor/bootstrap/js/bootstrap.min.js"
2025-02-22 12:41:06.074 CET [840/T12707] INFO: Local URI: "/admin/vendor/adminLTE/adminlte.min.js"
2025-02-22 12:41:06.081 CET [840/T12716] INFO: Local URI: "/admin/vendor/bootstrap-notify/bootstrap-notify.min.js"
2025-02-22 12:41:06.083 CET [840/T12706] INFO: Local URI: "/admin/vendor/font-awesome/all.min.js"
2025-02-22 12:41:06.084 CET [840/T12706] INFO: Local URI: "/admin/scripts/js/utils.js"
2025-02-22 12:41:06.089 CET [840/T12716] INFO: Local URI: "/admin/img/logo.svg"
2025-02-22 12:41:06.090 CET [840/T12706] INFO: Local URI: "/admin/vendor/waitMe/waitMe.min.js"
2025-02-22 12:41:06.099 CET [840/T12706] INFO: Local URI: "/admin/scripts/js/login.js"
2025-02-22 12:41:06.099 CET [840/T12707] INFO: Local URI: "/admin/vendor/waitMe/waitMe.min.css"
2025-02-22 12:41:06.099 CET [840/T12707] INFO: Local URI: "/admin/scripts/js/footer.js"
2025-02-22 12:41:06.137 CET [840/T12707] INFO: Local URI: "/admin/style/themes/default-light.css"
2025-02-22 12:41:06.138 CET [840/T12706] INFO: Local URI: "/admin/style/themes/default-dark.css"
2025-02-22 12:41:06.299 CET [840/T12716] INFO: Local URI: "/admin/img/boxed-bg-dark.png"
2025-02-22 12:41:06.307 CET [840/T12707] INFO: Local URI: "/admin/vendor/fonts/SourceSansPro/source-sans-pro-v13-latin-300.woff2"
2025-02-22 12:41:06.308 CET [840/T12706] INFO: Local URI: "/admin/vendor/fonts/SourceSansPro/source-sans-pro-v13-latin-700.woff2"
2025-02-22 12:41:06.308 CET [840/T12716] INFO: Local URI: "/admin/vendor/fonts/SourceSansPro/source-sans-pro-v13-latin-regular.woff2"
2025-02-22 12:41:06.309 CET [840/T12707] INFO: Local URI: "/admin/vendor/fonts/SourceSansPro/source-sans-pro-v13-latin-italic.woff2"
2025-02-22 12:41:06.311 CET [840/T12706] INFO: Local URI: "/admin/vendor/fonts/SourceSansPro/source-sans-pro-v13-latin-600italic.woff2"
2025-02-22 12:41:06.373 CET [840/T12706] INFO: Local URI: "/admin/vendor/icheck/icheck-bootstrap.min.css"
2025-02-22 12:41:11.036 CET [840/T12707] INFO: Local URI: "/admin/"
2025-02-22 12:41:11.162 CET [840/T12706] INFO: Local URI: "/admin/vendor/datatables/datatables.min.css"
2025-02-22 12:41:11.163 CET [840/T12716] INFO: Local URI: "/admin/vendor/datatables-buttons/datatables.buttons.min.css"
2025-02-22 12:41:11.164 CET [840/T12707] INFO: Local URI: "/admin/vendor/datatables-select/datatables.select.min.css"
2025-02-22 12:41:11.238 CET [840/T12706] INFO: Local URI: "/admin/vendor/daterangepicker/daterangepicker.min.css"
2025-02-22 12:41:11.238 CET [840/T12707] INFO: Local URI: "/admin/vendor/bootstrap-toggle/bootstrap-toggle.min.css"
2025-02-22 12:41:11.238 CET [840/T12716] INFO: Local URI: "/admin/scripts/js/logout.js"
2025-02-22 12:41:11.239 CET [840/T12706] INFO: Local URI: "/admin/vendor/select2/select2.min.js"
2025-02-22 12:41:11.240 CET [840/T12716] INFO: Local URI: "/admin/vendor/datatables-select/datatables.select.min.js"
2025-02-22 12:41:11.240 CET [840/T12707] INFO: Local URI: "/admin/vendor/datatables/datatables.min.js"
2025-02-22 12:41:11.240 CET [840/T12706] INFO: Local URI: "/admin/vendor/datatables-buttons/datatables.buttons.min.js"
2025-02-22 12:41:11.241 CET [840/T12716] INFO: Local URI: "/admin/vendor/chartjs/chart.umd.js"
2025-02-22 12:41:11.241 CET [840/T12707] INFO: Local URI: "/admin/vendor/chartjs-plugin-deferred/chartjs-plugin-deferred.min.js"
2025-02-22 12:41:11.241 CET [840/T12706] INFO: Local URI: "/admin/vendor/moment/moment.min.js"
2025-02-22 12:41:11.242 CET [840/T12706] INFO: Local URI: "/admin/vendor/chartjs-adapter-moment/chartjs-adapter-moment.js"
2025-02-22 12:41:11.242 CET [840/T12707] INFO: Local URI: "/admin/vendor/chartjs-plugin-zoom/chartjs-plugin-zoom.min.js"
2025-02-22 12:41:11.242 CET [840/T12716] INFO: Local URI: "/admin/vendor/hammer/hammer.min.js"
2025-02-22 12:41:11.243 CET [840/T12706] INFO: Local URI: "/admin/vendor/bstreeview/bstreeview.min.js"
2025-02-22 12:41:11.243 CET [840/T12707] INFO: Local URI: "/admin/scripts/js/charts.js"
2025-02-22 12:41:11.243 CET [840/T12716] INFO: Local URI: "/admin/scripts/js/index.js"
2025-02-22 12:41:11.413 CET [840/T12706] INFO: Local URI: "/admin/img/pihole_icon.svg"
2025-02-22 12:41:11.413 CET [840/T12707] INFO: Local URI: "/admin/vendor/fonts/SourceSansPro/source-sans-pro-v13-latin-600.woff2"
2025-02-22 12:41:57.368 CET [840/T12716] INFO: Local URI: "/admin/settings/system"
2025-02-22 12:41:57.487 CET [840/T12706] INFO: Local URI: "/admin/scripts/js/settings-system.js"
2025-02-22 12:41:57.487 CET [840/T12707] INFO: Local URI: "/admin/vendor/bootstrap-toggle/bootstrap-toggle.min.js"
2025-02-22 12:41:57.487 CET [840/T12707] INFO: Local URI: "/admin/vendor/jquery-confirm/jquery.confirm.min.js"
2025-02-22 12:41:57.501 CET [840/T12707] INFO: Local URI: "/admin/scripts/js/settings.js"
2025-02-22 12:42:08.750 CET [840/T12716] INFO: Local URI: "/admin/taillog"
2025-02-22 12:42:08.844 CET [840/T12707] INFO: Local URI: "/admin/scripts/js/taillog.js"
2025-02-22 12:42:19.256 CET [840/T12716] INFO: Local URI: "/admin/"
2025-02-22 12:42:37.063 CET [840/T12707] INFO: Local URI: "/admin/settings/dns"
2025-02-22 12:42:37.147 CET [840/T12716] INFO: Local URI: "/admin/scripts/js/settings-dns.js"
2025-02-22 12:42:44.750 CET [840/T12706] INFO: Local URI: "/admin/settings/dhcp"
2025-02-22 12:42:44.856 CET [840/T12716] INFO: Local URI: "/admin/scripts/js/settings-dhcp.js"
2025-02-22 12:42:44.861 CET [840/T12707] INFO: Local URI: "/admin/scripts/js/ip-address-sorting.js"
2025-02-22 12:42:45.198 CET [840/T12706] INFO: Local URI: "/admin/vendor/bootstrap/fonts/glyphicons-halflings-regular.woff2"
2025-02-22 12:42:52.236 CET [840/T12706] INFO: Local URI: "/admin/settings/dnsrecords"
2025-02-22 12:42:52.341 CET [840/T12716] INFO: Local URI: "/admin/scripts/js/settings-dns-records.js"
2025-02-22 12:42:57.383 CET [840/T12706] INFO: Local URI: "/admin/search"
2025-02-22 12:42:57.503 CET [840/T12707] INFO: Local URI: "/admin/scripts/js/search.js"
2025-02-22 12:42:58.843 CET [840/T12707] INFO: Local URI: "/admin/interfaces"
2025-02-22 12:42:58.933 CET [840/T12706] INFO: Local URI: "/admin/scripts/js/interfaces.js"
2025-02-22 12:43:00.808 CET [840/T12716] INFO: Local URI: "/admin/network"
2025-02-22 12:43:00.904 CET [840/T12706] INFO: Local URI: "/admin/scripts/js/network.js"
2025-02-22 12:43:07.734 CET [840/T12707] INFO: Local URI: "/admin/"
2025-02-22 12:48:01.301 CET [840/T879] INFO: Received 8/8 valid NTP replies from pool.ntp.org
2025-02-22 12:48:01.301 CET [840/T879] INFO: Time offset: -2.665907e+00 ms (excluded 0 outliers)
2025-02-22 12:48:01.301 CET [840/T879] INFO: Round-trip delay: 3.273147e+01 ms (excluded 0 outliers)
2025-02-22 12:48:01.302 CET [840/T44003] INFO: NTP server listening on 0.0.0.0:123 (IPv4)
2025-02-22 12:48:01.302 CET [840/T44004] INFO: NTP server listening on :::123 (IPv6)
2025-02-22 12:48:49.386 CET [840/T12716] INFO: Local URI: "/admin/taillog"
2025-02-22 12:53:14.593 CET [840/T12716] INFO: Local URI: "/admin/groups/clients"
2025-02-22 12:53:14.834 CET [840/T12707] INFO: Local URI: "/admin/scripts/js/groups-clients.js"
2025-02-22 12:53:14.834 CET [840/T12706] INFO: Local URI: "/admin/vendor/bootstrap-select/bootstrap-select.min.css"
2025-02-22 12:53:14.844 CET [840/T12716] INFO: Local URI: "/admin/vendor/bootstrap-select/bootstrap-select.min.js"
2025-02-22 12:53:14.844 CET [840/T12706] INFO: Local URI: "/admin/scripts/js/groups-common.js"
2025-02-22 12:53:19.157 CET [840/T12716] INFO: Local URI: "/admin/groups/lists"
2025-02-22 12:53:19.279 CET [840/T12706] INFO: Local URI: "/admin/scripts/js/groups-lists.js"
2025-02-22 12:53:28.036 CET [840/T12707] INFO: Local URI: "/admin/settings/api"
2025-02-22 12:53:28.146 CET [840/T12706] INFO: Local URI: "/admin/vendor/qrious/qrious.min.js"
2025-02-22 12:53:28.152 CET [840/T12716] INFO: Local URI: "/admin/scripts/js/settings-api.js"
2025-02-22 12:53:30.811 CET [840/T12706] INFO: Local URI: "/admin/settings/privacy"
2025-02-22 12:53:30.948 CET [840/T12707] INFO: Local URI: "/admin/scripts/js/settings-privacy.js"
2025-02-22 12:53:32.622 CET [840/T12706] INFO: Local URI: "/admin/settings/teleporter"
2025-02-22 12:53:32.734 CET [840/T12716] INFO: Local URI: "/admin/scripts/js/settings-teleporter.js"
2025-02-22 12:53:36.055 CET [840/T12716] INFO: Local URI: "/admin/settings/dnsrecords"
2025-02-22 12:53:42.292 CET [840/T12707] INFO: Local URI: "/admin/settings/all"
2025-02-22 12:53:42.417 CET [840/T12706] INFO: Local URI: "/admin/scripts/js/settings-advanced.js"
2025-02-22 12:55:12.436 CET [840/T12716] INFO: Local URI: "/admin/settings/dnsrecords"
2025-02-22 12:55:16.631 CET [840/T12706] INFO: Wrote config file:
2025-02-22 12:55:16.631 CET [840/T12706] INFO:  - 152 total entries
2025-02-22 12:55:16.631 CET [840/T12706] INFO:  - 148 entries are default
2025-02-22 12:55:16.631 CET [840/T12706] INFO:  - 4 entries are modified
2025-02-22 12:55:16.631 CET [840/T12706] INFO:  - 0 entries are forced through environment
2025-02-22 12:55:16.649 CET [840/T12706] INFO: Config file written to /etc/pihole/pihole.toml
2025-02-22 12:55:16.651 CET [840/T12706] INFO: Restarting FTL: dnsmasq config changed
2025-02-22 12:55:16.652 CET [840M] INFO: Asked to terminate by "/usr/bin/pihole-FTL -f" (PID 840, user pihole UID 997)
2025-02-22 12:55:16.667 CET [840/T883] INFO: Terminating timer thread
2025-02-22 12:55:16.667 CET [840/T882] INFO: Terminating resolver thread
2025-02-22 12:55:16.684 CET [840/T880] INFO: Terminating database thread
2025-02-22 12:55:16.846 CET [840/T881] INFO: Terminating GC thread
2025-02-22 12:55:16.921 CET [840M] INFO: Finished final database update
2025-02-22 12:55:16.921 CET [840M] INFO: Waiting for threads to join
2025-02-22 12:55:16.921 CET [840M] INFO: Thread ntp-client (4) is idle, terminating it.
2025-02-22 12:55:16.921 CET [840M] INFO: All threads joined
2025-02-22 12:55:16.923 CET [840M] INFO: PID file emptied
2025-02-22 12:55:16.926 CET [840M] INFO: Stored 1 API session in the database
2025-02-22 12:55:17.673 CET [840M] INFO: ########## FTL terminated after 18m  (internal restart)! ##########
2025-02-22 12:55:17.687 CET [840M] INFO: ########## FTL started on netpi! ##########
2025-02-22 12:55:17.687 CET [840M] INFO: FTL branch: master
2025-02-22 12:55:17.687 CET [840M] INFO: FTL version: v6.0.2
2025-02-22 12:55:17.688 CET [840M] INFO: FTL commit: ac500d5f
2025-02-22 12:55:17.688 CET [840M] INFO: FTL date: 2025-02-21 21:48:20 +0000
2025-02-22 12:55:17.688 CET [840M] INFO: FTL user: pihole
2025-02-22 12:55:17.688 CET [840M] INFO: Compiled for linux/arm64/v8 (compiled on CI) using cc (Alpine 14.2.0) 14.2.0
2025-02-22 12:55:17.696 CET [840M] INFO: Wrote config file:
2025-02-22 12:55:17.696 CET [840M] INFO:  - 152 total entries
2025-02-22 12:55:17.697 CET [840M] INFO:  - 148 entries are default
2025-02-22 12:55:17.697 CET [840M] INFO:  - 4 entries are modified
2025-02-22 12:55:17.697 CET [840M] INFO:  - 0 entries are forced through environment
2025-02-22 12:55:17.699 CET [840M] INFO: Parsed config file /etc/pihole/pihole.toml successfully
2025-02-22 12:55:17.700 CET [840M] INFO: PID file does not exist or not readable
2025-02-22 12:55:17.700 CET [840M] INFO: No other running FTL process found.
2025-02-22 12:55:17.705 CET [840M] INFO: PID of FTL process: 840
2025-02-22 12:55:17.707 CET [840M] INFO: listening on 0.0.0.0 port 53
2025-02-22 12:55:17.707 CET [840M] INFO: listening on :: port 53
2025-02-22 12:55:17.709 CET [840M] INFO: PID of FTL process: 840
2025-02-22 12:55:17.711 CET [840M] INFO: Database version is 21
2025-02-22 12:55:17.712 CET [840M] INFO: Database successfully initialized
2025-02-22 12:55:17.860 CET [840M] INFO: Imported 5694 queries from the on-disk database (it has 5694 rows)
2025-02-22 12:55:17.860 CET [840M] INFO: Parsing queries in database
2025-02-22 12:55:17.914 CET [840M] INFO: Imported 5694 queries from the long-term database
2025-02-22 12:55:17.914 CET [840M] INFO:  -> Total DNS queries: 5694
2025-02-22 12:55:17.914 CET [840M] INFO:  -> Cached DNS queries: 4903
2025-02-22 12:55:17.914 CET [840M] INFO:  -> Forwarded DNS queries: 720
2025-02-22 12:55:17.914 CET [840M] INFO:  -> Blocked DNS queries: 46
2025-02-22 12:55:17.915 CET [840M] INFO:  -> Unknown DNS queries: 0
2025-02-22 12:55:17.915 CET [840M] INFO:  -> Unique domains: 310
2025-02-22 12:55:17.915 CET [840M] INFO:  -> Unique clients: 15
2025-02-22 12:55:17.915 CET [840M] INFO:  -> DNS cache records: 5
2025-02-22 12:55:17.915 CET [840M] INFO:  -> Known forward destinations: 2
2025-02-22 12:55:18.207 CET [840M] INFO: FTL is running as user pihole (UID 997)
2025-02-22 12:55:18.208 CET [840M] INFO: Reading certificate from /etc/pihole/tls.pem ...
2025-02-22 12:55:18.208 CET [840M] INFO: Using SSL/TLS certificate file /etc/pihole/tls.pem
2025-02-22 12:55:18.209 CET [840M] WARNING: No web server ports configured!
2025-02-22 12:55:18.209 CET [840/T76015] INFO: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2025-02-22 12:55:18.210 CET [840/T76015] INFO: ---------------------------->  FTL crashed!  <----------------------------
2025-02-22 12:55:18.210 CET [840/T76015] INFO: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2025-02-22 12:55:18.210 CET [840/T76015] INFO: Please report a bug at https://github.com/pi-hole/FTL/issues
2025-02-22 12:55:18.210 CET [840/T76015] INFO: and include in your report already the following details:
2025-02-22 12:55:18.210 CET [840M] INFO: Restored 1 API session from the database
2025-02-22 12:55:18.210 CET [840/T76015] INFO: FTL has been running for 1 seconds
2025-02-22 12:55:18.210 CET [840/T76015] INFO: FTL branch: master
2025-02-22 12:55:18.210 CET [840/T76015] INFO: FTL version: v6.0.2
2025-02-22 12:55:18.210 CET [840/T76015] INFO: FTL commit: ac500d5f
2025-02-22 12:55:18.210 CET [840/T76015] INFO: FTL date: 2025-02-21 21:48:20 +0000
2025-02-22 12:55:18.210 CET [840/T76015] INFO: FTL user: started as pihole, ended as pihole
2025-02-22 12:55:18.210 CET [840/T76015] INFO: Compiled for linux/arm64/v8 (compiled on CI) using cc (Alpine 14.2.0) 14.2.0
2025-02-22 12:55:18.211 CET [840/T76015] INFO: Process details: MID: 840
2025-02-22 12:55:18.211 CET [840/T76015] INFO:                  PID: 840
2025-02-22 12:55:18.211 CET [840/T76015] INFO:                  TID: 76015
2025-02-22 12:55:18.211 CET [840/T76015] INFO:                  Name: civetweb-master
2025-02-22 12:55:18.211 CET [840/T76015] INFO: Received signal: Segmentation fault
2025-02-22 12:55:18.211 CET [840/T76015] INFO:      at address: 0
2025-02-22 12:55:18.211 CET [840/T76015] INFO:      with code:  SEGV_MAPERR (Address not mapped to object)
2025-02-22 12:55:18.211 CET [840/T76015] INFO: !!! INFO: pihole-FTL has not been compiled with glibc/backtrace support, not generating one !!!
2025-02-22 12:55:18.211 CET [840/T76015] INFO: ------ Listing content of directory /dev/shm ------
2025-02-22 12:55:18.211 CET [840/T76015] INFO: File Mode User:Group      Size  Filename
2025-02-22 12:55:18.212 CET [840/T76015] INFO: rwxrwxrwx root:root       360   .
2025-02-22 12:55:18.212 CET [840/T76015] INFO: rwxr-xr-x root:root        14K  ..
2025-02-22 12:55:18.212 CET [840/T76015] INFO: rw------- pihole:pihole   786K  FTL-840-recycler
2025-02-22 12:55:18.212 CET [840/T76015] INFO: rw------- pihole:pihole     4K  FTL-840-dns-cache-lookup
2025-02-22 12:55:18.212 CET [840/T76015] INFO: rw------- pihole:pihole     4K  FTL-840-domains-lookup
2025-02-22 12:55:18.243 CET [840/T76015] INFO: rw------- pihole:pihole     4K  FTL-840-clients-lookup
2025-02-22 12:55:18.243 CET [840/T76015] INFO: rw------- pihole:pihole   569K  FTL-840-fifo-log
2025-02-22 12:55:18.244 CET [840/T76015] INFO: rw------- pihole:pihole     4K  FTL-840-per-client-regex
2025-02-22 12:55:18.244 CET [840/T76015] INFO: rw------- pihole:pihole    20K  FTL-840-dns-cache
2025-02-22 12:55:18.245 CET [840/T76015] INFO: rw------- pihole:pihole     8K  FTL-840-overTime
2025-02-22 12:55:18.245 CET [840/T76015] INFO: rw------- pihole:pihole   590K  FTL-840-queries
2025-02-22 12:55:18.245 CET [840/T76015] INFO: rw------- pihole:pihole    29K  FTL-840-upstreams
2025-02-22 12:55:18.246 CET [840/T76015] INFO: rw------- pihole:pihole   348K  FTL-840-clients
2025-02-22 12:55:18.246 CET [840/T76015] INFO: rw------- pihole:pihole    12K  FTL-840-domains
2025-02-22 12:55:18.247 CET [840/T76015] INFO: rw------- pihole:pihole    82K  FTL-840-strings
2025-02-22 12:55:18.247 CET [840/T76015] INFO: rw------- pihole:pihole   144   FTL-840-settings
2025-02-22 12:55:18.247 CET [840/T76015] INFO: rw------- pihole:pihole   328   FTL-840-counters
2025-02-22 12:55:18.248 CET [840/T76015] INFO: rw------- pihole:pihole    88   FTL-840-lock
2025-02-22 12:55:18.248 CET [840/T76015] INFO: ---------------------------------------------------
2025-02-22 12:55:18.248 CET [840/T76015] INFO: Please also include some lines from above the !!!!!!!!! header.
2025-02-22 12:55:18.248 CET [840/T76015] INFO: Thank you for helping us to improve our FTL engine!
2025-02-22 12:55:18.249 CET [840/T76015] INFO: PID file emptied
2025-02-22 12:55:18.253 CET [840/T76015] INFO: Stored 1 API session in the database
2025-02-22 12:55:18.253 CET [840M] INFO: Blocking status is enabled
2025-02-22 12:55:18.314 CET [840/T76010] INFO: Compiled 0 allow and 0 deny regex for 15 clients in 0.6 msec
2025-02-22 12:55:22.538 CET [840/T76009] INFO: Received 8/8 valid NTP replies from pool.ntp.org
2025-02-22 12:55:22.538 CET [840/T76009] INFO: Time offset: -4.220009e-01 ms (excluded 0 outliers)
2025-02-22 12:55:22.538 CET [840/T76009] INFO: Round-trip delay: 3.206170e+01 ms (excluded 0 outliers)
2025-02-22 12:55:22.539 CET [840/T76045] INFO: NTP server listening on :::123 (IPv6)
2025-02-22 12:55:22.539 CET [840/T76044] INFO: NTP server listening on 0.0.0.0:123 (IPv4)
2025-02-22 13:55:22.846 CET [840/T76009] INFO: Received 8/8 valid NTP replies from pool.ntp.org
2025-02-22 13:55:22.846 CET [840/T76009] INFO: Time offset: -3.943137e+00 ms (excluded 1 outliers)
2025-02-22 13:55:22.847 CET [840/T76009] INFO: Round-trip delay: 2.999980e+01 ms (excluded 1 outliers)
2025-02-22 14:55:23.191 CET [840/T76009] INFO: Received 8/8 valid NTP replies from pool.ntp.org
2025-02-22 14:55:23.191 CET [840/T76009] INFO: Time offset: -2.968311e+00 ms (excluded 1 outliers)
2025-02-22 14:55:23.191 CET [840/T76009] INFO: Round-trip delay: 3.016894e+01 ms (excluded 1 outliers)
2025-02-22 15:55:23.448 CET [840/T76009] INFO: Received 8/8 valid NTP replies from pool.ntp.org
2025-02-22 15:55:23.448 CET [840/T76009] INFO: Time offset: 2.573150e+00 ms (excluded 1 outliers)
2025-02-22 15:55:23.448 CET [840/T76009] INFO: Round-trip delay: 1.997798e+01 ms (excluded 1 outliers)

@DaitiDay
Copy link
Author

After reading this article I've stopped and disabled lighttpd for the moment (have to check if it was needed by some other service).
I also saw that right before crashing the log file states WARNING: No web server ports configured!, so I launched pihole-FTL --config webserver.port (as suggested in the previous article) and got the following output:

80o,443os,[::]:80o,[::]:443os

Following the suggestion in #5948 I've also set listeningMode = "ALL" in /etc/pihole/pihole.toml.
The problem is still present.

@konopka
Copy link

konopka commented Feb 23, 2025

I've got same isssue

pi-hole-pihole-1  | 2025-02-23 16:16:41.099 CET [53/T154] INFO: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
pi-hole-pihole-1  | 2025-02-23 16:16:41.100 CET [53/T154] INFO: ---------------------------->  FTL crashed!  <----------------------------
pi-hole-pihole-1  | 2025-02-23 16:16:41.100 CET [53/T154] INFO: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
pi-hole-pihole-1  | 2025-02-23 16:16:41.100 CET [53/T154] INFO: Please report a bug at https://github.com/pi-hole/FTL/issues
pi-hole-pihole-1  | 2025-02-23 16:16:41.100 CET [53/T154] INFO: and include in your report already the following details:
pi-hole-pihole-1  | 2025-02-23 16:16:41.100 CET [53/T154] INFO: FTL has been running for 9 seconds
pi-hole-pihole-1  | 2025-02-23 16:16:41.100 CET [53/T154] INFO: FTL branch: master
pi-hole-pihole-1  | 2025-02-23 16:16:41.100 CET [53/T154] INFO: FTL version: v6.0.2
pi-hole-pihole-1  | 2025-02-23 16:16:41.100 CET [53/T154] INFO: FTL commit: ac500d5f
pi-hole-pihole-1  | 2025-02-23 16:16:41.100 CET [53/T154] INFO: FTL date: 2025-02-21 21:48:20 +0000
pi-hole-pihole-1  | 2025-02-23 16:16:41.100 CET [53/T154] INFO: FTL user: started as pihole, ended as pihole
pi-hole-pihole-1  | 2025-02-23 16:16:41.100 CET [53/T154] INFO: Compiled for linux/amd64 (compiled on CI) using cc (Alpine 14.2.0) 14.2.0
pi-hole-pihole-1  | 2025-02-23 16:16:41.101 CET [53/T154] INFO: Process details: MID: 53
pi-hole-pihole-1  | 2025-02-23 16:16:41.101 CET [53/T154] INFO:                  PID: 53
pi-hole-pihole-1  | 2025-02-23 16:16:41.101 CET [53/T154] INFO:                  TID: 154
pi-hole-pihole-1  | 2025-02-23 16:16:41.101 CET [53/T154] INFO:                  Name: civetweb-worker
pi-hole-pihole-1  | 2025-02-23 16:16:41.101 CET [53/T154] INFO: Received signal: Segmentation fault
pi-hole-pihole-1  | 2025-02-23 16:16:41.101 CET [53/T154] INFO:      at address: 0
pi-hole-pihole-1  | 2025-02-23 16:16:41.101 CET [53/T154] INFO:      with code:  SEGV_MAPERR (Address not mapped to object)
pi-hole-pihole-1  | 2025-02-23 16:16:41.101 CET [53/T154] INFO: !!! INFO: pihole-FTL has not been compiled with glibc/backtrace support, not generating one !!!
pi-hole-pihole-1  | 2025-02-23 16:16:41.101 CET [53/T154] INFO: ------ Listing content of directory /dev/shm ------
pi-hole-pihole-1  | 2025-02-23 16:16:41.101 CET [53/T154] INFO: File Mode User:Group      Size  Filename
pi-hole-pihole-1  | 2025-02-23 16:16:41.101 CET [53/T154] INFO: rwxrwxrwx root:root       360   .
pi-hole-pihole-1  | 2025-02-23 16:16:41.102 CET [53/T154] INFO: rwxr-xr-x root:root       340   ..
pi-hole-pihole-1  | 2025-02-23 16:16:41.102 CET [53/T154] INFO: rw------- pihole:pihole   786K  FTL-53-recycler
pi-hole-pihole-1  | 2025-02-23 16:16:41.102 CET [53/T154] INFO: rw------- pihole:pihole     4K  FTL-53-dns-cache-lookup
pi-hole-pihole-1  | 2025-02-23 16:16:41.102 CET [53/T154] INFO: rw------- pihole:pihole     4K  FTL-53-domains-lookup
pi-hole-pihole-1  | 2025-02-23 16:16:41.102 CET [53/T154] INFO: rw------- pihole:pihole     4K  FTL-53-clients-lookup
pi-hole-pihole-1  | 2025-02-23 16:16:41.102 CET [53/T154] INFO: rw------- pihole:pihole   569K  FTL-53-fifo-log
pi-hole-pihole-1  | 2025-02-23 16:16:41.103 CET [53/T154] INFO: rw------- pihole:pihole     4K  FTL-53-per-client-regex
pi-hole-pihole-1  | 2025-02-23 16:16:41.103 CET [53/T154] INFO: rw------- pihole:pihole    20K  FTL-53-dns-cache
pi-hole-pihole-1  | 2025-02-23 16:16:41.103 CET [53/T154] INFO: rw------- pihole:pihole     8K  FTL-53-overTime
pi-hole-pihole-1  | 2025-02-23 16:16:41.103 CET [53/T154] INFO: rw------- pihole:pihole   295K  FTL-53-queries
pi-hole-pihole-1  | 2025-02-23 16:16:41.103 CET [53/T154] INFO: rw------- pihole:pihole    29K  FTL-53-upstreams
pi-hole-pihole-1  | 2025-02-23 16:16:41.103 CET [53/T154] INFO: rw------- pihole:pihole   348K  FTL-53-clients
pi-hole-pihole-1  | 2025-02-23 16:16:41.104 CET [53/T154] INFO: rw------- pihole:pihole     4K  FTL-53-domains
pi-hole-pihole-1  | 2025-02-23 16:16:41.104 CET [53/T154] INFO: rw------- pihole:pihole    82K  FTL-53-strings
pi-hole-pihole-1  | 2025-02-23 16:16:41.104 CET [53/T154] INFO: rw------- pihole:pihole   144   FTL-53-settings
pi-hole-pihole-1  | 2025-02-23 16:16:41.104 CET [53/T154] INFO: rw------- pihole:pihole   328   FTL-53-counters
pi-hole-pihole-1  | 2025-02-23 16:16:41.104 CET [53/T154] INFO: rw------- pihole:pihole    88   FTL-53-lock
pi-hole-pihole-1  | 2025-02-23 16:16:41.104 CET [53/T154] INFO: ---------------------------------------------------
pi-hole-pihole-1  | 2025-02-23 16:16:41.104 CET [53/T154] INFO: Please also include some lines from above the !!!!!!!!! header.

@DL6ER
Copy link
Member

DL6ER commented Feb 23, 2025

It would be extremely helpful if you could follow the steps on https://docs.pi-hole.net/ftldns/gdb/ and provide the backtrace on this crash. I have tried to reproduce it and wasn't successful, thus far.

@DL6ER DL6ER added the Bug label Feb 23, 2025
@DaitiDay
Copy link
Author

DaitiDay commented Feb 23, 2025

I tried following the guide, but the command sudo gdb -p $(cat /run/pihole-FTL.pid) gave

cat: /run/pihole-FTL.pid: No such file or directory
gdb: option '-p' requires an argument
Use `gdb --help' for a complete list of options.

Do I need to restart, use the command and then try to add/delete a CNAME to make it crash?

Edit: After some tests looks like there is problem with the /run/pihole-FTL.pid file. The file is empty. If I restart the pihole-FTL service I get the error PID file does not exist or not readable in pihole-FTL status.

@DaitiDay
Copy link
Author

@DL6ER I restarted the pi and with everything running I started the debugger and followed the guide. This is the output of gdb until the crash

GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.2) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 841
[New LWP 1017]
[New LWP 1018]
[New LWP 1019]
[New LWP 1020]
[New LWP 1021]
[New LWP 1022]
[New LWP 1023]
__syscall_cp_asm () at src/thread/aarch64/syscall_cp.s:28
28      src/thread/aarch64/syscall_cp.s: No such file or directory.
(gdb) continue
Continuing.
[Detaching after vfork from child process 10720]
[Detaching after vfork from child process 10722]
[New LWP 11912]
[New LWP 11913]
[New LWP 11922]
[Detaching after vfork from child process 13658]
[Detaching after vfork from child process 13660]
[Detaching after fork from child process 14499]

Thread 1 "pihole-FTL" received signal SIGTERM, Terminated.

Thread 1 "pihole-FTL" received signal SIG41, Real-time event 41.
[LWP 1019 exited]
[LWP 1018 exited]
[LWP 1021 exited]

Thread 2 "ntp-client" received signal SIG33, Real-time event 33.

Thread 5 "dns-client" received signal SIG33, Real-time event 33.
[LWP 1017 exited]
[LWP 1020 exited]
[LWP 11922 exited]
[LWP 11913 exited]
[LWP 11912 exited]
[LWP 1023 exited]
[LWP 1022 exited]
process 841 is executing new program: /usr/bin/pihole-FTL
[New LWP 14680]
[New LWP 14681]
[New LWP 14682]
[New LWP 14683]
[New LWP 14684]
[New LWP 14685]
[New LWP 14686]

Thread 18 "civetweb-master" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 14686]
0x000000000045018c in master_thread_run (ctx=ctx@entry=0xffffb1bf3460)
    at /app/src/webserver/civetweb/civetweb.c:20683
20683   /app/src/webserver/civetweb/civetweb.c: No such file or directory.
(gdb)
Thread 1 "pihole-FTL" received signal SIG41, Real-time event 41.
[LWP 1019 exited]
[LWP 1018 exited]
[LWP 1021 exited]

Thread 2 "ntp-client" received signal SIG33, Real-time event 33.

Thread 5 "dns-client" received signal SIG33, Real-time event 33.
[LWP 1017 exited]
[LWP 1020 exited]
[LWP 11922 exited]
[LWP 11913 exited]
[LWP 11912 exited]
[LWP 1023 exited]
[LWP 1022 exited]
process 841 is executing new program: /usr/bin/pihole-FTL
[New LWP 14680]
[New LWP 14681]
[New LWP 14682]
[New LWP 14683]
[New LWP 14684]
[New LWP 14685]
[New LWP 14686]

Thread 18 "civetweb-master" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 14686]
0x000000000045018c in master_thread_run (ctx=ctx@entry=0xffffb1bf3460)
    at /app/src/webserver/civetweb/civetweb.c:20683
20683   /app/src/webserver/civetweb/civetweb.c: No such file or directory.

This is the output of the backtrace command

0x000000000045018c in master_thread_run (ctx=ctx@entry=0xffffb1bf3460)
    at /app/src/webserver/civetweb/civetweb.c:20683
20683   /app/src/webserver/civetweb/civetweb.c: No such file or directory.
(gdb) backtrace
#0  0x000000000045018c in master_thread_run (ctx=ctx@entry=0xffffb1bf3460)
    at /app/src/webserver/civetweb/civetweb.c:20683
#1  0x00000000004509a4 in master_thread (thread_func_param=0xffffb1bf3460)
    at /app/src/webserver/civetweb/civetweb.c:20800
#2  0x00000000007b79e4 in start (p=0xffffb1bcfb00) at src/thread/pthread_create.c:207
#3  0x00000000007c1ac0 in __clone () at src/thread/aarch64/clone.s:28
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Tell me if something else is needed.

@DL6ER
Copy link
Member

DL6ER commented Feb 25, 2025

#0 0x000000000045018c in master_thread_run (ctx=ctx@entry=0xffffb1bf3460)
at /app/src/webserver/civetweb/civetweb.c:20683

/* We listen on this socket just so that mg_stop() can cause mg_poll()
* to return ASAP. Don't worry, we did allocate an extra slot at the end
* of listening_socket_fds[] just to hold this
*/
pfd[ctx->num_listening_sockets].fd =
ctx->thread_shutdown_notification_socket;

This comment somehow feels fishy here...

If you can reproduce this (and the crash happens again at the same location), what is the output of

p ctx
p ctx->num_listening_sockets
p pfd
p pfd[ctx->num_listening_sockets]

?

@DaitiDay
Copy link
Author

I restarted and re-run all previous commands: backtrace output:

0x000000000045018c in master_thread_run (ctx=ctx@entry=0xffff7d1664b0)
    at /app/src/webserver/civetweb/civetweb.c:20683
20683   /app/src/webserver/civetweb/civetweb.c: No such file or directory.
(gdb) backtrace
#0  0x000000000045018c in master_thread_run (ctx=ctx@entry=0xffff7d1664b0)
    at /app/src/webserver/civetweb/civetweb.c:20683
#1  0x00000000004509a4 in master_thread (thread_func_param=0xffff7d1664b0)
    at /app/src/webserver/civetweb/civetweb.c:20800
#2  0x00000000007b79e4 in start (p=0xffff7d142b00) at src/thread/pthread_create.c:207
#3  0x00000000007c1ac0 in __clone () at src/thread/aarch64/clone.s:28
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Then output of p ctx

(gdb) p ctx
$1 = (struct mg_context *) 0xffff7d1664b0
(gdb) p ctx->num_listening_sockets
$2 = 0
(gdb) p pfd
$3 = (struct pollfd *) 0x0
(gdb) p pfd[ctx->num_listening_sockets]
Cannot access memory at address 0x0

@DL6ER
Copy link
Member

DL6ER commented Mar 1, 2025

Sorry for the delay in replying ...

Could you please try if the issue is resolved by running

sudo pihole checkout ftl update/civetweb

?

@DL6ER DL6ER added CivetWeb bug and removed Bug labels Mar 1, 2025
@DaitiDay
Copy link
Author

DaitiDay commented Mar 1, 2025

Not solved.
End of gdb

Thread 18 "civetweb-master" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 10351]
0x0000000000450e8c in master_thread_run (ctx=ctx@entry=0xffff8ad77570)
    at /app/src/webserver/civetweb/civetweb.c:20685
20685   /app/src/webserver/civetweb/civetweb.c: No such file or directory.

Output of backtrace

(gdb) backtrace
#0  0x0000000000450e8c in master_thread_run (ctx=ctx@entry=0xffff8ad77570)
    at /app/src/webserver/civetweb/civetweb.c:20685
#1  0x00000000004516a4 in master_thread (thread_func_param=0xffff8ad77570)
    at /app/src/webserver/civetweb/civetweb.c:20802
#2  0x00000000007b90c4 in start (p=0xffff8ad53b00) at src/thread/pthread_create.c:207
#3  0x00000000007c31a0 in __clone () at src/thread/aarch64/clone.s:28
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

@DL6ER
Copy link
Member

DL6ER commented Mar 1, 2025

Too bad, okay, let's dive deeper into this. I'm trying to see where pfd = 0 may be coming from and I may see a path but it'd mean none of the ports your webserver tried to bind to succeeded binding to. But even then ...

Do you see log lines like

2025-03-01 10:08:59.750 CET [2733749M] INFO: Web server ports:
2025-03-01 10:08:59.750 CET [2733749M] INFO:   - 0.0.0.0:80 (HTTP, IPv4, bound)
2025-03-01 10:08:59.750 CET [2733749M] INFO:   - [::]:80 (HTTP, IPv6, bound)
2025-03-01 10:08:59.751 CET [2733749M] INFO:   - 0.0.0.0:443 (HTTPS, IPv4, bound)
2025-03-01 10:08:59.751 CET [2733749M] INFO:   - [::]:443 (HTTPS, IPv6, bound)
2025-03-01 10:08:59.751 CET [2733749M] INFO:   - 127.0.0.67:445 (HTTPS, IPv4, bound)

in your /var/log/pihole/FTL.log file?

@DaitiDay
Copy link
Author

DaitiDay commented Mar 1, 2025

This is the content of a clean FTL.log after sudo service pihole-FTL restart

The only reference to ports I can find is the port 53 (DNS?), nothing else.

Two thing I did not specify before:

  1. On the same rpi I have a docker traefik instance running on port 80 and 443
  2. On the previous pihole installation I installed unbound to use the pi as a DNS, but then I switched back to Google DNS in pihole settings, leaving unbound settings unchanged. (for reference, I followed Craft computing guide and unbound documentation)

I did not report this before because the previous version of pihole was working fine with those settings.

@DL6ER
Copy link
Member

DL6ER commented Mar 1, 2025

Ah, I did just spot

WARNING: No web server ports configured!

in your log. This is about what I assumed:

I may see a path but it'd mean none of the ports your webserver tried to bind to succeeded binding to

just with the twist that it doesn't even try to bind anything. What is the output of

sudo pihole-FTL --config webserver.port

?

@DaitiDay
Copy link
Author

DaitiDay commented Mar 1, 2025

80o,443os,[::]:80o,[::]:443os

@DL6ER
Copy link
Member

DL6ER commented Mar 1, 2025

Thank you. It seems the CivetWeb server crashes if all optional ports are already occupied, I did just report this upstream: civetweb/civetweb#1327

Please try

sudo pihole checkout ftl fix/civetweb_all_ports_optional

if that fixes the issue for you. You should still set other ports so your Pi-hole can offer the API when there is already a webserver on the same host.

@DaitiDay
Copy link
Author

DaitiDay commented Mar 1, 2025

Please note that changing branches severely alters your Pi-hole subsystems
  Features that work on the master branch, may not on a development branch
  This feature is NOT supported unless a Pi-hole developer explicitly asks!
  Have you read and understood this? [y/N] y

  [i] Checking for availability of branch fix/civetweb_all_ports_optional on GitHub
  [✓] Branch fix/civetweb_all_ports_optional exists on GitHub
  [i] Checking for pihole-FTL-arm64 binary on https://ftl.pi-hole.net
  [✗] Binary for requested branch is not available, please try again later.
      If the issue persists, please contact Pi-hole Support and ask them to re-generate the binary.

@DL6ER
Copy link
Member

DL6ER commented Mar 1, 2025

You (or me) may have been too fast and the binary may have not been created. I did just check and everything is there now, please try again.

@DaitiDay
Copy link
Author

DaitiDay commented Mar 1, 2025

Should I change something else? Switching branch did not solve the problem. This is the content of FTL.log for reference: log

@DL6ER
Copy link
Member

DL6ER commented Mar 1, 2025

Please try again in ~ 5 minutes. I have a proposed fix which I will also send upstream to the CivetWeb project

@DaitiDay
Copy link
Author

DaitiDay commented Mar 1, 2025

Ok, I'll restart it in 10 min to be sure. In the meanwhile, can you clarify what you meant by

You should still set other ports so your Pi-hole can offer the API when there is already a webserver on the same host.

What should I do?

@DL6ER
Copy link
Member

DL6ER commented Mar 1, 2025

The arm64 binary is now built. The upstream bug fix is civetweb/civetweb#1328

What should I do?

Please first if the crash is gone after running the checkout command again. Then you should also see a clear message in the log file that tells you what to do. You can, e.g., set the webserver to listen on port 81 instead (TLS on 444) using

sudo pihole-FTL --config webserver.port "81,444s"

@DaitiDay
Copy link
Author

DaitiDay commented Mar 1, 2025

@DL6ER which branch should I checkout? sudo pihole checkout ftl update/civetweb?

@DL6ER
Copy link
Member

DL6ER commented Mar 1, 2025

@DaitiDay Yes, exactly as before.

@DaitiDay
Copy link
Author

DaitiDay commented Mar 1, 2025

I switched branch and launched sudo pihole-FTL --config webserver.port "81,444s".

After a reboot there seems to be no more crashes. I'll update tomorrow in case something happens, otherwise I'll close the issue if you agree @DL6ER

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants