NetworkManager Support (#2218)

* reworked autohotspot configuration option

* updated autohotspot creation (copied changes from v2)

* fix placeholder prefix/suffix (%%)

* moved files in subdir for dhcpcd

* renamed setup_autohotspot file

* added switch for network settings

* added autohotspot based on NetworkManager

* update function names

* fixed duplicate variables

* fix var check

* handle CI run for autohotspot setup

* changed network management check for ci

* fixed timer service name

* renamed script. delete obsolete file

* fix syntax

* fix timer service unit definition

* move is is_service_enabled to helper class

* extracted konstant and fixed formatting

* fix indentation

* added shebang again

* some refactorings in autohotspot script

* fix checks

* fix for local vars and arrays. refactor nw_profiles.

* refactored var value and log output

* refactor device and ssid checks

* refactore force_hotspot option

* refactor is_active_ap check and bugfixes

* update log output

* made timer reenablement configurable

* phoniebox header added

* moved check to helper. removed obsolete  code

* removed ip configuration. changed ip

* add note for static ip conf with autohotspot

* harmonize logging and opts

* changed dhcpcd timer from cron to systemd.
add seperate daemon service for wpa-supplicant handling

* fix wifi after restart if autohot spot is deactivated

* add hostname to default hotspot ssid

* updated autohotspot docs

* fix static ip for NetworkManager.
fix ipv6 disablement

* fix boot file paths

* some fixes

* pull config file backup to helpers

* combine service is-enabled calls

* update network related docs

* Update autohotspot.md

Aligned some language and typos

* Update installation.md

Aligning wording and removing typos

---------

Co-authored-by: pabera <[email protected]>

ci/ci-debian.Dockerfile

@@ -8,8 +8,8 @@ ENV TERM=xterm DEBIAN_FRONTEND=noninteractive
 ENV CI_RUNNING=true
 
 # create RPi configs to test installation
-RUN touch /boot/config.txt
-RUN echo "logo.nologo" > /boot/cmdline.txt
+RUN mkdir -p /boot && touch /boot/config.txt && echo "logo.nologo" > /boot/cmdline.txt
+RUN mkdir -p /boot/firmware && touch /boot/firmware/config.txt && echo "logo.nologo" > /boot/firmware/cmdline.txt
 
 RUN echo "--- install packages (1) ---" \
   && apt-get update \

ci/installation/run_install_common.sh

@@ -17,7 +17,7 @@ export ENABLE_WEBAPP_PROD_DOWNLOAD=true
 # n - use static ip
 # n - deactivate ipv6
 # y - setup autohotspot
-# n -   use custom password
+# n -   change default configuration
 # n - deactivate bluetooth
 # n - disable on-chip audio
 # - - mpd overwrite config (only with existing installation)

ci/installation/run_install_faststartup.sh

@@ -11,11 +11,13 @@ SCRIPT_DIR="$(dirname "$SOURCE")"
 LOCAL_INSTALL_SCRIPT_PATH="${INSTALL_SCRIPT_PATH:-${SCRIPT_DIR}/../../installation}"
 LOCAL_INSTALL_SCRIPT_PATH="${LOCAL_INSTALL_SCRIPT_PATH%/}"
 
+
 # Run installation (in interactive mode)
 # y - start setup
 # y - use static ip
 # y - deactivate ipv6
 # n - setup autohotspot
+# - -   change default configuration (only with autohotspot = y)
 # y - deactivate bluetooth
 # y - disable on-chip audio
 # - - mpd overwrite config (only with existing installation)

ci/installation/run_install_libzmq_local.sh

@@ -17,6 +17,7 @@ export BUILD_LIBZMQ_WITH_DRAFTS_ON_DEVICE=true
 # n - use static ip
 # n - deactivate ipv6
 # n - setup autohotspot
+# - -   change default configuration (only with autohotspot = y)
 # n - deactivate bluetooth
 # n - disable on-chip audio
 # - - mpd overwrite config (only with existing installation)

ci/installation/run_install_webapp_download.sh

@@ -11,11 +11,13 @@ SCRIPT_DIR="$(dirname "$SOURCE")"
 LOCAL_INSTALL_SCRIPT_PATH="${INSTALL_SCRIPT_PATH:-${SCRIPT_DIR}/../../installation}"
 LOCAL_INSTALL_SCRIPT_PATH="${LOCAL_INSTALL_SCRIPT_PATH%/}"
 
+
 # Run installation (in interactive mode)
 # y - start setup
 # n - use static ip
 # n - deactivate ipv6
 # n - setup autohotspot
+# - -   change default configuration (only with autohotspot = y)
 # n - deactivate bluetooth
 # n - disable on-chip audio
 # - - mpd overwrite config (only with existing installation)

ci/installation/run_install_webapp_local.sh

@@ -17,6 +17,7 @@ export ENABLE_WEBAPP_PROD_DOWNLOAD=false
 # n - use static ip
 # n - deactivate ipv6
 # n - setup autohotspot
+# - -   change default configuration (only with autohotspot = y)
 # n - deactivate bluetooth
 # n - disable on-chip audio
 # - - mpd overwrite config (only with existing installation)

documentation/builders/autohotspot.md

@@ -1,107 +1,63 @@
 # Auto-Hotspot
 
-The Auto-Hotspot function allows the Jukebox to switch between its
-connection between a known WiFi and an automatically generated hotspot
-so that you can still access via SSH or Web App.
+The Auto-Hotspot function enables the Jukebox to switch its connection between a known WiFi network and an automatically generated hotspot, allowing access via SSH or Web App.
 
 > [!IMPORTANT]
-> Please configure the WiFi connection to your home access point before enabling these feature!
-
-To create a hotspot and allow clients to connect
-[hostapd](http://w1.fi/hostapd/) and [dnsmasq](https://thekelleys.org.uk/dnsmasq/doc.html).
+> Please configure the WiFi connection to your home access point before enabling this feature!
 
 ## How to connect
 
-When the Jukebox is not able to connect to a known WiFi it will create a
-hotspot named `Phoniebox_Hotspot`. You will be able to connect to this
-hotspot using the given password in the installation or the default
-password: `PlayItLoud!`
-
-### Web App
-
-After connecting to the `Phoniebox_Hotspot` you are able to connect to
-the Web App accessing the website [10.0.0.5](http://10.0.0.5/).
+When the Jukebox cannot connect to a known WiFi, it will automatically create a hotspot. 
+You can connect to this hotspot using the password set during installation.
+Afterwards, you can access the Web App or connect via SSH as before, using the IP from the configuration.
 
-### ssh
-
-After connecting to the `Phoniebox_Hotspot` you are able to connect via
-ssh to your Jukebox
-
-``` bash
-ssh <username>@10.0.0.5
-```
-
-## Changing basic configuration of the hotspot
-
-The whole hotspot configuration can be found at
-`/etc/hostapd/hostapd.conf`.
-
-The following parameters are relevant:
-
-- `ssid` for the displayed hotspot name
-- `wpa_passphrase` for the password of the hotspot
-- `country_code` the country you are currently in
-
-``` bash
-$ cat /etc/hostapd/hostapd.conf
-
-#2.4GHz setup wifi 80211 b,g,n
-interface=wlan0
-driver=nl80211
-ssid=Phoniebox_Hotspot
-hw_mode=g
-channel=8
-wmm_enabled=0
-macaddr_acl=0
-auth_algs=1
-ignore_broadcast_ssid=0
-wpa=2
-wpa_passphrase==PlayItLoud!
-wpa_key_mgmt=WPA-PSK
-wpa_pairwise=CCMP TKIP
-rsn_pairwise=CCMP
-
-#80211n - Change GB to your WiFi country code
-country_code=DE
-ieee80211n=1
-ieee80211d=1
+The default configuration is
+``` text
+* SSID              : Phoniebox_Hotspot_<hostname>
+* Password          : PlayItLoud!
+* WiFi Country Code : DE
+* IP                : 10.0.0.1
 ```
 
 ## Disabling automatism
 
-Auto-Hotspot can be enabled or disabled using the Web App.
+Auto-Hotspot can be enabled or disabled using the Web App or RPC Commands.
+
+> [!NOTE]
+> Disabling the Auto-Hotspot will run the WiFi check again and maintain the last connection state until reboot.
 
 > [!IMPORTANT]
-> Disabling or enabling will keep the last state.
+> If you disable this feature, you will lose access to the Jukebox if you are not near a known WiFi after reboot!
 
 ## Troubleshooting
 
-### Phoniebox is not connecting to the known WiFi
-
-The script will fall back to the hotspot so you still have some type of
-connection.
-
-Check your password in `/etc/wpa_supplicant/wpa_supplicant.conf`.
-
 ### AutoHotspot functionality is not working
 
-You can check the output of the script by running the following script:
+Check the `autohotspot.service` status
+``` bash
+sudo systemctl status autohotspot.service
+```
 
+and logs
 ``` bash
-$ sudo /usr/bin/autohotspot
+sudo journalctl -u autohotspot.service -n 50
 ```
 
-### You need to add a new wifi network to the Raspberry Pi
+### Jukebox is not connecting to the known WiFi
+
+The script will fall back to the hotspot, ensuring you still have some type of connection.
+
+Check your WiFi configuration.
 
-Because it is in Auto-Hotspot mode, you won\'t be able to scan for new
-wifi signals.
+### You need to add a new WiFi network to the Raspberry Pi
 
-You will need to add a new network to
-`/etc/wpa_supplicant/wpa_supplicant.conf` manually. Enter the following
-details replacing mySSID and myPassword with your details. If your WiFi
-has a hidden SSID then include the line `scan_ssid=1`.
+#### Using the command line
+Connect to the hotspot and open a terminal. Use the [raspi-config](https://www.raspberrypi.com/documentation/computers/configuration.html#wireless-lan) tool to add the new WiFi.
 
 ## Resources
 
-[Raspberry Pi - Auto WiFi Hotspot Switch - Direct
-Connection](https://www.raspberryconnect.com/projects/65-raspberrypi-hotspot-accesspoints/158-raspberry-pi-auto-wifi-hotspot-switch-direct-connection)
+* [Raspberry Connect - Auto WiFi Hotspot Switch](https://www.raspberryconnect.com/projects/65-raspberrypi-hotspot-accesspoints/158-raspberry-pi-auto-wifi-hotspot-switch-direct-connection)
+* [Raspberry Pi - Configuring networking](https://www.raspberrypi.com/documentation/computers/configuration.html#using-the-command-line)
+* [dhcpcd / wpa_supplicant]()
+    * [hostapd](http://w1.fi/hostapd/)
+    * [dnsmasq](https://thekelleys.org.uk/dnsmasq/doc.html)

documentation/builders/installation.md

@@ -11,7 +11,6 @@ Before you can install the Phoniebox software, you need to prepare your Raspberr
 2. Download the [Raspberry Pi Imager](https://www.raspberrypi.com/software/) and run it
 3. Click on "Raspberry Pi Device" and select "No filtering"
 4. As operating system select **Raspberry Pi OS (other)** and then **Raspberry Pi OS Lite (Legacy, 32-bit)** (no desktop environment). *64-bit is currently not supported*.
-    * Bookworm support is partly broken, see [here](#workaround-for-network-related-features-on-bookworm).
     * For Pi 4 and newer also check [this](#workaround-for-64-bit-kernels-pi-4-and-newer).
 5. Select your Micro SD card (your card will be formatted)
 6. After you click `Next`, a prompt will ask you if you like to customize the OS settings
@@ -81,21 +80,21 @@ You will need a terminal, like PuTTY for Windows or the Terminal app for Mac to
 
 ### Pre-install preparation / workarounds
 
-#### Workaround for network related features on Bookworm
+#### Network management since Bookworm
 <details>
-With Bookworm the network settings have changed. Now "NetworkManager" is used instead of "dhcpcd".
-This breaks breaks network related features like "Static IP", "Wifi Setup" and "Autohotspot".
-Before running the installation, the network config has to be changed via raspi-config, to use the "old" dhcpcd network settings.
+With Bookworm, network management has changed. Now, "NetworkManager" is used instead of "dhcpcd". 
+Both methods are supported during installation, but "NetworkManager" is recommended as it is simpler to set up and use.
+For Bullseye, this can also be activated, though it requires a manual process before running the installation.
 
 :warning:
-If the settings are changed, your network will reset and Wifi will not be configured, so you lose ssh access via wireless connection.
-So make sure you use a wired connection or perform the following steps in a local terminal with a connected monitor and keyboard.
+If the settings are changed, your network will reset, and WiFi will not be configured, causing you to lose SSH access via wireless connection.
+Therefore, make sure you use a wired connection or perform the following steps in a local terminal with a connected monitor and keyboard.
 
 Change network config
 * run `sudo raspi-config`
 * select `6 - Advanced Options`
 * select `AA - Network Config`
-* select `dhcpcd`
+* select `NetworkManager`
 
 If you need Wifi, add the information now
 * select `1 - System Options`

installation/includes/00_constants.sh

@@ -1,7 +1,6 @@
-RPI_BOOT_CONFIG_FILE="/boot/config.txt"
-RPI_BOOT_CMDLINE_FILE="/boot/cmdline.txt"
 SHARED_PATH="${INSTALLATION_PATH}/shared"
 SETTINGS_PATH="${SHARED_PATH}/settings"
+SYSTEMD_PATH="/etc/systemd/system"
 SYSTEMD_USR_PATH="/usr/lib/systemd/user"
 VIRTUAL_ENV="${INSTALLATION_PATH}/.venv"
 # Do not change this directory! It must match MPDs expectation where to find the user configuration

installation/includes/01_default_config.sh

@@ -4,8 +4,11 @@ BUILD_LIBZMQ_WITH_DRAFTS_ON_DEVICE=${BUILD_LIBZMQ_WITH_DRAFTS_ON_DEVICE:-"false"
 ENABLE_STATIC_IP=true
 DISABLE_IPv6=true
 ENABLE_AUTOHOTSPOT=false
-AUTOHOTSPOT_CHANGE_PASSWORD=false
+AUTOHOTSPOT_PROFILE="Phoniebox_Hotspot"
+AUTOHOTSPOT_SSID="$AUTOHOTSPOT_PROFILE"
 AUTOHOTSPOT_PASSWORD="PlayItLoud!"
+AUTOHOTSPOT_IP="10.0.0.1"
+AUTOHOTSPOT_COUNTRYCODE="DE"
 DISABLE_BLUETOOTH=true
 DISABLE_SSH_QOS=true
 DISABLE_BOOT_SCREEN=true
@@ -18,7 +21,6 @@ ENABLE_SAMBA=true
 ENABLE_WEBAPP=true
 ENABLE_KIOSK_MODE=false
 DISABLE_ONBOARD_AUDIO=false
-DISABLE_ONBOARD_AUDIO_BACKUP="${RPI_BOOT_CONFIG_FILE}.backup.audio_on_$(date +%d.%m.%y_%H.%M.%S)"
 # Always try to use GIT with SSH first, and on failure drop down to HTTPS
 GIT_USE_SSH=${GIT_USE_SSH:-"true"}