Add swap file adjustment for webapp build (#2204)

* adjust swapfile if memory is too low * add option for webapp build * update ci tests * Update run_install_webapp_local.sh * update ENABLE_WEBAPP_BUILD handling * updated logging * add build webapp test workflow * fix path * fix actions * fix: add shell * fix trigger path * adjust node mem calculation leave enough memory for the system. increase swap with min size * use shell script for builds * add env CI * use new action for release build * adjusted min sizes * remove obsolete code * update workflow name * move update dependencies entirely to rebuild script. update docs * move existing build folder to backup. update docs * increase swap with lower step size. minor fixes * refactor vars and logging. added verbose param. * update logging * fix indentation * update logging. moved NODEMEM check * refactor webapp build during installation removed unneccessay vars (only use ENABLE_WEBAPP_PROD_DOWNLOAD). trimmed webapp build option. updated messages. harmonized var access and queries * Update docs * Update messages * harmonize webapp wording * fix flake8 * fix flake8 * update node installation. update node version for armv6. use recommended setup from nodesource. removed update version on new installation run. * update docs * ci add platform linux/armv6 (deactivated) * reverted to https://deb.nodesource.com/ installation * update docs * align fin message * Update docs. Remove full URL references to branch * ignore stderr if node is not installed before * ignore stderr if node is not installed before * bugfix. removed additional char * add npm project config better network handling especially for armv6l devices * use performance optimized install command update warning for armv6l devices * Update docs Update installation steps. split for version. add tip from installation- Add webapp doc for developers. update docs and link to new webapp doc. eliminate duplications. merged developer-issues into new webapp doc * also raise fetch-retry-mintimeout * update docs harmonize wording for webapp. add build output. change ARMv6 installation note. * add note for webserver restart * update installation docs. add bullet list * change npm config values * show "slow hardware message" at installation start * update armv6l warning for webapp rebuild * make script return value more clear * Make local webapp build an optional step due to unpredictable npm connection errors. Show webapp fin message only if webapp build fails * use prebuilt webapp bundle also for develop * update docs * Update webapp.md * Update customize_options.sh (typos) * Update installation.md * Update system.md * Update installation.md * Update webapp.md * Update customize_options.sh * change npm config values * update docs * update welcome to match changes in wiki * update rebuild message * Add doc links * update ignorefiles for build.bak * Update doc and link for constributors * Update CONTRIBUTING.md --------- Co-authored-by: pabera <[email protected]>
MiczFlor · Jan 15, 2024 · 6e9e4f7 · 6e9e4f7
1 parent 03e64ff
commit 6e9e4f7
Show file tree

Hide file tree

Showing 41 changed files with 585 additions and 367 deletions.
diff --git a/.dockerignore b/.dockerignore
@@ -14,3 +14,4 @@ shared
 src/webapp/node_modules
 src/webapp/npm-debug.log
 src/webapp/build
+src/webapp/build.bak
diff --git a/.githooks/post-merge b/.githooks/post-merge
@@ -5,7 +5,7 @@
 # TO ACTIVATE: cp .githooks/post-merge .git/hooks/.
 #
 # Checks:
-# - Changes to web app
+# - Changes to Web App
 # - Changes to web dependency
 # - Changes to python requirements
 #
@@ -20,7 +20,7 @@ warn_npm_dependency() {
   echo "************************************************************"
   echo "ATTENTION: npm dependencies have changed since last pull!"
   echo ""
-  echo "To update dependencies and rebuilt WebApp run:"
+  echo "To update dependencies and rebuilt Web App run:"
   echo "$ cd src/webapp && ./run_rebuild.sh -u"
   echo "************************************************************"
   echo -e "\n"
@@ -31,7 +31,7 @@ warn_webapp() {
   echo "************************************************************"
   echo "ATTENTION: Web App sources have changed since last pull!"
   echo ""
-  echo "To rebuilt the WebApp run:"
+  echo "To rebuilt the Web App run:"
   echo "$ cd src/webapp && ./run_rebuild.sh"
   echo "************************************************************"
   echo -e "\n"

diff --git a/.github/actions/build-webapp/action.yml b/.github/actions/build-webapp/action.yml
@@ -0,0 +1,25 @@
+name: Build Web App
+description: 'Build Web App with Node'
+inputs:
+  webapp-root-path:
+    description: 'root path of the Web App sources'
+    required: false
+    default: './src/webapp'
+outputs:
+  webapp-root-path:
+    description: 'used root path of the Web App sources'
+    value: ${{ inputs.webapp-root-path }}
+
+runs:
+  using: "composite"
+  steps:
+    - name: Setup Node.js 20.x
+      uses: actions/setup-node@v3
+      with:
+        node-version: 20.x
+    - name: run build
+      working-directory: ${{ inputs.webapp-root-path }}
+      shell: bash
+      env:
+        CI: false
+      run: ./run_rebuild.sh -u
diff --git a/.github/workflows/bundle_webapp_and_release_v3.yml b/.github/workflows/bundle_webapp_and_release_v3.yml
@@ -1,4 +1,4 @@
-name: Bundle Webapp and Release
+name: Bundle Web App and Release
 
 on:
   push:
@@ -18,7 +18,7 @@ jobs:
       check_abort: ${{ steps.vars.outputs.check_abort }}
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set Output vars
         id: vars
@@ -72,9 +72,6 @@ jobs:
     if: ${{ needs.check.outputs.check_abort == 'false' }}
     runs-on: ubuntu-latest
 
-    env:
-      WEBAPP_ROOT_PATH: ./src/webapp
-
     outputs:
       tag_name: ${{ needs.check.outputs.tag_name }}
       release_type: ${{ needs.check.outputs.release_type }}
@@ -83,7 +80,7 @@ jobs:
       webapp_bundle_name_latest: ${{ steps.vars.outputs.webapp_bundle_name_latest }}
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set Output vars
         id: vars
@@ -94,29 +91,20 @@ jobs:
           echo "webapp_bundle_name=webapp-build-${COMMIT_SHA:0:10}.tar.gz" >> $GITHUB_OUTPUT
           echo "webapp_bundle_name_latest=webapp-build-latest.tar.gz" >> $GITHUB_OUTPUT
 
-      - name: Setup Node.js 20.x
-        uses: actions/setup-node@v3
-        with:
-          node-version: 20.x
-      - name: npm install
-        working-directory: ${{ env.WEBAPP_ROOT_PATH }}
-        run: npm install
-      - name: npm build
-        working-directory: ${{ env.WEBAPP_ROOT_PATH }}
-        env:
-          CI: false
-        run: npm run build
+      - name: Build Web App
+        id: build-webapp
+        uses: ./.github/actions/build-webapp
 
       - name: Create Bundle
-        working-directory: ${{ env.WEBAPP_ROOT_PATH }}
+        working-directory: ${{ steps.build-webapp.outputs.webapp-root-path }}
         run: |
           tar -czvf ${{ steps.vars.outputs.webapp_bundle_name }} build
 
       - name: Artifact Upload
         uses: actions/upload-artifact@v3
         with:
           name: ${{ steps.vars.outputs.webapp_bundle_name }}
-          path: ${{ env.WEBAPP_ROOT_PATH }}/${{ steps.vars.outputs.webapp_bundle_name }}
+          path: ${{ steps.build-webapp.outputs.webapp-root-path }}/${{ steps.vars.outputs.webapp_bundle_name }}
           retention-days: 5
 
   release:

diff --git a/.github/workflows/test_build_webapp_v3.yml b/.github/workflows/test_build_webapp_v3.yml
@@ -0,0 +1,33 @@
+name: Test Build Web App v3
+
+on:
+  schedule:
+    # run at 18:00 every sunday
+    - cron: '0 18 * * 0'
+  push:
+    branches:
+        - 'future3/**'
+    paths:
+        - '.github/workflows/test_build_webapp_v3.yml'
+        - '.github/actions/build-webapp/**'
+        - 'src/webapp/**'
+  pull_request:
+    # The branches below must be a subset of the branches above
+    branches:
+        - future3/develop
+        - future3/main
+    paths:
+        - '.github/workflows/test_build_webapp_v3.yml'
+        - '.github/actions/build-webapp/**'
+        - 'src/webapp/**'
+
+jobs:
+
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Build Web App
+        uses: ./.github/actions/build-webapp
diff --git a/.github/workflows/test_docker_debian_codename_sub_v3.yml b/.github/workflows/test_docker_debian_codename_sub_v3.yml
@@ -1,4 +1,4 @@
-name: Subworkflow Test Install Scripts Debian V3
+name: Subworkflow Test Install Scripts Debian v3
 
 on:
     workflow_call:
@@ -46,6 +46,7 @@ jobs:
         cache_key: ${{ steps.vars.outputs.cache_key }}
         image_file_name: ${{ steps.vars.outputs.image_file_name }}
         image_tag_name: ${{ steps.vars.outputs.image_tag_name }}
+        docker_run_options: ${{ steps.vars.outputs.docker_run_options }}
 
     # create local docker registry to use locally build images
     services:
@@ -83,13 +84,17 @@ jobs:
       id: vars
       env:
         LOCAL_REGISTRY_PORT: ${{ inputs.local_registry_port }}
+        PLATFORM: ${{ inputs.platform }}
       run: |
         echo "image_tag_name=${{ steps.pre-vars.outputs.image_tag_name }}" >> $GITHUB_OUTPUT
         echo "image_tag_name_local_base=localhost:${{ env.LOCAL_REGISTRY_PORT }}/${{ steps.pre-vars.outputs.image_tag_name }}-base" >> $GITHUB_OUTPUT
         echo "image_file_name=${{ steps.pre-vars.outputs.image_file_name }}" >> $GITHUB_OUTPUT
         echo "image_file_path=./${{ steps.pre-vars.outputs.image_file_name }}" >> $GITHUB_OUTPUT
         echo "cache_scope=${{ steps.pre-vars.outputs.cache_scope }}" >> $GITHUB_OUTPUT
         echo "cache_key=${{ steps.pre-vars.outputs.cache_scope }}-${{ github.sha }}#${{ github.run_attempt }}" >> $GITHUB_OUTPUT
+        if [ "${{ env.PLATFORM }}" == "linux/arm/v6" ] ; then
+            echo "docker_run_options=-e QEMU_CPU=arm1176" >> $GITHUB_OUTPUT
+        fi
 
     # Build base image for debian version name. Layers will be cached and image pushes to local registry
     - name: Build Image - Base
@@ -167,7 +172,7 @@ jobs:
       uses: tj-actions/docker-run@v2
       with:
         image: ${{ needs.build.outputs.image_tag_name }}
-        options: --platform ${{ inputs.platform }} --user ${{ env.TEST_USER_NAME }} --init
+        options: ${{ needs.build.outputs.docker_run_options }} --platform ${{ inputs.platform }} --user ${{ env.TEST_USER_NAME }} --init
         name: ${{ matrix.test_script }}
         args: |
           ./${{ matrix.test_script }}

diff --git a/.github/workflows/test_docker_debian_v3.yml b/.github/workflows/test_docker_debian_v3.yml
@@ -45,9 +45,25 @@ jobs:
       debian_codename: 'bookworm'
       platform: linux/arm/v7
 
+  # # can be activate on test branches, currently failing
+  # run_bookworm_armv6:
+  #   name: 'bookworm armv6'
+  #   uses: ./.github/workflows/test_docker_debian_codename_sub_v3.yml
+  #   with:
+  #       debian_codename: 'bookworm'
+  #       platform: linux/arm/v6
+
   run_bullseye_armv7:
     name: 'bullseye armv7'
     uses: ./.github/workflows/test_docker_debian_codename_sub_v3.yml
     with:
       debian_codename: 'bullseye'
       platform: linux/arm/v7
+
+  # # can be activate on test branches, currently failing
+  # run_bullseye_armv6:
+  #   name: 'bullseye armv6'
+  #   uses: ./.github/workflows/test_docker_debian_codename_sub_v3.yml
+  #   with:
+  #       debian_codename: 'bullseye'
+  #       platform: linux/arm/v6
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -45,12 +45,17 @@ as local, temporary scratch areas.
 
 Contributors have played a bigger role over time to keep Phoniebox on the edge of innovation :)
 
-We want to keep it as easy as possible to contribute changes that get things working in your environment.
-There are a few guidelines that we need contributors to follow so that we can have a chance of keeping on top of things.
+Our goal is to make it simple for you to contribute changes that improve functionality in your specific environment. 
+To achieve this, we have a set of guidelines that we kindly request contributors to adhere to. 
+These guidelines help us maintain a streamlined process and stay on top of incoming contributions.
 
-Development for Version 3 is done on the git branch `future3/develop`. How to move to that branch, see below.
+To report bug fixes and improvements, please follow the steps outlined below:
+1. For bug fixes and minor improvements, simply open a new issue or pull request (PR).
+2. If you intend to port a feature from Version 2.x to future3 or wish to implement a new feature, we recommend reaching out to us beforehand.
+   - In such cases, please create an issue outlining your plans and intentions.
+   - We will ensure that there are no ongoing efforts on the same topic.
 
-For bug fixes and improvements just open an issue or PR as described below. If you plan to port a feature from Version 2.X or implement a new feature, it is advisable to contact us first. In this case, also open an issue describing what you are planning to do. We will just check that nobody else is already on the subject. We are looking forward to your work. Check the current [feature list](https://rpi-jukebox-rfid.readthedocs.io/en/latest/featurelist.html) for available features and work in progress.
+We eagerly await your contributions! You can review the current [feature list](documentation/developers/status.md) to check for available features and ongoing work.
 
 ## Getting Started
 
@@ -60,31 +65,21 @@ For bug fixes and improvements just open an issue or PR as described below. If y
     Version 2 will continue to live for quite a while.
   * Clearly describe the issue including steps to reproduce when it is a bug
   * Make sure you fill in the earliest version that you know has the issue
-* By default this will get you to the `future3/main` branch. You will move to the `future3/develop` branch, do this:
-
-~~~bash
-cd ~/RPi-Jukebox-RFID
-git checkout future3/develop
-git fetch origin
-git reset --hard origin/future3/develop
-git pull
-~~~
 
 The preferred way of code contributions are [pull requests (follow this link for a small howto)](https://www.digitalocean.com/community/tutorials/how-to-create-a-pull-request-on-github).
-And, ideally pull requests use the "running code" on the `future3/develop` branch of your Phoniebox.
+And ideally pull requests use the "running code" of your Phoniebox.
 Alternatively, feel free to post tweaks, suggestions and snippets in the ["issues" section](https://github.com/MiczFlor/RPi-Jukebox-RFID/issues).
 
 ## Making Changes
 
+* Create a fork of this repository
 * Create a topic branch from where you want to base your work.
-  * This is usually the master branch or the develop branch.
-  * Only target release branches if you are certain your fix must be on that
+  * This is usually the `future3/develop` branch.
+  * Only target the `future3/main` branch if you are certain your fix must be on that
     branch.
-  * To quickly create a topic branch based on master, run `git checkout -b
-    fix/master/my_contribution master`. Please avoid working directly on the
-    `master` branch.
 * Make commits of logical and atomic units.
 * Check for unnecessary whitespace with `git diff --check` before committing.
+* See also the [documentation for developers](documentation/developers/README.md)
 
 ## Making Trivial Changes
 
@@ -168,8 +163,8 @@ The original contributor will be notified of the revert.
 
 ## Guidelines
 
-* Phoniebox runs on Raspian **Buster**. Therefore, all Python code should work at least with **Python 3.7**.
-* For GPIO all code should work with **RPi.GPIO**. gpiozero is currently not intended to use.
+* Phoniebox runs on Raspberry Pi OS. 
+* Minimum python version is currently **Python 3.9**.
 
 ## Additional Resources
 

diff --git a/ci/installation/run_install_common.sh b/ci/installation/run_install_common.sh
@@ -24,7 +24,7 @@ export ENABLE_WEBAPP_PROD_DOWNLOAD=true
 # n - setup rfid reader
 # y - setup samba
 # y - setup webapp
-# - - install node (forced WebApp Download)
+# - - build webapp (skipped due to forced webapp Download)
 # n - setup kiosk mode
 # n - reboot
 

diff --git a/ci/installation/run_install_faststartup.sh b/ci/installation/run_install_faststartup.sh
@@ -22,7 +22,7 @@ LOCAL_INSTALL_SCRIPT_PATH="${LOCAL_INSTALL_SCRIPT_PATH%/}"
 # n - setup rfid reader
 # n - setup samba
 # n - setup webapp
-# - - install node (only with webapp = y)
+# - - build webapp (only with webapp = y)
 # - - setup kiosk mode (only with webapp = y)
 # n - reboot
 

diff --git a/ci/installation/run_install_libzmq_local.sh b/ci/installation/run_install_libzmq_local.sh
@@ -23,7 +23,7 @@ export BUILD_LIBZMQ_WITH_DRAFTS_ON_DEVICE=true
 # n - setup rfid reader
 # n - setup samba
 # n - setup webapp
-# - - install node (only with webapp = y)
+# - - build webapp (only with webapp = y)
 # - - setup kiosk mode (only with webapp = y)
 # n - reboot
 

diff --git a/ci/installation/run_install_webapp_download.sh b/ci/installation/run_install_webapp_download.sh
@@ -4,7 +4,7 @@
 # Used e.g. for tests on Docker
 
 # Objective:
-# Test for the WebApp (download) and dependent features path.
+# Test for the Web App (download) and dependent features path.
 
 SOURCE="${BASH_SOURCE[0]}"
 SCRIPT_DIR="$(dirname "$SOURCE")"
@@ -22,7 +22,7 @@ LOCAL_INSTALL_SCRIPT_PATH="${LOCAL_INSTALL_SCRIPT_PATH%/}"
 # n - setup rfid reader
 # n - setup samba
 # y - setup webapp
-# n - install node
+# n - build webapp
 # y - setup kiosk mode
 # n - reboot
 

diff --git a/ci/installation/run_install_webapp_local.sh b/ci/installation/run_install_webapp_local.sh
@@ -4,7 +4,7 @@
 # Used e.g. for tests on Docker
 
 # Objective:
-# Test for the WebApp (build locally) and dependent features path.
+# Test for the Web App (build locally) and dependent features path.
 
 SOURCE="${BASH_SOURCE[0]}"
 SCRIPT_DIR="$(dirname "$SOURCE")"
@@ -23,7 +23,7 @@ export ENABLE_WEBAPP_PROD_DOWNLOAD=false
 # n - setup rfid reader
 # n - setup samba
 # y - setup webapp
-# y - install node
+# y - build webapp
 # y - setup kiosk mode
 # n - reboot
 

diff --git a/documentation/builders/autohotspot.md b/documentation/builders/autohotspot.md
@@ -2,7 +2,7 @@
 
 The Auto-Hotspot function allows the Jukebox to switch between its
 connection between a known WiFi and an automatically generated hotspot
-so that you can still access via SSH or Webapp.
+so that you can still access via SSH or Web App.
 
 > [!IMPORTANT]
 > Please configure the WiFi connection to your home access point before enabling these feature!
@@ -17,10 +17,10 @@ hotspot named `Phoniebox_Hotspot`. You will be able to connect to this
 hotspot using the given password in the installation or the default
 password: `PlayItLoud!`
 
-### Webapp
+### Web App
 
 After connecting to the `Phoniebox_Hotspot` you are able to connect to
-the webapp accessing the website [10.0.0.5](http://10.0.0.5/).
+the Web App accessing the website [10.0.0.5](http://10.0.0.5/).
 
 ### ssh
 
@@ -69,7 +69,7 @@ ieee80211d=1
 
 ## Disabling automatism
 
-Auto-Hotspot can be enabled or disabled using the Webapp.
+Auto-Hotspot can be enabled or disabled using the Web App.
 
 > [!IMPORTANT]
 > Disabling or enabling will keep the last state.