Merge pull request #62 from HEnquist/next30

Next30

.github/workflows/build.yml

@@ -9,7 +9,7 @@ jobs:
     outputs:
       fe_tag: ${{ steps.fe_tag_step.outputs.fe_tag }}
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Get CamillaGUI tag from versions.yml
       id: fe_tag_step
       run: |
@@ -21,19 +21,19 @@ jobs:
     runs-on: ubuntu-latest
     needs: read_fe_tag
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
       name: Check out frontend ${{ needs.read_fe_tag.outputs.fe_tag }}
       with:
         repository: HEnquist/camillagui
         ref: ${{ needs.read_fe_tag.outputs.fe_tag }}
     - name: Build and publish
-      uses: actions/setup-node@v3
+      uses: actions/setup-node@v4
       with:
-        node-version: '16'
+        node-version: '20'
     - run: npm install
     - run: npm run build
     - name: Upload build
-      uses: actions/upload-artifact@v3
+      uses: actions/upload-artifact@v4
       with:
         name: build
         path: build
@@ -42,9 +42,9 @@ jobs:
     runs-on: ubuntu-latest
     needs: build_fe
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Setup Python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: '3.12'
 
@@ -71,13 +71,13 @@ jobs:
         rm -rf tests
 
     - name: Download frontend
-      uses: actions/download-artifact@v3
+      uses: actions/download-artifact@v4
 
     - name: Create zip
       run: zip -r camillagui.zip *
 
     - name: Upload all as artifact
-      uses: actions/upload-artifact@v3
+      uses: actions/upload-artifact@v4
       with:
         name: camillagui-backend
         path: |

.gitignore

@@ -1,2 +1,3 @@
 *.pyc
 build/*
+.venv

README.md

@@ -2,7 +2,7 @@
 
 This is the server part of CamillaGUI, a web-based GUI for CamillaDSP.
 
-This version works with CamillaDSP 2.0.x.
+This version works with CamillaDSP 3.0.x.
 
 The complete GUI is made up of two parts:
 - a frontend based on React: https://reactjs.org/
@@ -35,7 +35,7 @@ for more information.
 
 ## Configuration
 
-The backend configuration is stored in `config/camillagui.yml`
+The backend configuration is stored in `config/camillagui.yml` by default.
 
 ```yaml
 ---
@@ -45,6 +45,7 @@ bind_address: "0.0.0.0"
 port: 5005
 ssl_certificate: null (*)
 ssl_private_key: null (*)
+gui_config_file: null (*)
 config_dir: "~/camilladsp/configs"
 coeff_dir: "~/camilladsp/coeffs"
 default_config: "~/camilladsp/default_config.yml"
@@ -62,6 +63,8 @@ The web interface will be served on port 5005 using plain HTTP.
 It is possible to run the gui and CamillaDSP on different machines,
 just point the `camilla_host` to the right address.
 
+The optional `gui_config_file` can be used to override the default path to the gui config file.
+
 **Warning**: By default the backend will bind to all network interfaces.
 This makes the gui available on all networks the system is connected to, which may be insecure.
 Make sure to change the `bind_address` if you want it to be reachable only on specific
@@ -149,32 +152,51 @@ The styling can be customized by editing `build/css-variables.css`.
 
 ### Adding custom shortcut settings
 It is possible to configure custom shortcuts for the `Shortcuts` section and the compact view.
-The included config file contains the default Bass and Treble filters.
+The included config file contains the default Bass and Treble filters,
+as well as a few commented out examples.
+
 To add more, edit the file `config/gui-config.yml` to add
 the new shortcuts to the list under `custom_shortcuts`.
 
-Here is an example config to set the gain of a filter called `MyFilter`
-within the range from 0 to 10 db in steps of 0.1 dB.
+Here is an example config to set the gain of the filters called `MyFilter` and `MyOtherFilter`.
+within the range from -10 to 0 db in steps of 0.1 dB.
+For `MyOtherFilter`, the scale is reversed, such that moving the slider from -10 to -9 dB
+changes the gain of `MyOtherFilter` fom 0 to -1 dB.
+The `type` property is set to `number`.
+This creates a slider control, used to control numerical values.
+It can also be set to `boolean` which creates a checkbox.
+For `number`, the `range_from`, `range_to` and `step` properties are required.
+They are not used by `boolean` controls and may be left out.
+
 ```yaml
 custom_shortcuts:
   - section: "My custom section"
-    description: "Optional description for the section. Omit the attribute, if unwanted"
+    description: |
+      Optional description for the section.
+      Omit this attribute, if unwanted.
+      The text will be shown in the gui with line breaks.
     shortcuts:
       - name: "My filter gain"
-        description: "Optional description for the setting. Omit the attribute, if unwanted"
-        path_in_config: ["filters", "MyFilter", "parameters", "gain"]
-        range_from: 0
-        range_to: 10
+        description: |
+          Optional description for the setting.
+          Omit this attribute, if unwanted.
+        config_elements:
+          - path: ["filters", "MyFilter", "parameters", "gain"]
+            reverse: false
+          - path: ["filters", "MyOtherFilter", "parameters", "gain"]
+            reverse: true
+        range_from: -10
+        range_to: 0
         step: 0.1
-      - name: "The next setting"
-        ...
+        type: "number"
 ```
+When letting a shortcut control more than one element in the config,
+the first one is considered the main one, that controls the slider position.
+The first element must be present in the config in order for the shortcut to function.
 
-The gui config is checked when the backend starts, and any problems are logged.
-For example, `range_from` must be a number. If it is not, this results in a message such as this:
-```
-ERROR:root:Parameter 'custom_shortcuts/0/shortcuts/1/range_from': 'hello' is not of type 'number'
-```
+If any of the others is not at the expected value, the GUI will show a warning.
+The same happens if any of the others is missing in the config.
+The control can then still be used, but may not give the wanted result.
 
 ### Hiding GUI Options
 Options can be hidden from your users by editing `config/gui-config.yml`.
@@ -186,6 +208,7 @@ hide_silence: false
 hide_capture_device: false
 hide_playback_device: false
 hide_rate_monitoring: false
+hide_multithreading: false
 ```
 
 ### Styling the GUI
@@ -199,6 +222,14 @@ To enable it by default, in `config/gui-config.yml` set `apply_config_automatica
 The update rate of the level meters can be adjusted by changing the `status_update_interval` setting.
 The value is in milliseconds, and the default value is 100 ms.
 
+### Gui config syntax check
+The gui config is checked when the backend starts, and any problems are logged.
+For example, the `range_from` property of a config shortcut must be a number.
+If it is not, this results in a message such as this:
+```
+ERROR:root:Parameter 'custom_shortcuts/0/shortcuts/1/range_from': 'hello' is not of type 'number'
+```
+
 ## Running
 Start the server with:
 ```sh
@@ -210,6 +241,31 @@ The gui should now be available at: http://localhost:5005/gui/index.html
 If accessing the gui from a different machine, replace "localhost" by the IP
 or hostname of the machine running the gui server.
 
+### Command line options
+The logging level for the backend itself as well as the AIOHTTP framework are set to `WARNING` by default.
+These can both be changed with command line arguments, which may be useful when debugging some problem.
+
+The backend norally reads its configuration from a default location.
+This can be changed by providing a different path as a command line argument.
+
+Use the `-h` or `--help` argument to view the built-in help:
+```
+> python main.py --help
+usage: python main.py [-h] [-c CONFIG] [-l {CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET}]
+                      [-a {CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET}]
+
+Backend for the CamillaDSP web GUI
+
+options:
+  -h, --help            show this help message and exit
+  -c CONFIG, --config CONFIG
+                        Provide a path to a backend config file to use instead of the default
+  -l {CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET}, --log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET}
+                        Logging level
+  -a {CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET}, --aiohttp-log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET}
+                        AIOHTTP logging level
+```
+
 
 ## Development
 ### Render the environment files

backend/convolver_config_import.py

@@ -39,10 +39,10 @@ def channels_factors_and_inversions_as_list(
     ]
 
 
-def make_filter_step(channel: int, names: List[str]) -> dict:
+def make_filter_step(channels: List[int], names: List[str]) -> dict:
     return {
         "type": "Filter",
-        "channel": channel,
+        "channels": channels,
         "names": names,
         "bypassed": None,
         "description": None,
@@ -157,7 +157,7 @@ def _input_delay_pipeline_steps(self) -> List[dict]:
 
     def _delay_pipeline_steps(self, delays: List[int]) -> List[dict]:
         return [
-            make_filter_step(channel, [self._delay_name(delay)])
+            make_filter_step([channel], [self._delay_name(delay)])
             for channel, delay in enumerate(delays)
             if delay != 0
         ]
@@ -210,4 +210,4 @@ def _mixer_out_pipeline_step() -> List[dict]:
         return [{"type": "Mixer", "name": "Mixer out", "description": None}]
 
     def _filter_pipeline_steps(self) -> List[dict]:
-        return [make_filter_step(f.channel, [f.name()]) for f in self._filters]
+        return [make_filter_step([f.channel], [f.name()]) for f in self._filters]

backend/eqapo_config_import.py

@@ -292,21 +292,6 @@ def postprocess(self):
             for idx, dest in enumerate(list(mixer["mapping"])):
                 if len(dest["sources"]) == 0:
                     mixer["mapping"].pop(idx)
-        # Expand filter steps to all channels
-        pipeline = []
-        for step in self.pipeline:
-            if step["type"] != "Filter":
-                pipeline.append(step)
-            else:
-                channels = step["channels"]
-                if channels is None:
-                    channels = range(self.nbr_channels)
-                for channel in channels:
-                    new_step = deepcopy(step)
-                    new_step["channel"] = channel
-                    del new_step["channels"]
-                    pipeline.append(new_step)
-        self.pipeline = pipeline
 
     def build_config(self):
         config = {

backend/filemanagement.py

@@ -12,6 +12,7 @@
     isabs,
     commonpath,
     getmtime,
+    getsize,
 )
 import logging
 import traceback
@@ -22,6 +23,8 @@
 
 from camilladsp import CamillaError
 
+from .legacy_config_import import identify_version
+
 DEFAULT_STATEFILE = {
     "config_path": None,
     "mute": [False, False, False, False, False],
@@ -62,28 +65,72 @@ async def store_files(folder, request):
     return web.Response(text="Saved {} file(s)".format(i))
 
 
-def list_of_files_in_directory(folder):
+def list_of_files_in_directory(folder, file_stats=True, title_and_desc=False, validator=None):
     """
     Return a list of files (name and modification date) in a folder.
     """
-    files = [
-        file_in_folder(folder, file)
-        for file in os.listdir(folder)
-        if isfile(file_in_folder(folder, file))
-    ]
-    files_list = map(
-        lambda file: {
-            "name": (os.path.basename(file)),
-            "lastModified": (getmtime(file)),
-        },
-        files,
-    )
+
+    files_list = []
+    for file in os.listdir(folder):
+        filepath = file_in_folder(folder, file)
+        if not isfile(filepath) or file.startswith("."):
+            # skip directories and hidden files
+            continue
+
+        file_data = {
+            "name": file,
+        }
+        if file_stats:
+            file_data["lastModified"] = getmtime(filepath)
+            file_data["size"] = getsize(filepath)
+
+        if title_and_desc:
+            valid = False
+            version = None
+            errors = None
+            title = None
+            desc = None
+            with open(filepath) as f:
+                try:
+                    parsed = yaml.safe_load(f)
+                    title = parsed.get("title")
+                    desc = parsed.get("description")
+                    version = identify_version(parsed)
+                    if version == 3 and validator is not None:
+                        parsed_abs = make_config_filter_paths_absolute(parsed, folder)
+                        validator.validate_config(parsed_abs)
+                        error_list = validator.get_errors()
+                        if len(error_list) > 0:
+                            errors = error_list
+                        else:
+                            valid = True
+                    elif version < 3:
+                        valid = False
+                        errors = [([], f"This config is made for the previous version {version} of CamillaDSP.")]
+                except yaml.YAMLError as e:
+                    if hasattr(e, 'problem_mark'):
+                        mark = e.problem_mark
+                        errordesc = f"This file has a YAML syntax error on line: {mark.line + 1}, column: {mark.column + 1}"
+                    else:
+                        errordesc = "This config file has a YAML syntax error."
+                    errors = [([], errordesc)]
+                except (AttributeError, UnicodeDecodeError) as e:
+                    errors = [([], "This does not appear to be a YAML file.")]
+                except Exception as e:
+                    errors = [([], f"Error: {e}")]
+            file_data["title"] = title
+            file_data["description"] = desc
+            file_data["version"] = version
+            file_data["valid"] = valid
+            file_data["errors"] = errors
+        files_list.append(file_data)
+
     sorted_files = sorted(files_list, key=lambda x: x["name"].lower())
     return sorted_files
 
 
 def list_of_filenames_in_directory(folder):
-    return map(lambda file: file["name"], list_of_files_in_directory(folder))
+    return [file["name"] for file in list_of_files_in_directory(folder, file_stats=False)]
 
 
 def delete_files(folder, files):