Merge pull request #671 from int-brain-lab/iblrigv8dev

8.19.3

.github/workflows/main.yaml

@@ -53,9 +53,7 @@ jobs:
         run: sudo apt-get install -y libportaudio2
 
       - name: iblrig unit tests
-        run: |
-          coverage run -m unittest discover -s ./iblrig/test -t .
-          coverage xml
+        run: pytest
 
       - name: coveralls.io
         uses: coverallsapp/github-action@v2

CHANGELOG.md

@@ -2,6 +2,12 @@ Changelog
 ---------
 
 
+8.19.3
+------
+* hotfix: force stimulus to freeze in center of screen during "freeze_reward" state
+* method for copying snapshots to the local server using the SessionCopier
+* documentation for transition to Bpod HiFi
+
 8.19.2
 ------
 * hotfix: only register water administrations once per protocol

docs/source/faq.rst

@@ -80,3 +80,25 @@ Frame2TTL
 
 *  Version 1 of Frame2TTL won't be detected after restarting the computer.
    Unplugging and replugging the USB cable should make it responsive again.
+*  If IBLRIG complains about not receiving any TTL signals from Frame2TTL:
+
+   *  Ensure Frame2TTL's sensor is positioned over the bottom-right corner of the rig's screen.
+      Secure the sensor's cable to the screen mount with a zip-tie to prevent it from slipping off the screen.
+      Additionally, use a piece of electrical tape to hold the sensor in place.
+
+   *  Verify that the sensor is connected to Frame2TTL with the correct polarity
+
+      *  Version 1: GND = black cable, SIG = white cable
+      *  Version 2 and 3: BLK = black cable, WHT = white cable
+   *  Ensure that Frame2TTL's TTL Output is plugged into Bpod's TTL Input #1.
+      Note that versions 2 and 3 of Frame2TTL have a second BNC output labeled "analog" - this is *not* the TTL output.
+   *  Recalibrate Frame2TTL using the calibration routine in IBLRIG's Tools menu and check for any errors.
+
+*  If the above steps do not resolve the issue, try the following:
+
+   #. Swap out the BNC cable between Frame2TTL and Bpod.
+      Use a single cable without any branches.
+   #. Connect an oscilloscope to the Bpod end of the cable and run a calibration.
+      Look for a voltage step in Frame2TTL's output when the calibration routine switches from dark to light.
+   #. If you *do* see the change in the TTL signal, the Bpod might be faulty. Try using a different Bpod unit.
+   #. If you do *not* see the voltage step, the Frame2TTL might be faulty. Try using a different Frame2TTL unit.

docs/source/hardware.rst

@@ -0,0 +1,94 @@
+Hardware Guide
+==============
+
+
+Upgrading Xonar AE to Bpod HiFi Module
+------------------------------------------------
+
+.. note:: The following guide only concerns behavior rigs using `Asus Xonar AE` sound cards.
+   The `Harp` sound cards used in our Ephys and Mesoscope rigs work fine and do not require replacing!
+
+Background
+""""""""""
+
+In their original design, IBL’s behavioral training rigs relied on a consumer-grade sound-card - the `ASUS Xonar AE <https://www.asus.com/motherboards-components/sound-cards/gaming/xonar-ae/>`_ - for delivering auditory cues to the subject. We have identified the Xonar AE as a weak point in the design of the training rig as this choice of hardware has several severe drawbacks:
+
+*  Synchronization between individual components of the rig relies on TTL Signals, a standardized way of communicating between digital devices.
+   With the Xonar sound-card, we emulate TTL pulses on one of the soundcard’s analog audio outputs.
+   The resulting signal does not conform well to the TTL standard since (a) we cannot guarantee the required signal rise-times, and (b) pulse-amplitude depends on the computer’s audio volume and impedance settings. The effects are increased latencies and accidental misconfigurations.
+   The latter have the potential to cause synchronization issues and more problems further down the pipeline.
+*  The ASUS Xonar AE does not seem to be a particularly robust piece of equipment - we’ve experienced several catastrophic failures across our rigs at IBL. Replacement hardware is increasingly hard to come by as the Asus Xonar AE is not being produced anymore.
+*  The soundcard’s driver has been causing issues with specific Windows updates that require manual intervention / rollback of the respective updates.
+*  There is no Windows 11 specific driver available.
+
+For these reasons, we decided to replace the Xonar AE with the `Bpod HiFi Module HD <https://sanworks.io/shop/viewproduct?productID=1033>`_ by Sanworks:
+The HiFi Module is capable of delivering high fidelity sound stimuli at low latencies, supports proper TTL signaling as well as serial communication with the Bpod Finite State Machine and does not require any dedicated drivers.
+The upgrade procedure is straightforward and should take no longer than 5-10 minutes.
+
+Requirements
+""""""""""""
+
+To replace your existing Xonar AE with the Bpod HiFi Module HD you will require the following:
+
+*  1x Bpod HiFi Module HD
+*  1x Micro-USB to USB-A cable
+*  1x Ethernet cable (RJ45)
+*  1x RCA to 3.5 mm audio adapter
+*  1x 3 mm flathead screwdriver
+
+.. warning:: Support for the Bpod HiFi Module was introduced with IBLRIG 8.16.0 and will not be backported to older version of IBLRIG.
+             Make sure to upgrade to the latest version of IBLRIG before continuing with this guide.
+
+Upgrade Procedure
+"""""""""""""""""
+
+1. Use the 3 mm flathead screwdriver to unscrew the BNC to wire adapter from the amplifier board.
+   While the adapter itself is not needed anymore we will continue using the BNC cable.
+   Leave the other end of the BNC cable plugged into the Bpod as it is.
+
+   .. figure:: img/amp2x15_labels.png
+      :width: 100%
+      :class: with-border
+
+      Disconnect the BNC to wire adapter from the amplifier board.
+
+2. Unplug the 3.5 mm audio cable from the Xonar AE sound card on the backside of the rig's computer.
+   Leave the other end of the 3.5 mm audio cable connected to the amplifier board.
+
+   .. figure:: img/xonar_labels.png
+      :width: 100%
+      :class: with-border
+
+      Unplug the 3.5 mm audio cable from the Xonar AE sound card.
+
+3. Connect the Bpod HiFi Module as follows:
+
+   * the BNC cable connects to TTL In 2 of the Bpod (cf. step 1),
+   * the 3.5 mm audio cable connects to the amplifier board via the RCA adapter (cf. step 2),
+   * the USB cable connects to the rig's computer
+   * the Ethernet cable connects to one of the Bpod's Module ports.
+     Warning: Bpod uses identical connectors for its Behavior ports - do not mix them up!
+
+   .. figure:: img/hifi_labels.png
+      :width: 100%
+      :class: with-border
+
+      The Bpod HiFi Modules and its connections.
+
+4. Open `C:/iblrigv8/settings/hardware_settings.yaml` in a text-editor.
+   Find the section `device_sound` and adapt it as follows:
+
+   .. code-block:: yaml
+
+      device_sound:
+        OUTPUT: hifi
+        COM_SOUND: COMx  # replace with the HiFi Module's actual COM port!
+        AMP_TYPE: AMP2X15
+
+   .. tip::
+
+      Use Windows' device manager to identify the HiFi Module's COM port.
+      The device should show up in the section labelled "Ports (COM & LPT)" after plugging it in.
+
+5. Start IBLRIG and make sure that the hardware validation during start-up does not find any issues.
+   Finally, start a session and verify that you can hear the audio cues.

docs/source/index.rst

@@ -13,4 +13,6 @@ IBLRIG documentation
    installation
    usage
    reference
+   hardware
+   reference_developer_guide
    faq

docs/source/reference_developer_guide.rst

@@ -1,15 +1,6 @@
 Developer Guide
 ===============
 
-Release Checklist
------------------
-
-1) update CHANGELOG.md including changes from the last tag
-2) Pull request to ``iblrigv8dev``
-3) Check CI and eventually wet lab test
-4) Pull request to ``iblrigv8``
-5) Merge PR
-6) git tag the release in accordance to the version number below (after merge!)
 
 Versioning Scheme
 -----------------
@@ -44,14 +35,67 @@ Here,
 Both of these fields are inferred by means of git describe and do not require manual interaction from the developer.
 
 
-Running Tests Locally
----------------------
+Installing Developer Dependencies
+---------------------------------
+
+To install additional dependencies needed for working on IBLRIG's code-base, run the following within the venv:
+
+.. code-block:: console
+
+   pip install -U -e .[DEV]
+
+
+Running Unit Tests Locally
+--------------------------
+
+To run unit tests locally, run the following within IBLRIG's venv:
+
+.. code-block:: console
+
+   pytest
+
+This will also generate a coverage report which can be found in the ``htmlcov`` directory.
+
+
+Linting & Formatting
+--------------------
+
+To lint your code, run the following within IBLRIG's venv:
 
 .. code-block:: console
 
    ruff check .
+
+Adding the commandline flag ``--fix`` will automatically fix issues that are deemed safe to handle:
+
+.. code-block:: console
+
+   ruff check . --fix
+
+To *check* if your code conforms to the `Black code style <https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html>`_, run:
+
+.. code-block:: console
+
+   ruff format . --check
+
+To reformat your code according to the `Black code style <https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html>`_, run:
+
+.. code-block:: console
+
    ruff format .
-   python -m unittest discover ./iblrig/test
+
+Refer to `Ruff Formater's documentation <https://docs.astral.sh/ruff/formatter/>`_ for further details.
+
+
+Release Checklist
+-----------------
+
+1) update CHANGELOG.md including changes from the last tag
+2) Pull request to ``iblrigv8dev``
+3) Check CI and eventually wet lab test
+4) Pull request to ``iblrigv8``
+5) Merge PR
+6) git tag the release in accordance to the version number below (after merge!)
 
 
 Building the documentation
@@ -79,7 +123,7 @@ To write the documentation:
 * If you are writing in a new file, add it to the ``index.rst`` so it appears in the table of content
 * Push all your changes to the ``iblrigv8dev`` branch ; if this branch does not exist, create it first
 
-To release the documentation onto the `website <https://int-brain-lab.github.io/iblrig>`__:
+To release the documentation onto the `website <https://int-brain-lab.github.io/iblrig>`_:
 
 * Wait for the next release, or
-* Manually trigger the GitHub action by clicking "Run Workflow" (select ``master``) here: https://github.com/int-brain-lab/iblrig/actions/workflows/docs.yaml
+* Manually trigger the GitHub action by clicking "Run Workflow" (select ``master``) `here <https://github.com/int-brain-lab/iblrig/actions/workflows/docs.yaml>`_

iblrig/__init__.py

@@ -6,7 +6,7 @@
 # 5) git tag the release in accordance to the version number below (after merge!)
 # >>> git tag 8.15.6
 # >>> git push origin --tags
-__version__ = '8.19.2'
+__version__ = '8.19.3'
 
 
 from iblrig.version_management import get_detailed_version_string

iblrig/base_choice_world.py

@@ -38,7 +38,7 @@ class ChoiceWorldParams(BaseModel):
     ADAPTIVE_REWARD: bool = False
     BONSAI_EDITOR: bool = False
     CALIBRATION_VALUE: float = 0.067
-    CONTRAST_SET: list[Probability] = Field([1.0, 0.25, 0.125, 0.0625, 0.0], min_items=1)
+    CONTRAST_SET: list[Probability] = Field([1.0, 0.25, 0.125, 0.0625, 0.0], min_length=1)
     CONTRAST_SET_PROBABILITY_TYPE: Literal['uniform', 'skew_zero'] = 'uniform'
     GO_TONE_AMPLITUDE: float = 0.0272
     GO_TONE_DURATION: float = 0.11
@@ -389,7 +389,7 @@ def get_state_machine_trial(self, i):
         sma.add_state(
             state_name='freeze_reward',
             state_timer=0,
-            output_actions=[self.bpod.actions.bonsai_freeze_stim],
+            output_actions=[self.bpod.actions.bonsai_show_center],
             state_change_conditions={'Tup': 'reward'},
         )
 

iblrig/base_tasks.py

@@ -680,7 +680,7 @@ def trigger_bonsai_cameras(self):
         workflow_file = self._camera_mixin_bonsai_get_workflow_file(configuration, 'recording')
         if workflow_file is None:
             return
-        iblrig.path_helper.create_bonsai_layout_from_template(workflow_file)  # FIXME What does this do?
+        iblrig.path_helper.create_bonsai_layout_from_template(workflow_file)
         # FIXME Use parameters in configuration map
         parameters = {
             'FileNameLeft': self.paths.SESSION_FOLDER.joinpath('raw_video_data', '_iblrig_leftCamera.raw.avi'),

iblrig/path_helper.py

@@ -345,6 +345,31 @@ def iterate_collection(session_path: str, collection_name='raw_task_data') -> st
 
 
 def create_bonsai_layout_from_template(workflow_file: Path) -> None:
+    """
+    Create a Bonsai layout file from a template if it does not already exist.
+
+    If the file with the suffix `.bonsai.layout` does not exist for the given
+    workflow file, this function will attempt to create it from a template
+    file with the suffix `.bonsai.layout_template`. If the template file also
+    does not exist, the function logs that no template layout is available.
+
+    Background: Bonsai stores dialog settings (window position, control
+    visibility, etc.) in an XML file with the suffix `.bonsai.layout`. These
+    layout files are user-specific and may be overwritten locally by the user
+    according to their preferences. To ensure that a default layout is
+    available, a template file with the suffix `.bonsai.layout_template` can
+    be provided as a starting point.
+
+    Parameters
+    ----------
+    workflow_file : Path
+        The path to the Bonsai workflow for which the layout is to be created.
+
+    Raises
+    ------
+    FileNotFoundError
+        If the provided workflow_file does not exist.
+    """
     if not workflow_file.exists():
         FileNotFoundError(workflow_file)
     if not (layout_file := workflow_file.with_suffix('.bonsai.layout')).exists():

iblrig/pydantic_definitions.py

@@ -123,10 +123,10 @@ class HardwareSettingsSound(BunchModel):
 
 class HardwareSettingsValve(BunchModel):
     WATER_CALIBRATION_DATE: date
-    WATER_CALIBRATION_RANGE: list[PositiveFloat] = Field(min_items=2, max_items=2)  # type: ignore
+    WATER_CALIBRATION_RANGE: list[PositiveFloat] = Field(min_length=2, max_length=2)  # type: ignore
     WATER_CALIBRATION_N: PositiveInt = Field(ge=3, default=5)
-    WATER_CALIBRATION_OPEN_TIMES: list[PositiveFloat] = Field(min_items=2)  # type: ignore
-    WATER_CALIBRATION_WEIGHT_PERDROP: list[float] = Field(PositiveFloat, min_items=2)  # type: ignore
+    WATER_CALIBRATION_OPEN_TIMES: list[PositiveFloat] = Field(min_length=2)  # type: ignore
+    WATER_CALIBRATION_WEIGHT_PERDROP: list[float] = Field(PositiveFloat, min_length=2)  # type: ignore
     FREE_REWARD_VOLUME_UL: PositiveFloat = 1.5
 
 

iblrig/rig_component.py

@@ -1,15 +1,16 @@
-from abc import abstractmethod
+from abc import ABC, abstractmethod
+
+from pydantic import BaseModel
 
 from iblrig.hardware_validation import Validator
-from iblrig.pydantic_definitions import BunchModel
 
 
-class RigComponent:
-    @abstractmethod
+class RigComponent(ABC):
     @property
+    @abstractmethod
     def pretty_name(self) -> str:
         """
-        Get the pretty name of the component.
+        Get the component's pretty name.
 
         Returns
         -------
@@ -18,11 +19,11 @@ def pretty_name(self) -> str:
         """
         ...
 
-    @abstractmethod
     @property
+    @abstractmethod
     def validator(self) -> Validator:
         """
-        Get the validator for the component.
+        Get the component's validator.
 
         Returns
         -------
@@ -31,15 +32,15 @@ def validator(self) -> Validator:
         """
         ...
 
-    @abstractmethod
     @property
-    def settings(self) -> BunchModel:
+    @abstractmethod
+    def settings(self) -> BaseModel:
         """
-        Get the settings for the component.
+        Get the component's settings.
 
         Returns
         -------
-        BunchModel
+        BaseModel
             The pydantic model for the component's settings.
         """
         ...