5. TEMPLATE DOCUMENTATION
The Muse-Unity Neurogame Template for Android was designed to be plug-and-play. You can find example scenes in the '/Assets/NeuroGameDev/Scenes/' folder.
This is where the entrails of the Interaxon SDK reside and where all the functions and data from the previous section comes from. For full documentation, consult README_Android.html in the root folder, which will redirect you to the doc folder in the root.
Always make sure that your Muse 2 headset’s battery has sufficient charge before each use.
Press the On button on your headset (inside-right).
Take off any headgear that could encumber the fitting of the headset. Large glasses and headphones, for example. Fit the headset one centimeter (half an inch) above your eyebrows. Press so that it fits comfortably tight. When putting glasses or headphones on, make sure they don’t skew the headband’s position or compromise the contact at the forehead or behind the ears (the plastic wires/springs).
On your Android device, just turn your Bluetooth on and hit play on the game/editor. The headset’s Bluetooth will connect automatically with Unity. No need to connect your Muse 2 headband before playing, like you would with an XBox controller.
You’ll see the Muse HUD (usually on the lower-left) lighting up and start streaming information from the InteraxonInterfacer prefab.
The Muse HUD is a prefab Panel that can be added to any canvas and which connects to the InteraxonInterfacer Singleton, which provides functionality and information to be displayed in the Heads-Up Display.
The HUD displays contact quality, battery percentage and a calculated rate from 0 to 1 of values regarding how focused the player is, how calm the player is and how deep in a state of flow the player is at any given time. A threshold between 0.35 and 0.40 is indicative of a change of state of mind. For instance, a calm score of 0.2 would reflect a not calm mindset while a 0.65 is nearing meditation territory. These values are calculated using brainwave scores, which are values that take a while to calculate since cleanup and averaging is taking place behind the scenes. Absolute and Relative values are streamed immediately but are somewhat unreliable for scoring purposes.
Below, an explanation on how to read the HUD.
This is the HUD reflecting a state wherein there is no connection to the Muse 2. All contact indicators are black. The battery indicator declares that the headset is OFF (even though it might just be disconnected). Focus, Flow and Calm indicators are showing zero values.
The HUD’s battery indicator will switch from OFF to WAIT when it has connected to the Muse 2 headset but it still has not yet received values to display, especially battery percentage. The headband contact indicators may start illuminating to showcase that the headband is starting to calculate electric contact with the skin.
The battery indicator will change from WAIT to a number to indicate the percentage of battery left on the headset.
Ideally, all contact indicators should be green. A comfortable pressure on the forehead and back of the ears will lead to all greens. A few yellows are acceptable too.
As the head moves or as facial muscles bulge, they may shift the contact of the electrodes on the skin and the contact coloring may be lowered to yellow or even red.
Using cumbersome headgear like large glasses or headphones may shift the skin contact with the electrodes, lowering the contact score and skewing the focus, flow and calm values.
Green is ideal, yellow is passable but, if many electrodes display a red color, scores may stop being streamed and the focus, flow and calm scores may reset to zero while ideal conditions are not met.
Do take into account that this HUD is merely informative, it has no buttons to allow scan, connection and disconnection from the Muse headset. These capabilities are present in the Settings panel in the TitleScene. It is recommended to include these settings in a Pause Panel in your game. For testing purposes, this HUD should be enough.
This is where the Neurogame Template helps game developers to access parsed brain data out of the box. The InteraxonInterfacer prefab automatically connects to the first Muse headset detected by your computer’s Bluetooth. It also has methods that can be called to re-scan for other headsets, offer a list of headsets found and connect to a headset by name. This is showcased visibly in the TitleScreen settings panel. There is, however, a lot more information being streamed at any given time.
The Muse device sends packets of JSON data at varying intervals. These packets contain strings, bools, enums and doubles, which can be cast to floats in Unity (you would not be losing that much resolution when it comes to ludic purposes). The InteraxonInterfacer singleton calls and parses these packets to make them public in the code and visible in the inspector.
Some of these can be directly observed in the '/Assets/NeuroGameDev/Scenes/TestScene'.
This is but an accessible taste of what the InteraxonInterfacer is streaming. Below is an entire list and explanation of what is being streamed. This might be a good time to re-read the Neuroscience for Beginners chapter.
If you look up the contents of the InteraxonInterfacer script in the Inspector, you’ll find a barrage of information. Some of it might be intuitively familiar, some of it may feel alien. Here’s a detailed explanation of these contents.
-
User Muse : The name of the selected Muse headset currently connected to this Interfacer.
-
Current Connection State : The connection enum being streamed from the Muse. Possibilities are: UNKNOWN, CONNECTED, CONNECTING, DISCONNECTED, NEEDS_UPDATE, NEEDS_LICENSE.
-
Previous Connection State : A record of what the previous connection enum was. This may serve your game state logic.
-
Bluetooth Mac : The Bluetooth Device Address or MAC address is a unique 48-bit identifier assigned to each Bluetooth device by the manufacturer. Bluetooth Addresses are usually displayed as 6 bytes written in hexadecimal and separated by colons (example - 00:11:22:33:FF:EE).
This section shows the data being streamed that pertains to the Muse headset in and of itself, this information is unique to Interaxon models, although some of these features exist in other brands as well.
For ease of use and, to differentiate from gyroscope values, X, Y and Z have been labeled as forward, rightward and downward, respectively.
The axes are oriented to a Right Hand Coordinate System along the headband axes. Values are given in g (9.81 m/s^2) and are negated to align with the orientation of the headband in Earth's gravitational field.
Values along the X axis increase as the head tilts down aligning the X axis with the downward force of gravity. Negative values indicate a tilt upwards. Values along the Y axis increase as the head tilts to the right. Negative values indicate a tilt to the left.
When worn on a level head, or held in the level position shown in the figure above, the net acceleration of the device will only be caused from gravity. It will be in the direction of the ground aligned with the Z axis. This will give a_x =0, a_y = 0 and a_z = 1 g. As the headband tilts out of this orientation, the value of Z will decrease. -1 in Z represents a headband that is upside down.
The axes are oriented to a Right Hand Coordinate System.
Rotation about the X axis corresponds to tilting the head side to side. Positive values increase when tilting to the right. The X axis is also known as roll.
Rotation about the Y axis corresponds to tilting the head up and down. Positive values increase when looking up. The Y axis is also known as pitch.
Rotation about the Z axis corresponds to looking left and right. Positive values increase when looking to the right. The Z axis is also known as yaw.
This data packet showcases information regarding how much battery is left in percentage (level) as well as its voltage and temperature. For ease of use, the Muse HUD only showcases the level of charge.
The headband has two sensors on the forehead and two sensors on the ears. They are placed and labeled according to the International 10-20 System as discussed in the Brain-Computer Interfaces for Beginners chapter.
The packet streams data from left to right (TP9, AF7, AF8, TP10) and it shows the contact quality of the sensors according to a series of Ints.
A value of 1 represents a good fit,
2 represents a mediocre fit,
and a value or 4 represents a poor fit.
Number 3 is not used.
A good fit is ideal but a mediocre fit will still yield good results. While there is no value for no contact, if the fit is bad enough, the Headband On boolean in Artifacts will be displayed as false.
These artifacts are calculated by the headband from noise, movement and other brainwave irregularities. They are streamed as booleans and can be used to inform the game of specific states which can be used as game mechanics or for diagnostic purposes. This data has a slight delay but they are generally reliable with a properly fitted headband. On the other hand, they become unreliable if the headband is loose or is being shifted or encumbered by headgear such as bulky glasses or headphones.
- Headband On : Whether the sensors are contacting skin.
- Blink : Whether the user is blinking.
- Jaw Clench : Whether the user is clenching their jaw… with force.
- Timestamp : At which time these artifacts were registered.
These values reflect a level of calmness, focus and flow according to mathematical relations between brainwaves as explained in the Brainwaves for Beginners chapter. In order to get the most accurate results, the model uses “scored” or derived alpha, beta, delta, theta and gamma brainwaves (see EEG Data). These values are normalized, which makes them less noisy. It takes a few seconds for them to start computing, since they need to take samples. For a faster, if somewhat more unreliable streaming, substitute scored values for relative values in the script.
The Calm, Focus and Flow values are represented as floats from 0 to 1. Usually, a threshold between ~0.35 and ~0.4 is understood as a change of mindset: from active to calm, from scattered to focused, from disinterested to flowing. Values exceeding ~0.65 are considered to be reflective of a mind in deep meditation. Due to the analog nature of brainwaves, it should be impossible to arrive at a score of 1, barring hardware failure or severe radio/electronic interference… in theory. First time users may find it difficult to exceed a value of 0.3. Test your thresholds with a variety of users.
The Heart Monitor is "broken". Even though there are equations trying to figure out the heart beats per minute, all I got is... the rate of the light being deviated every frame? ... I think? When used to make a dot grow larger or smaller, it almost looks like there is a heartbeat being animated there, but the rate is just too frequent. The datum is provided in case you can figure out what to do with it. But it is definitely not useable out of the box as of yet.
The script behind these values, while based on scientific literature, still has the disadvantage of having been produced by ChatGPT and having had no scientific review by a trained human being. The validity of these values have been tested empirically by Diego Saldivar and have been found to be somewhat satisfactory to those offered by other commercially available headbands.
It works, somehow, but only because of a series of fortunate events.
No rigorous thought went into its development, thus the moniker: Stupid Model.
It is recommended to refine these equations for a commercial neurogame.
Photphletismography (PPG) uses light detection to derive the blood flow under the skin. From these numbers, one can use machine learning to derive the user’s heartrate. The ChatGPT model proved insufficient when trying to calculate heartrate using peak counting, since there is variance between the height of peaks. The data being streamed is there for informational purposes but still needs development to be usable in a ludic setting.
- Ambient : Ambient light as a reference
- Infrared : Infrared light values
- Red : Red light values
This is where the most raw, neuroscientific data is being collected and streamed. One can divide this section thus:
-
(Raw) EEG : Pure electroencephalography data from each sensor without filtering.
-
DRL Ref : Driven Right Leg ground values.
-
[Brainwave] Absolute : Absolute power represents the average individual energy of these brainwaves expressed in microvolts squared over 2.05 seconds. This is a standard neurological value.
-
[Brainwave] Relative : Relative power represents the percent amount of a given frequency band power from the global local energy. This is a standard neurological value.
-
[Brainwave] Scored : Scored or derived alpha values represent frequency bands computed by comparing the current value of a band power to its history. This is an Interaxon proprietary equation. The value is mapped to a score between 0 and 1 using a linear function that returns 0 if the current value is equal to or below the 10th percentile of the distribution of band powers, and returns 1 if it’s equal to or above the 90th percentile. Linear scoring between 0 and 1 is done for any value between these two percentiles.