Skip to content

Commit

Permalink
Fback points 2
Browse files Browse the repository at this point in the history
  • Loading branch information
mikelgg93 committed Jan 17, 2024
1 parent a40f8ec commit 7785d6e
Show file tree
Hide file tree
Showing 6 changed files with 45 additions and 42 deletions.
File renamed without changes
20 changes: 10 additions & 10 deletions core/getting-started/index.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,27 @@
---
description: Welcome to Pupil Core! This section will guide you through setup (software and hardware) in order for you to make your first recording with Pupil Core.

sidebar: false
sidebar: false
---

<script setup>
import DownloadLinks from '../../components/DownloadLinks.vue/'
import DownloadLinks from '@components/DownloadLinks.vue'
</script>

# Getting Started

Welcome to Pupil Core! The first thing you need to do is download the latest Pupil Core software.

<download-links
src="https://api.github.com/repos/pupil-labs/pupil/releases/latest"
text="Download Pupil Core Bundles"
icon="/bundle-core.svg"
icon="./bundle-core.svg"
/>


Follow the steps below to get up and running and become familiar with the workflow.


## 1. Put on Pupil Core

Put on the Pupil Core headset and plug it into your computer.

Make sure there is space between the headset frame and your forehead. Headsets are adjustable and shipped with additional parts. For more information head over to the [Hardware](/hardware/ "Pupil Core hardware documentation") section of the docs.
Expand All @@ -41,16 +42,12 @@ Take a look at the eye window(s).

Slide the cameras along the headset frame and rotate them on the ball joint, until you get a clear image of both eyes. Make sure that your pupil is always visible, even when looking at extreme angles.


<Youtube src="kjjPL7gLy7s"/>


Pupil Core uses a 3D model of the eye to improve pupil detection. Slowly move your eyes around until the eye model (blue circle) is adjusted to fit your eyeball. If everything is set up properly, you should see a blue circle around the eyeball and a red circle around the pupil with a red dot in the center.


<Youtube src="_1ZRgfLJ3hc"/>


Next, check the world window.

You will see confidence graphs in the top for each eye. 1.0 = high confidence pupil detection. 0.0 = no confidence.
Expand All @@ -60,14 +57,15 @@ Tip - There is another way to build the 3d model. You can also move your head ar
:::

## 4. Calibration

In order to know what someone is looking at, we must establish a mapping between pupil and gaze positions. This is what we call calibration. The calibration process establishes a mapping from pupil to gaze coordinates.

<video width="100%" controls>
<source src="./clb-hd.mp4" type="video/mp4">
</video>


#### Screen Marker Calibration Method

Click `c` on the world screen or press `c` on the keyboard to start calibrating.
Follow the marker on the screen with your eyes and try to keep your head stationary.

Expand Down Expand Up @@ -106,13 +104,15 @@ recordings/
```

## 7. Visualize in Pupil Player

Launch Pupil Player. Then drag and drop the recording folder (e.g `001`) onto the Pupil Player window.

![Pupil Player Icon](./pp.png)

Player comes with a number of plugins. Plugins are classified by their use-case. Visualization plugins can be additive. This means that you can add multiple instances of a plugin to build up a visualization.

## 8. Export Data

You can export videos and data by pressing `e` on your keyboard or the down arrow button in the Pupil Player window.

Exports are saved within a dedicated folder named exports within the original recording folder.
Expand Down
15 changes: 9 additions & 6 deletions neon/neon-player/fixation-and-blink-detectors/index.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,29 @@
# Fixation & Blink Detectors

<!-- TODO: Add tunable parameters in the fixation as table -->
Neon Player runs the same [fixation](./../../data-collection/data-streams/#fixations) and [blink detectors](./../../data-collection/data-streams/#blinks) that are employed in Pupil Cloud.

Neon Player runs the same [fixation](./../../data-collection/data-streams/#fixations) and [blink detectors](./../../data-collection/data-streams/#blinks) that are employed in Pupil Cloud.

The exported data also follows the same structure as reported in the [Data Format](./../../data-collection/data-format/).

## Fixation Detector

The fixation detector calculates fixations for the whole recording. The menu gives feedback about the progress of the detection, how many fixations were found, and shows detailed information about the current fixation. Press `f` or click the `f` hot key button on the left hand side of the window to seek forward to the next fixation.

![Fixations](./np-fixation.webp)

Toggle `Show fixations` to show a visualization of fixations. The blue number is the number of the fixation (0 being the first fixation). You can export fixation reports for your current trim section by pressing `e` on your keyboard or the `e` hot key button on the left hand side of the window.

You can also modify the appearance of the fixation.
You can also modify the appearance of the fixation.

## Blink Detector
Read more information about our blink detector [here](./../../data-collection/data-streams/#blinks).

Read more information about our blink detector [here](./../../data-collection/data-streams/#blinks).

![Blinks](./np-blinks.webp)

You can see a graph on the bottom as a timeline of blinks, this will allow you to visualise.

:::tip
Want to compute blink rate or use this detector in realtime? Have a look at [Detect Eye Blinks With Neon]
<!-- (./../../../alpha-lab/blink-detection/#detecting-eye-blinks-using-pupil-labs-blink-detection-pipeline) -->
:::
Want to compute blink rate or use this detector in realtime? Have a look at [Detect Eye Blinks With Neon](./../../../alpha-lab/blink-detection/#detecting-eye-blinks-using-pupil-labs-blink-detection-pipeline)
:::
52 changes: 26 additions & 26 deletions neon/neon-player/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@
Neon Player is a cross-platform desktop application for playing back and exporting Neon recordings offline.

<script setup>
import DownloadLinks from '../../components/DownloadLinks.vue/'
import DownloadLinks from '@components/DownloadLinks.vue'
</script>

<download-links
src="https://api.github.com/repos/pupil-labs/neon-player/releases/latest"
text="Download Neon Player"
icon="/neon-player.svg"
icon="./neon-player.svg"
/>

## Why Neon Player?
Expand Down Expand Up @@ -49,7 +49,7 @@ Recordings downloaded from [Pupil Cloud](./../pupil-cloud/) would be at 200Hz.
1. **Timeline Events**: Plugins can add temporal events to this expandable panel.
1. **Timeline**: Control the playback of the video with the play/pause button (or spacebar on your keyboard). Drag the playhead (vertical line) to the desired point in time.

- **Trimming**: Drag either end of the timeline to set a trim beginning and ending trim marks. The trim section marks directly inform the section of video/data to export.
- **Trimming**: Drag the green rounded rectangles at either end of the timeline to set beginning and ending trim markers. The trim section markers specify the section of the video/data to export.
- **Frame Stepping**: You can use the arrow keys on your keyboard or the `<<` `>>` buttons to advance one frame at a time while the playback is paused.
- **Playback Speed**: To change the playback speed, use the arrow keys on your keyboard or the `<<` `>>` buttons during playback. There are 5 available playback speeds: `0.25x`, `0.5x`, `1x` (default), `2x`, `4x`.

Expand All @@ -58,19 +58,28 @@ Recordings downloaded from [Pupil Cloud](./../pupil-cloud/) would be at 200Hz.

### Keyboard Shortcuts

| Keyboard Shortcut | Description |
| :---------------- | :----------------------------------------------------- |
| `<space>` | Play and pause video |
| `<arrow left>` | Step to previous frame\* / Decrease playback speed\*\* |
| `<arrow right>` | Step to next frame\* / Increase playback speed\*\* |
| `e` | Start export |
| `a` | Surface tracker: Add new surface |
| `x` | Add annotation (default keyboard shortcut) |
| `f` | Fixation: Show next |
| `F` | Fixation: Show previous |
| Keyboard Shortcut | Description |
| :---------------- | :------------------------------------------------------------------------- |
| `<space>` | Play and pause video |
| `<arrow left>` | Step to previous frame <sup>1</sup> / Decrease playback speed <sup>2</sup> |
| `<arrow right>` | Step to next frame <sup>1</sup> / Increase playback speed <sup>2</sup> |
| `e` | Start export |
| `a` | Surface tracker: Add new surface |
| `x` | Add annotation (default keyboard shortcut) |
| `f` | Fixation: Show next |
| `F` | Fixation: Show previous |

\* While paused
\*\* During playback
<sup>1</sup> While paused
<sup>2</sup> During playback

## Plugins

Neon Player uses a plugin framework to add functionality. Visualizations, marker tracking, and the exporter are all implemented using this structure.

There are two general types of plugins:

- **Unique**: You can only launch one instance of this plugin.
- **Not unique**: You can launch multiple instances of this type of plugin. For example, you can load one `Vis Circle` plugin to render the gaze position with a translucent green circle, and another `Vis Circle` plugin to render the gaze circle with a green stroke of 3 pixel thickness. You can think of these types of plugins as _additive_.

## Workflow

Expand All @@ -79,26 +88,17 @@ Neon Player is similar to a video player. You can playback recordings and can lo
Here is an example workflow:

- Start **Neon Player** and load a recording.
- Open a Plugin - From the `Plugin Manager` GUI menu load the `Vis Circle` plugin.
- Open a Plugin - From the `Plugin Manager` GUI menu (which can be found in the Sidebar), toggle any Plugin of your choice. The `Vis Circle` plugin is activated by default.
- Playback - press the play button or `space` bar on your keyboard to view the video playback with visualization overlay, or drag the playhead in the seek bar to scrub through the dataset.
- Set trim marks - you can drag the green rounded rectangle at the beginning and end of the seekbar to set the trim marks. This will set the start and end frame for the exporter and for other plugins.
- Export Video & Raw Data - From the `Plugin Manager` view, load the `World Video Exporter` plugin and the `Raw Data Exporter` plugin. Press `e` on your keyboard or the `download icon` button in the left hand side of the window to start the export.
- Export Video & Raw Data - From the `Plugin Manager` view, load the `World Video Exporter` plugin and the `Raw Data Exporter` plugin. Press `e` on your keyboard or the `download icon` button in the left hand side of the window to start the export.
- Check out exported data in the `exports` directory within your recording directory.

::: info
Neon Player will **never** remove or overwrite any of your raw data gathered during capture. Instead, it will create a new folder, `neon_player`, which contains the Neon Player compatible files.
All exports are isolated within a sub-directory named `exports` and will never be overwritten.
:::

## Plugins

Neon Player uses a plugin framework to add functionality. Visualizations, marker tracking, and the exporter are all implemented using this structure.

There are two general types of plugins:

- **Unique**: You can only launch one instance of this plugin.
- **Not unique**: You can launch multiple instances of this type of plugin. For example, you can load one `Vis Circle` plugin to render the gaze position with a translucent green circle, and another `Vis Circle` plugin to render the gaze circle with a green stroke of 3 pixel thickness. You can think of these types of plugins as _additive_.

::: tip
Looking for a command line interface that allow you to export recordings as CSV files programmatically? `pl-rec-export` is available [here](https://github.com/pupil-labs/pl-rec-export).
:::
File renamed without changes
Binary file modified neon/neon-player/pp-start.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 7785d6e

Please sign in to comment.