+ + + + + +
+

The largest EEG-based Benchmark for Open Science#

+

We report the results of the benchmark study performed in: +The largest EEG-based BCI reproducibility study for open science: the MOABB benchmark

+

This study conducts an extensive Brain-computer interfaces (BCI) reproducibility analysis on open electroencephalography datasets, +aiming to assess existing solutions and establish open and reproducible benchmarks for effective comparison within the field. Please note that the results are obtained using Within-Session evaluation. +The results are reported regarding mean accuracy and standard deviation across all folds for all sessions and subjects.

+

If you use the same evaluation procedure, you should expect similar results if you use the same pipelines and datasets, with some minor variations due to the randomness of the cross-validation procedure.

+

You can copy and use the table in your work, but please **cite the paper** if you do so.

+
+
+

Motor Imagery#

+

Motor Imagery is a BCI paradigm where the subject imagines performing a movement. +Each imagery task is associated with a different class, and each task has its difficulty level related to how the brain generates the signal.

+

Here, we present three different scenarios for Motor Imagery classification:

+
    +
  1. Left vs Right Hand: We use only the classes Left Hand and Right Hand.

  2. +
  3. Right Hand vs Feet: We use only Right Hand and Feet classes.

  4. +
  5. All classes: We use all the classes in the dataset, when there are more than classes that are not Left Hand and Right Hand.

  6. +
+

All the results here are for within-session evaluation, a 5-fold cross-validation, over the subject’s session.

+
+
+

Motor Imagery - Left vs Right Hand#

+

Left vs Right Hand: We use only the classes Left Hand and Right Hand.

+ + + + + + + + + + + + + + + +

Pipelines

BNCI2014_001

BNCI2014_004

Cho2017

GrosseWentrup2009

Lee2019_MI

PhysionetMI

Schirrmeister2017

Shin2017A

Weibo2014

Zhou2016

+ +
+
+

Motor Imagery - Right Hand vs Feet#

+

Right Hand vs Feet: We use only Right Hand and Feet classes.

+ + + + + + + + + + + + + + +

Pipeline

AlexMI

BNCI2014_001

BNCI2014_002

BNCI2015_001

BNCI2015_004

PhysionetMI

Schirrmeister2017

Weibo2014

Zhou2016

+ + +
+
+

Motor Imagery - All classes#

+

All classes: We use all the classes in the dataset, when there are more than classes that are not Left Hand and Right Hand.

+ + + + + + + + + + + +

Pipelines

AlexMI

BNCI2014_001

PhysionetMI

Schirrmeister2017

Weibo2014

Zhou2016

+
+
+

SSVEP (All classes)#

+

Here, we have the results of the within-session evaluation, a 5-fold cross-validation, over the subject’s session. +We use all the classes available in the dataset.

+ + + + + + + + + + + + + + + + +

Pipeline

Kalunga2016

Lee2019_SSVEP

MAMEM1

MAMEM2

MAMEM3

Nakanishi2015

Wang2016

+ + +
+
+

P300/ERP (All classes)#

+

Here, we have the results of the within-session evaluation, a 5-fold cross-validation, over the subject’s session. +We use all the classes available in the dataset.

+ + + + + + + + + + + + + + + + + + + + +

Pipelines

BNCI2014_008

BNCI2014_009

BNCI2015_003

BI2012

BI2013a

BI2014a

BI2014b

BI2015a

BI2015b

Cattan2019_VR

EPFLP300

Huebner2017

Huebner2018

Lee2019_ERP

Sosulski2019

+ + + +

+
+
+ + +