DOC: Need to update instances of mayavi.core.api.Scene in docstrings

[chapochn]

Hello,

I am trying to better understand the visualization capabilities of mne.
The return of the function mne.viz.plot_alignment is said to be
fig : instance of mayavi.mlab.Figure
The mayavi figure.

On the other hand, I understand that mne is using one of 3 backends: mayavi, pyvista, notebook:
https://mne.tools/stable/generated/mne.viz.set_3d_backend.html

So should the documentation of mne.viz.plot_alignment be updated?

[drammock]

The return of the function mne.viz.plot_alignment is said to be
fig : instance of mayavi.mlab.Figure
The mayavi figure.

yep, that's outdated. Care to open a pull request to fix it? I think the correct text is something like "pyvista Figure or mayavi Scene". I'm not sure whether plot_alignment will work with the notebook backend... @larsoner or @GuillaumeFavelier might know.

[chapochn]

I tried with notebook backend, but I get errors (below).
However, this link shows that it should work:
https://mne.tools/stable/generated/mne.viz.set_3d_backend.html

Do you know what this error is about?
Thank you!

fsaverage_path = pathlib.Path(mne.datasets.fetch_fsaverage())
mne.viz.plot_alignment(subject='', subjects_dir=fsaverage_path,
                     surfaces={'pial':1}, coord_frame='mri', show_axes=True)

---------------------------------------------------------------------------

RuntimeError                              Traceback (most recent call last)

/var/folders/w9/3c0pg_qj35x4m8cq745x1txxj4dh2s/T/ipykernel_17313/342389656.py in <module>
     10 
     11 fsaverage_path = pathlib.Path(mne.datasets.fetch_fsaverage())
---> 12 mne.viz.plot_alignment(subject='', subjects_dir=fsaverage_path,
     13                      surfaces={'pial':1}, coord_frame='mri', show_axes=True)
     14 

<decorator-gen-152> in plot_alignment(info, trans, subject, subjects_dir, surfaces, coord_frame, meg, eeg, fwd, dig, ecog, src, mri_fiducials, bem, seeg, fnirs, show_axes, dbs, fig, interaction, verbose)

~/opt/anaconda3/envs/mne/lib/python3.9/site-packages/mne/viz/_3d.py in plot_alignment(***failed resolving arguments***)
   1096         renderer.tube(origin=origin, destination=destination)
   1097 
-> 1098     renderer.set_camera(azimuth=90, elevation=90,
   1099                         distance=0.6, focalpoint=(0., 0., 0.))
   1100     renderer.show()

~/opt/anaconda3/envs/mne/lib/python3.9/site-packages/mne/viz/backends/_pyvista.py in set_camera(self, azimuth, elevation, distance, focalpoint, roll, reset_camera, rigid)
    597                    focalpoint='auto', roll=None, reset_camera=True,
    598                    rigid=None):
--> 599         _set_3d_view(self.figure, azimuth=azimuth, elevation=elevation,
    600                      distance=distance, focalpoint=focalpoint, roll=roll,
    601                      reset_camera=reset_camera, rigid=rigid)

~/opt/anaconda3/envs/mne/lib/python3.9/site-packages/mne/viz/backends/_pyvista.py in _set_3d_view(figure, azimuth, elevation, focalpoint, distance, roll, reset_camera, rigid)
    994         figure.plotter.camera.SetRoll(figure.plotter.camera.GetRoll() + roll)
    995 
--> 996     figure.plotter.update()
    997     _process_events(figure.plotter)
    998 

~/opt/anaconda3/envs/mne/lib/python3.9/site-packages/pyvista/plotting/plotting.py in update(self, stime, force_redraw)
   1401             update_rate = self.iren.get_desired_update_rate()
   1402             if (curr_time - Plotter.last_update_time) > (1.0/update_rate):
-> 1403                 self.right_timer_id = self.iren.create_repeating_timer(stime)
   1404                 self.render()
   1405                 Plotter.last_update_time = curr_time

~/opt/anaconda3/envs/mne/lib/python3.9/site-packages/pyvista/plotting/render_window_interactor.py in create_repeating_timer(self, stime)
    607         timer_id = self.interactor.CreateRepeatingTimer(stime)
    608         if hasattr(self.interactor, 'ProcessEvents'):
--> 609             self.process_events()
    610         else:
    611             self.interactor.Start()

~/opt/anaconda3/envs/mne/lib/python3.9/site-packages/pyvista/plotting/render_window_interactor.py in process_events(self)
    629         # Note: This is only available in VTK 9+
    630         if not self.initialized:
--> 631             raise RuntimeError('Render window interactor must be initialized '
    632                                'before processing events.')
    633         self.interactor.ProcessEvents()

RuntimeError: Render window interactor must be initialized before processing events.

[larsoner]

I think all 3D functions should work with the notebook backend, so I consider this is a bug, cc @GuillaumeFavelier

[GuillaumeFavelier]

Can you share the full code snippet @chapochn ? Did you set the 3d backend explicitly with (for example):

%matplotlib inline
import mne
mne.viz.set_3d_backend('notebook')  # set the 3d backend

I tried my usual notebook (based on our examples) which uses set_3d_view() on main (47c0112):

%matplotlib inline
import mne
mne.viz.set_3d_backend('notebook')  # set the 3d backend

import os.path as op

import numpy as np
import nibabel as nib
from scipy import linalg

import mne
from mne.io.constants import FIFF

data_path = mne.datasets.sample.data_path()
subjects_dir = op.join(data_path, 'subjects')
raw_fname = op.join(data_path, 'MEG', 'sample', 'sample_audvis_raw.fif')
trans_fname = op.join(data_path, 'MEG', 'sample',
                      'sample_audvis_raw-trans.fif')
raw = mne.io.read_raw_fif(raw_fname)
trans = mne.read_trans(trans_fname)
src = mne.read_source_spaces(op.join(subjects_dir, 'sample', 'bem',
                                     'sample-oct-6-src.fif'))

# load the T1 file and change the header information to the correct units
t1w = nib.load(op.join(data_path, 'subjects', 'sample', 'mri', 'T1.mgz'))
t1w = nib.Nifti1Image(t1w.dataobj, t1w.affine)
t1w.header['xyzt_units'] = np.array(10, dtype='uint8')
t1_mgh = nib.MGHImage(t1w.dataobj, t1w.affine)

fig = mne.viz.plot_alignment(raw.info, trans=trans, subject='sample',
                             subjects_dir=subjects_dir, surfaces='head-dense',
                             show_axes=True, dig=True, eeg=[], meg='sensors',
                             coord_frame='meg')
mne.viz.set_3d_view(fig, 45, 90, distance=0.6, focalpoint=(0., 0., 0.))
print('Distance from head origin to MEG origin: %0.1f mm'
      % (1000 * np.linalg.norm(raw.info['dev_head_t']['trans'][:3, 3])))
print('Distance from head origin to MRI origin: %0.1f mm'
      % (1000 * np.linalg.norm(trans['trans'][:3, 3])))
dists = mne.dig_mri_distances(raw.info, trans, 'sample',
                              subjects_dir=subjects_dir)
print('Distance from %s digitized points to head surface: %0.1f mm'
      % (len(dists), 1000 * np.mean(dists)))

And I don't reproduce the bug:

[osaaso3]

Hi GuillaumeFavelier,
Your code didn't work on Colab. After installing some modules, mne, vtk, pyvista,...
the last error was this:
RuntimeError: Render window interactor must be initialized before processing events.

[agramfort]

it's not supported on colab presently. I don't think anyone made it work there so far

…

[GuillaumeFavelier]

Indeed support for Colab still requires development upstream (ipycanvas maybe then ipyvtklink).

You can follow the progress in:

#8704

[chapochn]

Hi,

I installed the dev version of mne in a new env, and your code works there (but not in my usual env).
Here are the 2 mne.sys_info()

My usual env where it is not working:

Platform:      macOS-11.6-x86_64-i386-64bit
Python:        3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 20:33:18)  [Clang 11.1.0 ]
Executable:    /Users/chapon01/opt/anaconda3/envs/mne/bin/python
CPU:           i386: 12 cores
Memory:        16.0 GB

mne:           0.23.4
numpy:         1.21.2 {blas=NO_ATLAS_INFO, lapack=lapack}
scipy:         1.7.1
matplotlib:    3.4.3 {backend=module://matplotlib_inline.backend_inline}

sklearn:       1.0
numba:         0.53.1
nibabel:       3.2.1
nilearn:       0.8.1
dipy:          1.3.0
cupy:          Not found
pandas:        1.3.4
mayavi:        4.7.2
pyvista:       0.32.1 {pyvistaqt=0.5.0, OpenGL 4.1 ATI-4.6.20 via AMD Radeon Pro 560X OpenGL Engine}
vtk:           9.0.3
PyQt5:         5.12.3

Working here, installed dev version of mne 0.24 with pip, and installed other packages it asked for.

Platform:      macOS-10.16-x86_64-i386-64bit
Python:        3.9.7 (default, Sep 16 2021, 08:50:36)  [Clang 10.0.0 ]
Executable:    /Users/chapon01/opt/anaconda3/envs/mne_dev/bin/python
CPU:           i386: 12 cores
Memory:        Unavailable (requires "psutil" package)
mne:           0.24.dev0
numpy:         1.21.2 {blas=mkl_rt, lapack=mkl_rt}
scipy:         1.7.1
matplotlib:    3.4.3 {backend=module://matplotlib_inline.backend_inline}

sklearn:       Not found
numba:         Not found
nibabel:       3.2.1
nilearn:       Not found
dipy:          Not found
cupy:          Not found
pandas:        1.3.4
mayavi:        Not found
pyvista:       0.32.1 {OpenGL 4.1 ATI-4.6.20 via AMD Radeon Pro 560X OpenGL Engine}
pyvistaqt:     0.5.0
ipyvtklink:    0.2.1
vtk:           9.0.3
PyQt5:         5.9.2
ipympl:        Not found
/var/folders/w9/3c0pg_qj35x4m8cq745x1txxj4dh2s/T/ipykernel_38389/1166074264.py:1: RuntimeWarning: macOS users should use PyQt5 >= 5.10 for GUIs, got 5.9.2. Please upgrade e.g. with:

    pip install "PyQt5>=5.10,<5.14"

I see my usual install doesn't have a separate line for pyvistaqt and doesn't mention ipyvtklink

So I imagine the notebook 3D plotting was fixed in the dev version of mne?

[larsoner]

So I imagine the notebook 3D plotting was fixed in the dev version of mne?

There have been many fixes since 0.23, probably some for the notebook, yes

[chapochn]

ok thanks! I can update the doc to mne.viz.plot_alignment with a PR

[chapochn]

Although I see it needs to be updated in many places in all 3d related functions

[larsoner]

mayavi is deprecated and will be removed soon, so we can just update all the docstrings as part of the deprecation process.

Though @GuillaumeFavelier I think we have a bit of work to do in terms of saying what it is we actually return, IIRC in the PyVista cases it's some private-namespace object

[chapochn]

I see! I guess I will let you take care of it then, as it might be beyond my understanding at this point :) Thanks a lot!

[larsoner]

Fixed by #9945