PR: Backport of PR 22196 (Make QtWebEngine Optional)

[hmaarrfk]

xref: #22196

Affirmation

By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct:

Mark Harfouche hmaarrfk

[ccordoba12]

I have a quick question for you: do you think this is really necessary for the 5.x branch? I'm asking it because:

1. This branch can't and won't work with Qt6 because it doesn't contain any fixes to support it. In addition, it's in maintenance mode now, so no major features will be merged to it. However, master should work with Qt6.
2. Packages for Qt5 QtWebEngine are available in Conda-forge, so I don't see how this could improve the situation there.

[hmaarrfk]

I find that if you are supporting both, fixing "import errors" makes it really difficult to backport future fixes in the long term.

Given I had already done the work, I figured I could do the backport to test.

Honestly, I've been somewhat managing the cf qt-webengine for Qt5, but at one point, it will break.....

We can definitely "delay" since I got this patch in:
conda-forge/conda-forge-repodata-patches-feedstock#782

but yea..... if you want to have an future release of 5.X, this backport would make future backports easier. Whether or not you decide to remove the pyqtwebengine from the conda-forge releases.

[ccordoba12]

Ok, but our workflow is doing forward-porting, i.e. PRs are opened against 5.x first (if people want them in the next Spyder 5 release), and then we merge them to master.

But given that PR #22196 is almost ready, I think it's better to merge that first and do a manual backport (i.e. taking that PR's full patch and applying it to 5.x).

If you feel comfortable doing that, then please go ahead and apply the patch to this branch in a single commit (and drop your current ones). If not, I can do it for you.

[hmaarrfk]

Ok but I think I needed:

#22199

[hmaarrfk]

Without:

Traceback (most recent call last):
  File "/home/mark/miniforge3/envs/mcam_dev/lib/python3.12/site-packages/qtpy/QtWebEngineWidgets.py", line 28, in <module>
    from PyQt5.QtWebEngineWidgets import (
ModuleNotFoundError: No module named 'PyQt5.QtWebEngineWidgets'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mark/miniforge3/envs/mcam_dev/bin/spyder", line 33, in <module>
    sys.exit(load_entry_point('spyder', 'gui_scripts', 'spyder')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mark/git/spyder/spyder/app/start.py", line 252, in main
    from spyder.app import mainwindow
  File "/home/mark/git/spyder/spyder/app/mainwindow.py", line 57, in <module>
    from qtpy import QtWebEngineWidgets  # analysis:ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/mcam_dev/lib/python3.12/site-packages/qtpy/QtWebEngineWidgets.py", line 36, in <module>
    raise QtModuleNotInstalledError(
qtpy.QtModuleNotInstalledError: The QtWebEngineWidgets module was not found. It must be installed separately as PyQtWebEngine.

With

[hmaarrfk]

PS. there are some merge conflicts between the 6.x branch (master) and 5.x branch. I've been fixing them but for now there is 1 commit in both branches that has "my suggestions" in a single commit. let me know if there is anything else to do.

[hmaarrfk]

Note that this includes the other backport that I needed.

[ccordoba12]

Ok but I think I needed:

#22199

Right, let's try to merge that one first, then we'll return to this one.

[ccordoba12]

@hmaarrfk, a rebase on top of the latest 5.x should be the last thing to do for this one.

[hmaarrfk]

(I made it a draft to make it a little harder to merge until my last question is addressed)

[hmaarrfk]

Without updating the setup file I get:

$ QT_API=pyqt5 spyder
Traceback (most recent call last):
  File "/home/mark/miniforge3/envs/dev/bin/spyder", line 33, in <module>
    sys.exit(load_entry_point('spyder', 'gui_scripts', 'spyder')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/dev/bin/spyder", line 25, in importlib_load_entry_point
    return next(matches).load()
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/dev/lib/python3.12/importlib/metadata/__init__.py", line 205, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/dev/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/mark/git/spyder/spyder/app/start.py", line 59, in <module>
    from spyder.utils.external import lockfile
  File "/home/mark/git/spyder/spyder/utils/external/lockfile.py", line 31, in <module>
    from spyder.utils.programs import is_spyder_process
  File "/home/mark/git/spyder/spyder/utils/programs.py", line 27, in <module>
    import pkg_resources
  File "/home/mark/miniforge3/envs/dev/lib/python3.12/site-packages/pkg_resources/__init__.py", line 3624, in <module>
    @_call_aside
     ^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/dev/lib/python3.12/site-packages/pkg_resources/__init__.py", line 3608, in _call_aside
    f(*args, **kwargs)
  File "/home/mark/miniforge3/envs/dev/lib/python3.12/site-packages/pkg_resources/__init__.py", line 3637, in _initialize_master_working_set
    working_set = _declare_state('object', 'working_set', WorkingSet._build_master())
                                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/dev/lib/python3.12/site-packages/pkg_resources/__init__.py", line 657, in _build_master
    ws.require(__requires__)
  File "/home/mark/miniforge3/envs/dev/lib/python3.12/site-packages/pkg_resources/__init__.py", line 1062, in require
    needed = self.resolve(parse_requirements(requirements))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/dev/lib/python3.12/site-packages/pkg_resources/__init__.py", line 889, in resolve
    dist = self._resolve_dist(
           ^^^^^^^^^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/dev/lib/python3.12/site-packages/pkg_resources/__init__.py", line 930, in _resolve_dist
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'pyqtwebengine<5.16,>=5.10' distribution was not found and is required by spyder

[hmaarrfk]

I've reverted the changes to setup.py

Well i'm ok with this as is, for conda packages, we can always have a 1 line patch that removes the dependency. (Of course this would have to be a decision the maintainers at conda-forge including yourself are OK with, but I often release my own packages to move more at my pace while things go through the regular QC)

I feel like dealing with pkg_resources quirks can be left to an other time when we remove it spyder given this deprecation notice.

https://setuptools.pypa.io/en/latest/pkg_resources.html

[hmaarrfk]

Thank you so much for helping with this alltogether, I know this kind of churn is tiring.

[ccordoba12]

One last suggestion, then this should be ready.

[ccordoba12]

Thank you so much for helping with this alltogether, I know this kind of churn is tiring.

No worries 👍🏽

[ccordoba12]

Without updating the setup file I get:
File "/home/mark/miniforge3/envs/dev/lib/python3.12/site-packages/pkg_resources/init.py", line 930, in _resolve_dist
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'pyqtwebengine<5.16,>=5.10' distribution was not found and is required by spyder
I feel like dealing with pkg_resources quirks can be left to an other time when we remove it spyder given this deprecation notice.

Agreed (I saw that you opened another PR about it).

Well i'm ok with this as is, for conda packages, we can always have a 1 line patch that removes the dependency. (Of course this would have to be a decision the maintainers at conda-forge including yourself are OK with, but I often release my own packages to move more at my pace while things go through the regular QC)

Ok, that's going to be tricky because we don't want to loose support for WebEngine for the PyQt versions that still support it. So, I guess we'll have to create a spyder-base package without WebEngine (and probably no bindings so it can be used with PySide too).

[ccordoba12]

Thanks @hmaarrfk!

[hmaarrfk]

Ok, that's going to be tricky

Yeah, i'm somewhat perfectly fine maintaining small sets of patches (for my own channel) for "integration" so long as the "core features" are supported upstream.

I do dream of a day where we can just compile qt6-webengine at conda-forge, but it is just difficult....