Conda path

[joaoponceleao]

Hi.
Thanks for the new installer.
I can see pip3 in the Contents/bin folder, but I can't see where the conda env is to activate it. How does one install new packages for plugins? I assume conda would be preferred over pip here (I realise plugins will call pip anyway).
On a related note, please consider pinning any relevant conda packages to prevent dependency issues when installing or updating extra plugin dependencies.

[m-kuhn]

There is an activate script in Contents/lib/python3.11/venv/scripts/common/activate, does that help somehow?
There is a conda executable packaged and used during the installation, however it is no longer on the system after the setup completes, this would need to be investigated.
I did not yet look into how to manage a conda environment and more into how to get an installer for users that are not at ease with the command line. I would try to use pip first and let plugins do their work. Or install conda the command line way.

If we want a better integration of conda for plugin dependencies, I think we should do something like qgis/QGIS-Enhancement-Proposals#202 or https://github.com/opengisch/qpip, where plugins can add a requirements.txt and we could ship a dedicated conda backend to install these.

[joaoponceleao]

Thanks for the input. Personally it would be great if we had access to conda here. My system conda install is not seeing the QGIS venv. I assume I can activate it with conda activate <path to qgis conda venv> and proceed with installs this way? A means to use conda to install packages would be beneficial, particularly with more ML tools and workflows relying on modules that cannot be easily installed through pip (pytorch and the like).

(having said that, I was quite surprised to see most of my dependencies install without issues with the QGIS pip here)

As for plugin dependencies, I agree that this should be up to plugin publishers who will simply use pip, but they too sometimes have to either provide instructions for custom installs, or produce hackish scripts for some edge cases (for instance, we added behaviour in our plugin to install miniconda in a custom folder just to add 'triangle' which tends to trip pip).

[joaoponceleao]

An update on this.
Installing QGIS again removes all pip installs from the system, meaning users that have installed dependencies, as well as plugins, will have to reinstall them. I can imagine this becoming a problem for some plugins that have their requirement install behaviour inside some installation function rather than on load.

Could be worth exploring installing the venv somewhere outside the QGIS.app package (Lib/App Support or ~/.config maybe?), and add a menu item or an option on the installer to delete that location for those who need a clean install.

[m-kuhn]

I think ideally the venv would go into the profile folder (i.e. you can have one venv per profile, just like plugins), just like done with qpip listed above

[joaoponceleao]

That is a great idea. The only issue I can see is what happens when you change profiles. QGIS will pick up the new folder for packages, but will it also activate the new venv (with all the new c library paths)?

[m-kuhn]

C libraries are always a tricky topic, however, I think it boils down to this:

• if a library is already installed on the base system (typical example gdal), ideally its corresponding python libs are installed on the base system as well
• if a python libs for a base system library is installed in a virtualenv, only a wrapper can be installed. Loading the same C lib (in the same or different version) results in UB -> trouble ahead.
• if a python lib with no corresponding C lib is installed, it's free to install the C lib also in the virtualenv. If two python installed modules do this approach we are in UB again.

[tsmcgrath]

I similarly chose the default, not a specific drive, and cannot find the install directory to activate the env. I ran a search on my file system for Contents directory and found some related to other apps (like mito) but not for QGIS. I'm still a relative newb at the nuances of venv's and have very little conda env experience. Mostly use pipenv - when it works. I will re-install into a specific folder but I wanted to capture the experience here.

[tsmcgrath]

I nearly always struggle with environments. I'm pretty sure these are operator errors (mine) but I don't seem to have the env set up right. When I right click and "open" the QGIS3.32.app application I get this error:
An error occurred during execution of following code:
faulthandler.enable(file=qgis.utils.__qgis_fault_handler_file)

Traceback (most recent call last):
File "", line 1, in
PermissionError: [Errno 1] Operation not permitted

Python version:
3.11.6 | packaged by conda-forge | (main, Oct 3 2023, 10:39:39) [Clang 15.0.7 ]

QGIS version:
3.32.3-Lima 'Lima', exported

Python path:
['/Users/tim/DocumentsLocal/Envs/QGIS-3.32.app/Contents/MacOS/../share/qgis/python', '/Users/tim/Library/Application Support/QGIS/QGIS3/profiles/default/python', '/Users/tim/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins', '/Users/tim/DocumentsLocal/Envs/QGIS-3.32.app/Contents/MacOS/../share/qgis/python/plugins', '/Users/tim/DocumentsLocal/Envs/QGIS-3.32.app/Contents/lib/python311.zip', '/Users/tim/DocumentsLocal/Envs/QGIS-3.32.app/Contents/lib/python3.11', '/Users/tim/DocumentsLocal/Envs/QGIS-3.32.app/Contents/lib/python3.11/lib-dynload', '/Users/tim/DocumentsLocal/Envs/QGIS-3.32.app/Contents/lib/python3.11/site-packages']

[tsmcgrath]

So, I re-installed miniconda. created an env. Tried to run the app from the command line:
tim@Timothys-MacBook-Pro Env % /Users/tim/DocumentsLocal/Envs/QGIS-3.32.app
zsh: permission denied: /Users/tim/DocumentsLocal/Envs/QGIS-3.32.app

[tsmcgrath]

Then, with the new install of minconda, I tried to open the QGIS3.32.app and QGIS quit on me. Error file attached.
installerror1.txt

[tsmcgrath]

Next I tried a conda install from a new conda env. This completed:
conda install -n qgis332 -c conda-forge qgis
This completed, but I still cannot start the application. I'm sure its because I'm doing something wrong.
tim@Timothys-MacBook-Pro Env % /Users/tim/DocumentsLocal/Envs/QGIS-3.32.app
zsh: permission denied: /Users/tim/DocumentsLocal/Envs/QGIS-3.32.app
tim@Timothys-MacBook-Pro Env % qgis
zsh: command not found: qgis

[joaoponceleao]

Hi @tsmcgrath

I think you are getting qgis for conda (developed here: https://github.com/conda-forge/qgis-feedstock) and the new qgis installer (this repository, and which happens to be based on conda) confused. The first one is installed through conda like you just did, and is launched through the terminal by activating the qgis conda environment and typing qgis. The second one is installed through the dmg installation file in this repository, and is launched through the finder. It does not include conda, it just happens to be built with it.

The error message message you described is an issue that is common to both versions and being tracked in their respective repositories.

[tsmcgrath]

Hi @joaoponceleao You're correct. I did confuse them. I'll try again with just the pkg.

[tsmcgrath]

This is the error I get when I open it. Installed to applications folder. Somehow I have a permissions error to sort out. There is read/write on the app file. The Lima splash screen comes up. Then this pops up:
`An error occurred during execution of following code:
faulthandler.enable(file=qgis.utils.__qgis_fault_handler_file)

Traceback (most recent call last):
File "", line 1, in
PermissionError: [Errno 1] Operation not permitted

Python version:
3.11.6 | packaged by conda-forge | (main, Oct 3 2023, 10:39:39) [Clang 15.0.7 ]

QGIS version:
3.32.3-Lima 'Lima', exported

Python path:
['/Applications/QGIS-3.32.app/Contents/MacOS/../share/qgis/python', '/Users/tim/Library/Application Support/QGIS/QGIS3/profiles/default/python', '/Users/tim/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins', '/Applications/QGIS-3.32.app/Contents/MacOS/../share/qgis/python/plugins', '/Applications/QGIS-3.32.app/Contents/lib/python311.zip', '/Applications/QGIS-3.32.app/Contents/lib/python3.11', '/Applications/QGIS-3.32.app/Contents/lib/python3.11/lib-dynload', '/Applications/QGIS-3.32.app/Contents/lib/python3.11/site-packages']`

[tsmcgrath]

Ok, got it working. Somehow an "allow" screen didn't show up for me before the error. 2nd time it did. Set to "Always allow".

[tsmcgrath]

So far testing is going well. It's more than 3x faster than the Rosetta version I was using (3.30.3). Well done @m-kuhn! I do still get a PATH error which I think is a conflict with the previous conda installs I did. I will try to get rid of that version and see if it clears up the error. But, the error doesn't stop the app from launching. The error I'm seeing.
`An error occurred during execution of following code:
faulthandler.enable(file=qgis.utils.__qgis_fault_handler_file)

Traceback (most recent call last):
File "", line 1, in
PermissionError: [Errno 1] Operation not permitted

Python version:
3.11.6 | packaged by conda-forge | (main, Oct 3 2023, 10:39:39) [Clang 15.0.7 ]

QGIS version:
3.32.3-Lima 'Lima', exported

Python path:
['/Applications/QGIS-3.32.app/Contents/MacOS/../share/qgis/python', '/Users/tim/Library/Application Support/QGIS/QGIS3/profiles/default/python', '/Users/tim/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins', '/Applications/QGIS-3.32.app/Contents/MacOS/../share/qgis/python/plugins', '/Applications/QGIS-3.32.app/Contents/lib/python311.zip', '/Applications/QGIS-3.32.app/Contents/lib/python3.11', '/Applications/QGIS-3.32.app/Contents/lib/python3.11/lib-dynload', '/Applications/QGIS-3.32.app/Contents/lib/python3.11/site-packages']`