macOS: Conda version

These instructions are to install slideToolKit and CellProfiler through conda on macOS Big Sur (version 11.6.[x]), macOS Monterey (version 12.0.[x]), and macOS Ventura (version 13.0.[x]).

A very easy and straightforward method to get slideToolKit and CellProfiler to work, is by creating a virtual environment using conda for whole-slide image analyses. This will also enable other approaches, e.g. deeplearning.

There are two assumptions, when using slideToolKit and CellProfiler on macOS. First, as slideToolKit was designed to work on bash, we expect bash to be the standard shell. Second, macOS doesn't come with some very useful command-line programs, therefore we expect brew to be installed to enable installation of some of these programs.

Please review the first chapter, Setting up, before going through the the second chapter.

Chapter 1: Setting up

Step 1: change to bash

Since macOS Catalina the standard shell of Terminal is zsh. I prefer bash. You can easily change this.

List the available shell languages.

cat /etc/shells

Change the shell language to bash.

chsh -s /bin/bash

Make sure you close your Terminal to have the command take effect.

Step 2: make some directories

Before you start, open a Terminal on your macOS. If you have it open make sure to go to the home directory. The following command will take you there (cd means change directory).

cd $HOME

Make .ssh directory this is necessary to store some keys needed with ssh for the HPC, and Git to work.

mkdir .ssh

Also make sure you have the rights set correctly.

chmod -vR 0700 ~/.ssh

Copy the necessary files to this directory.

Make a bin directory, we will use this to make some softlinks to programs installed.

mkdir bin

Also make sure you have the rights set correctly.

chmod -vR 0777 ~/bin

Step 3: get brew

Make sure you have brew installed.

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Also install these packages through brew; you will need them somewhere down the road.

brew install coreutils gnu-sed rename git gd libxml2 gdal libgit2 openblas curl geos findutils zlib boost gsl bison

Some packages for deep learning with python and r on macOS.

brew install cmake pkg-config wget

brew install jpeg libpng libtiff openexr

brew install eigen tbb hdf5

Some packages to use with CellProfiler and slideToolKit. Make sure you follow the required guide to install slideToolKit if you need this.

brew install libharu imagemagick lzo opencv openslide libsvg

It is useful to also install R and RStudio to analyse results obtained through the slideToolKit method.

You can either install r with its GUI or the brew-version that omits the GUI.

with GUI

brew install --cask r

without GUI

brew install r

Now, you can install rstudio.

brew install --cask rstudio

Chapter 2: get conda

Step 1a: install conda

We require conda to have full control on the installation of required libraries and packages for slideToolKit, it will work with python 3.7+.

Note: If you want to also use this environment for deeplearning with TensorFlow make sure you install the python 3.7 version of conda, TensorFlow is not yet optimized for newer python versions.

If you have one of these versions already installed you can skip this step.

Installation of miniconda

We are using miniconda. Make sure to download the right package from the offical miniconda-website, this should be for python 3.7:

• Miniconda3 X86
• Miniconda3 arm64

Note: one other thing, we assume here the use of a M1-processor from Apple (MacBook Air M1 2020) with Rosetta 2. Be sure you download the right version for your architecture. So far, installation with the arm-version has caused issues with miniconda and the required packages. My advise is - for now - to use the X86 version.

Intel or M1-version with Rosetta 2

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh

M1-version

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh

Note that wget only works if you installed this via brew (see Chapter 1).

Next execute the following code.

bash ~/Downloads/Miniconda3-latest-MacOSX-arm64.sh

Installation of anaconda

You could also install anaconda but be aware this is a quite bloated version with a lot of things you do not necessarily need:

• python 3.7, this should be Anaconda3-2019.03-MacOSX-x86_64.
• python 3.8, this should be Anaconda3-2021.05-MacOSX-x86_64.

In this case execute the following code.

bash ~/Downloads/Anaconda3-2019.03-MacOSX-x86_64.sh

Step 1b: update conda

If conda is available, make sure it is up-to-date.

conda update -n base conda

And follow the instructions.

Step 1c: cleanup

It's good practice to cleanup afterwards.

rm -v ~/Downloads/Miniconda3-latest-MacOSX-arm64.sh

Or:

rm -v ~/Downloads/Anaconda3-2019.03-MacOSX-x86_64.sh

Installation issues with brew

It could be that conda is constricting your brew installs, you can prevent this using these instructions.

If you don't want conda to be loaded on startup, you can execute the following command.

conda config --set auto_activate_base false

Step 2: installation slideToolkit

Download and install the latest version of the slideToolKit from GitHub. First create and go to the git directory, then download the slideToolkit.

~/git: this is the folder where you are supposed to install software on your system.

mkdir -p ~/git/ && cd ~/git

if [ -d ~/git/.git ]; then \
		cd ~/git/slideToolKit && git pull; \
	else \
		cd ~/git/ && git clone https://github.com/swvanderlaan/slideToolKit.git; \
	fi

Add symbolic links in ~/bin/. Now the slideToolkit will be availabe in your PATH. Adding the slideToolkit tools to your PATH makes it easier to acces the slideToolkit commands.

mkdir -p ~/bin/ && ln -s -f -v ~/git/slideToolKit/slide* ~/bin/

Step 3: install a virtual environment

Next you should create a virtual environment within which we will install the required packages for slideToolKit and CellProfiler. We created a yml which will setup a virtual environment and installs the required packages through conda and pip3. You can find the yml here: [PATHTO]/slideToolKit/conda_yml/conda3_8_cp413.v1.yml.

Next enter the following.

conda env create -f conda3_8_cp413.v1.yml

Controlling the virtual environment

Activating

Activating (or switching between) your virtual environment(s) is easy.

conda activate cp4

This modifies the PATH and shell variables to point your macOS to the specific python set-up you (just) installed. You'll note that the command prompt now indicates which Conda environment you are currently in by prepending (cp4).

Listing available environments

You can also list the available environments. I tend to forget what I installed, so it comes in handy for me 🙈.

conda env list

This results in the following:

# conda environments:
#
base                  *  /Users/username/miniconda3
cp4                      /Users/username/miniconda3/envs/cp4

Installing additional python packages

The big advantage of using conda is that you can create a virtual environment that contains a specific set-up of python packages for a specific purpose. Just select your virtual environment -n cp4 and the [package] you wish to install.

conda install -n cp4 [package]

Don't forget to specify the virtual environment, because otherwise the package will be installed in the root python installation.

Deactivating/exiting the virtual environment

To end a virtual environment session, i.e. deactivating or exiting, is easy. This will reset the PATH and shell to the base settings of macOS.

conda deactivate

Deleting a virtual environment

You may want to delete a specific conda environment. You can do this by entering the following.

conda remove -n cp4 -all

Other useful commands

List all the conda environments available:

conda info --envs

Create new environment named as envname.

conda create --name envname

Remove environment and its dependencies.

conda remove --name envname --all

Clone an existing environment.

conda create --name clone_envname --clone envname

Step 4: test the environment

And now you should not have any problem running the following script.

python slideToolKitTest.py

This will calculate your age, just for fun. But the most important is to check whether you have the right versions of openslide, opencv and cellprofiler installed. These should be the following.

Printing the installed versions.
* Python version:  3.8.12 | packaged by conda-forge | (default, Oct 12 2021, 21:50:38)
[Clang 11.1.0 ]
* OpenSlide version:  1.1.2
* OpenSlide library version:  3.4.1
* CellProfiler version:  4.1.3

Step 5 - Install CellProfiler, we prefer version 4.1.3.

Install CellProfiler version 4.1.3 following instructions on their website. Using the downloaded installer, CellProfiler will be installed in the default location (/Applications/CellProfiler).

To make the CellProfiler command line interface (CLI) available, we create a cellprofiler script in your ~/bin folder. This scripts links to CellProfiler installed in your /Applications folder.

printf '#!/bin/bash\n# run cellprofiler from CLI\n/Applications/CellProfiler.app/Contents/MacOS/cp "$@"\n' \
    > ~/bin/cellprofiler && chmod -v 0755 ~/bin/cellprofiler

Alternative versions of CellProfiler

We prefer version 4.1.3 because this version works best on our high-performance computing cluster. That said, one could easily install a given version (4.1.3 or the latest version) by simply changing the version number of CellProfiler in the command above, if needed. For example:

printf '#!/bin/bash\n# run cellprofiler from CLI\n/Applications/CellProfiler-macOS-4.1.3.app/Contents/MacOS/cp "$@"\n' \
    > ~/bin/cellprofiler && chmod -v 0755 ~/bin/cellprofiler

Some notes

macOS security

It is very likely you can't run CellProfiler because of the tight security settings - through Gatekeeper - within macOS.

You can try and run this:

sudo spctl --master-disable

This will disable Gatekeeper. Now, you should be able to open the CellProfiler-app and run CellProfiler on the command line:

cellprofiler413 -c -r -p $(pwd)/pipeLines/HE.cp413.v1.0.cppipe --file-list files2cp.txt -o IMG1.HE/cp_output/

Does it work? Be sure to enable Gatekeeper again:

sudo spctl --master-enable

Using --file-list

When you're running CellProfiler on the command line and use --file-list you should make sure that the files in the files-list are given with the full path.

For instance, your file should look like this:

/Users/username/git/swvanderlaan/slideToolKit/upgrade_study/HE/IMG1.HE/IMG1.HE.tiles/IMG1.HE_009.normalized.tile.tissue.png
/Users/username/git/swvanderlaan/slideToolKit/upgrade_study/HE/IMG1.HE/IMG1.HE.tiles/IMG1.HE_022.normalized.tile.tissue.png
/Users/username/git/swvanderlaan/slideToolKit/upgrade_study/HE/IMG1.HE/IMG1.HE.tiles/IMG1.HE_023.normalized.tile.tissue.png

Java & Javabridge

Make sure you have java installed:

java -version && /usr/libexec/java_home -V

Otherwise, install Java JDK:

brew install openjdk

Next, when you do brew info openjdk, you'll get this:

==> openjdk: stable 19.0.2 (bottled) [keg-only]
Development kit for the Java programming language
https://openjdk.java.net/
/usr/local/Cellar/openjdk/19.0.2 (636 files, 318.7MB)
  Poured from bottle on 2023-02-09 at 20:39:19
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/openjdk.rb
License: GPL-2.0-only with Classpath-exception-2.0
==> Dependencies
Build: autoconf ✘, pkg-config ✔
Required: giflib ✔, harfbuzz ✔, jpeg-turbo ✔, libpng ✔, little-cms2 ✔
==> Requirements
Build: Xcode ✘
Required: macOS >= 10.15 ✔
==> Caveats
For the system Java wrappers to find this JDK, symlink it with
  sudo ln -sfn /usr/local/opt/openjdk/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk.jdk

openjdk is keg-only, which means it was not symlinked into /usr/local,
because macOS provides similar software and installing this software in
parallel can cause all kinds of trouble.

If you need to have openjdk first in your PATH, run:
  echo 'export PATH="/usr/local/opt/openjdk/bin:$PATH"' >> /Users/username/.bash_profile

For compilers to find openjdk you may need to set:
  export CPPFLAGS="-I/usr/local/opt/openjdk/include"

==> Analytics
install: 255,113 (30 days), 578,968 (90 days), 2,618,349 (365 days)
install-on-request: 108,095 (30 days), 217,031 (90 days), 809,984 (365 days)
build-error: 490 (30 days)

Be sure to do this:

sudo ln -sfn /usr/local/opt/openjdk/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk.jdk

Also, make sure you have JAVA_HOME in your .bash_profile

export JAVA_HOME="/usr/local/Cellar/openjdk/19.0.2/libexec/openjdk.jdk/Contents/Home"
export PATH="$JAVA_HOME/bin:$PATH"

Chapter 3: A note on installing with mamba

Installing through mamba works a bit different - so far I haven't been able to create a proper .yml-file to help with this.

Step 1: get mamba

First, you'll need mamba. Download the right version for you system - here I used MacOSX-x86_64 because we use Rosetta2.

wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-MacOSX-x86_64.sh --no-check-certificate

You might get a message regarding the certificates, hence the addition of --no-check-certificate.

Next, install and following the instructions. Make sure to close and restart your Terminal-app once it's done.

bash Mambaforge-MacOSX-x86_64.sh

Step 2: create the cp4 environment

Now you're ready to create the cp4 environment.

mamba create --name cp4 python=3.8

Step 3: install the dependencies

We need quite a few packages to be able to (automatically) read barcodes on whole-slide images, run analyses, etc. We do this first.

mamba install -c conda-forge pip numpy matplotlib pandas polars-lts-cpu openjdk scikit-learn mahotas gtk2 gtk3 Jinja2 inflect wxpython mysqlclient sentry-sdk centrosome gensim FuzzyTM xarray python-javabridge bftools cairo freetype gettext giflib imagemagick java-jdk jpeg wmctrl zbar tclap openslide openslide-python

Step 4: install cellprofiler and opencv

Finally, we're ready to install cellprofiler and opencv to handle and analyse WSI.

pip install cellprofiler arrow pathlib opencv-contrib-python 

---

^{Inspired by}

^{https://www.pyimagesearch.com/2019/01/30/macos-mojave-install-tensorflow-and-keras-for-deep-learning/
https://medium.com/swlh/how-to-setup-your-python-projects-1eb5108086b1
https://towardsdatascience.com/how-to-successfully-install-anaconda-on-a-mac-and-actually-get-it-to-work-53ce18025f97
https://gist.github.com/rxaviers/7360908
https://github.com/CellProfiler/CellProfiler/wiki/Conda-Installation
https://dani.gg/en/blog/install-r-and-rstudio-with-homebrew-on-macos/}