openssl on Mac

[hpages]

See Huber-group-EMBL/Rhdf5lib#58 and #245 for some background.

A central idea to the business of building and distributing Mac binaries for CRAN and Bioconductor packages is that these binaries should work on a pristine Mac where those optional system libraries are not necessarily available.

On the Mac builders we want to install Simon's openssl binary from https://mac.r-project.org/bin/. These binaries contain static libraries only (.a files, e.g. /opt/R/arm64/lib/libcrypto.a). They don't contain dynamic libraries (.dylib files). Bioconductor packages that need to link against the crypto library should link against /opt/R/arm64/lib/libcrypto.a.

Also, for the sake of keeping the Mac builders as clean as possible, and to avoid the risk of linking to the wrong libraries, we should try to avoid having the brewed openssl. Note that this one provides both libcrypto.a and libcrypto.dylib.

[jwokaty]

In order to brew uninstall openssl, I had to use the --ignore-dependencies option since it's a dependency for mono and open-babel. I think other packages will also install it. I want to confirm again that these two are fine without it. I checked open-babel yesterday but I didn't run the packages that they are associated with.

Note: (mostly for myself) brew uninstall won't remove dependencies installed for a package; however, you can run brew autoremove for clean up.

• confirm okay with rawrr (R CMD check) and ChemmieOB
• confirm removal on all mac machines (also /etc/profile and symlinks)

I think the documentation is mostly correct now; however, there is one section https://github.com/Bioconductor/BBS/blob/devel/Doc/Prepare-macOS-Big-Sur-HOWTO.md#46-install-the-mysql-client that states you should have a "brewed" openssl. Maybe we need an alternative way to install mysql-client.

[vjcitn]

because that piece relates to ensemblVEP which is a) probably going to be deprecated and b) uses REST API in current implementation, the mysql dimension may be ignored for now? i think a meaningful revision of ensemblVEP would eliminate use of mysql

[hpages]

In order to brew uninstall openssl, I had to use the --ignore-dependencies option

Yeah bummer, that's what I suspected. Let's see what happens on the next report for kjohnson1 for ChemmineOB and rawrr and their rev deps:

• https://bioconductor.org/checkResults/3.18/bioc-mac-arm64-LATEST/ChemmineOB/
• https://bioconductor.org/checkResults/3.18/bioc-mac-arm64-LATEST/rawrr/

Note that ChemmineOB is already broken (and has been for a while on kjohnson1. Looks like the package has never built on Mac arm64 as I don't see the binary in BioC 3.18, BioC 3.17, or BioC 3.16. But that's apparently for other reasons (i.e. not related to openssl), so we might not be able to learn much until that one is addressed first.

Thanks!

[jwokaty]

I just wanted to note that there is an "default" installed /usr/bin/openssl. Doing which openssl, points to this path, which we may want to change. Since kjohnson3 started a run today, I set PATH=/opt/R/arm64/bin:$PATH in /etc/profile such that Simon's openssl is selected. I can undo this on the next run if it's not what we want.

It's not clear to me how the configure script knows what openssl to choose. Are these configure scripts made with autoconf?

[jwokaty]

I think something brew installs openssl? I removed it on lconway last week then wanted to check/compare settings with kjohnson1 today when I noticed it in the brew list, so I checked history and it was uninstalled and its files were removed. I'll check again tomorrow to see if it has reappeared.

Note to remind myself what I did today:

lconway:~ biocbuild$ brew list
==> Formulae
boost			gcc			icu4c			libgit2			libunistring		libyaml			open-babel		readline		xorgproto
ca-certificates		gdbm			infernal		libidn2			libx11			lzo			openssl@1.1		salt			xz
cairo			gettext			isl			libmpc			libxau			mono			pcre			six			zeromq
cpanminus		glib			libcbor			libpng			libxcb			mpdecimal		pixman			sqlite			zstd
eigen			gmp			libevent		libpthread-stubs	libxdmcp		mpfr			pstree			tmux
fontconfig		htslib			libffi			libsodium		libxext			mysql-client		python@3.10		utf8proc
freetype		hwloc			libfido2		libssh2			libxrender		ncurses			python@3.9		wget

lconway:~ biocbuild$ brew uninstall openssl@1.1
Uninstalling /usr/local/Cellar/openssl@1.1/1.1.1o... (8,089 files, 18.5MB)

Warning: The following openssl@1.1 configuration files have not been removed!
If desired, remove them manually with `rm -rf`:
  /usr/local/etc/openssl@1.1
  /usr/local/etc/openssl@1.1/cert.pem
  /usr/local/etc/openssl@1.1/ct_log_list.cnf
  /usr/local/etc/openssl@1.1/ct_log_list.cnf.dist
  /usr/local/etc/openssl@1.1/misc
  /usr/local/etc/openssl@1.1/misc/CA.pl
  /usr/local/etc/openssl@1.1/misc/tsget
  /usr/local/etc/openssl@1.1/misc/tsget.default
  /usr/local/etc/openssl@1.1/misc/tsget.pl
  /usr/local/etc/openssl@1.1/misc/tsget.pl.default
  /usr/local/etc/openssl@1.1/openssl.cnf
  /usr/local/etc/openssl@1.1/openssl.cnf.dist

Warning: The following may be openssl@1.1 configuration files and have not been removed!
If desired, remove them manually with `rm -rf`:
  /usr/local/etc/openssl@3
lconway:~ biocbuild$ rm -rf /usr/local/etc/openssl\@3
lconway:~ biocbuild$ rm -rf /usr/local/etc/openssl\@1.1

[jwokaty]

I don't see openssl@1.1 today, which is a good sign.

[jwokaty]

We should also put export PATH=/opt/R/x86_64/bin:$PATH or export PATH=/opt/R/arm64/bin:$PATH in /etc/profile so that it uses Simon's openssl.

[hpages]

/opt/R/x86_64/bin or /opt/R/arm64/bin contains standalone executables. I'm not sure who/what needs them. AFAIK Rhdf5lib needs to be linked to some of the libraries provided by Simon's openssl, like libcrypto.a. These are located in /opt/R/x86_64/lib or /opt/R/arm64/lib.

Adding export PATH=/opt/R/x86_64/bin:$PATH or export PATH=/opt/R/arm64/bin:$PATH in /etc/profile has the potential to mask other executables with consequences that are hard to predict, so I would recommend holding off until we have gathered evidences that some packages actually need that.

[jwokaty]

rawrr and ChemmieOB are building fine (except for ChemmieOB on Mac ARM64, which fail for another reason). I'm adding a note in the documentation that we should skip installing the mysql client on macOSX since ensembl-VEP will be replaced: #386.