S3 VFD in Rhdf5lib support for TENxIO tests on Mac

[jwokaty]

From @LiNk-NY

TENxIO requires S3 VFD in Rhdf5lib for tests and shows an error on Mac because the needed OpenSSL header files are not found (where they should be ?).
Error in H5Pset_fapl_ros3(fapl, s3credentials) :
Rhdf5lib was not compiled with support for the S3 VFD
Build report: https://bioconductor.org/checkResults/devel/bioc-LATEST/TENxIO/lconway-checksrc.html
There was a resolution of this issue posted on the support site: https://support.bioconductor.org/p/9134972/#9135002
Can S3 VFD be supported on the Mac builder?

[PeteHaitch]

This error has again been brought to my attention in a package review: Bioconductor/Contributions#3105 (comment).
It's a bit of a distraction/complication for new package submissions.

Could we please get an update on this?

[hpages]

For some reason it looks like Rhdf5lib's configure script is not able to find the openssl headers on lconway 😞

I believe that openssl got installed via Homebrew on the machine, but, unfortunately, Homebrew installations tend to cause all sorts of problems for R and R packages.

@jwokaty Can you please try to install Simon's openssl binary on lconway? It's available at https://mac.r-project.org/bin/ Make sure to grab the one in darwin20/x86_64 (macOS 11, Intel (x86_64)). To confirm that this does the trick, simply run Rhdf5lib's configure script:

cd ~/bbs-3.18-bioc/meat/Rhdf5lib
./configure

Right now this produces the following ouput (showing the last lines only):

...
checking for openssl/evp.h... no
checking for openssl/hmac.h... no
checking for openssl/sha.h... no
S3_VFD=--enable-ros3-vfd=no
configure: creating ./config.status
config.status: creating src/Makevars

but we want to see:

...
checking for openssl/evp.h... yes
checking for openssl/hmac.h... yes
checking for openssl/sha.h... yes
checking for curl_global_init in -lcurl... yes
checking for EVP_sha256 in -lcrypto... yes
S3_VFD=--enable-ros3-vfd
configure: creating ./config.status
config.status: creating src/Makevars

(Note that this is what we see on all other Mac builders including arm64 builders kjohnson1 and kjohnson2.)

Once this is confirmed, there are a few extra steps:

1. Adjust the export OPENSSL_LIBS= line in /etc/profile. This is for the rtracklayer package which relies on this environment variable to find the openssl static libraries. This is something Michael asked us to do on the Mac builders a long time ago to accomodate rtracklayer. However, it's a little bit hacky, and, ideally, we should not need to do anything like that. Should probably be revisited at some point.
2. Try to remove the Homebrew installation of openssl.
3. Make sure to update our documentation (Prepare-macOS-Big-Sur-HOWTO.md file).
4. Finally Rhdflib5's version should be bumped to trigger propagation of the new Intel Mac binary (this binary should now support S3_VFD).

Let me know if you need help with this.

H.

[hpages]

@jwokaty Do you think you can take a look at this? Thanks

[jwokaty]

Sorry I didn't see this earlier! I'll look at it tomorrow.

[jwokaty]

I've made the adjustments on lconway and I made some of the edits to the documentation in #357 and 9aa8f62. Only #377 remained.

I just need to ask Rhdflib5's maintainer to bump the version--or should I bump it and inform the maintainer?

[hpages]

Thanks! Were you able to remove the homebrewed openssl? The openssl story on Mac is quite complicated (see grimbough/Rhdf5lib#58). I'll open a new issue where I'll try to provide more details and we can discuss.

[auroramaurizio]

Hi, I'm encountering persistent issues with the Mac check on Iconway for my package, SurfR. The specific error I got and still get (I do not see the failing report anymore with Bioconductor v3.18, only the ERROR flag) is "Rhdf5lib was not compiled with support for the S3 VFD". My previous reviewer raised this concern #245 (comment). The check is successful on Linux nebbiolo1. I would greatly appreciate any advice or insights you might have regarding this matter. Thank you. Aurora

[jwokaty]

Hi @auroramaurizio I've reached out to the Rhdf5lib maintainer about bumping the package version so that the mac binaries can be rebuilt. I have fixed the configurations on the macs according to the suggestions from @hpages. After we have the new version of Rhdf5lib, we should check SurfR's report on the merida1, the 3.19 intel mac.

[grimbough]

I've updated Rhdf5lib to create a new version of the package. Hopefully that will become available on the package builders in the next day or so.