libhdf5 not defined (Apple M1) when installing HDF5.jl

[tamasgal]

Although HDF5_jll is downloaded, HDF5.jl will not precompile and gives the following error on Julia 1.7:

(@v1.7) pkg> generate foo
  Generating  project foo:
    foo/Project.toml
    foo/src/foo.jl

(@v1.7) pkg> add HDF5
    Updating registry at `~/.julia/registries/General`
    Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Resolving package versions...
   Installed Blosc ───── v0.7.2
   Installed Blosc_jll ─ v1.21.1+0
   Installed Compat ──── v3.41.0
  Downloaded artifact: Blosc
    Updating `~/.julia/environments/v1.7/Project.toml`
  [f67ccb44] + HDF5 v0.15.7
    Updating `~/.julia/environments/v1.7/Manifest.toml`
  [a74b3585] + Blosc v0.7.2
  [34da2185] + Compat v3.41.0
  [f67ccb44] + HDF5 v0.15.7
  [692b3bcd] + JLLWrappers v1.3.0
  [21216c6a] + Preferences v1.2.2
  [0b7ba130] + Blosc_jll v1.21.1+0
  [0234f1f7] + HDF5_jll v1.12.0+1
  [5ced341a] + Lz4_jll v1.9.3+0
  [458c3c95] + OpenSSL_jll v1.1.10+0
  [3161d3a3] + Zstd_jll v1.5.0+0
  [8bb1440f] + DelimitedFiles
  [37e2e46d] + LinearAlgebra
  [a63ad114] + Mmap
  [1a1011a3] + SharedArrays
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [8dfed614] + Test
  [e66e0078] + CompilerSupportLibraries_jll
  [4536629a] + OpenBLAS_jll
  [8e850b90] + libblastrampoline_jll
Precompiling project...
  ✗ HDF5
  3 dependencies successfully precompiled in 2 seconds (18 already precompiled)
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package

(@v1.7) pkg> precompile
Precompiling project...
  ✗ HDF5
  0 dependencies successfully precompiled in 1 seconds (21 already precompiled)

ERROR: The following 1 direct dependency failed to precompile:

HDF5 [f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f]

Failed to precompile HDF5 [f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f] to /Users/tamasgal/.julia/compiled/v1.7/HDF5/jl_quHb64.
ERROR: LoadError: UndefVarError: libhdf5 not defined
Stacktrace:
  [1] top-level scope
    @ ~/.julia/packages/HDF5/pIJra/src/api_types.jl:119
  [2] include(mod::Module, _path::String)
    @ Base ./Base.jl:418
  [3] include(x::String)
    @ HDF5 ~/.julia/packages/HDF5/pIJra/src/HDF5.jl:1
  [4] top-level scope
    @ ~/.julia/packages/HDF5/pIJra/src/HDF5.jl:51
  [5] include
    @ ./Base.jl:418 [inlined]
  [6] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1318
  [7] top-level scope
    @ none:1
  [8] eval
    @ ./boot.jl:373 [inlined]
  [9] eval(x::Expr)
    @ Base.MainInclude ./client.jl:453
 [10] top-level scope
    @ none:1
in expression starting at /Users/tamasgal/.julia/packages/HDF5/pIJra/src/api_types.jl:119
in expression starting at /Users/tamasgal/.julia/packages/HDF5/pIJra/src/HDF5.jl:1

See JuliaIO/HDF5.jl#870 where the original discussion about HDF5 and M1 has started.

[fingolfin]

This means that HDF5_jll was not compiled for M1. Often it suffices to rebuild a Jll with new platforms (with a few wrinkles added; here is a typical example for what is required: https://github.com/JuliaPackaging/Yggdrasil/pull/3781/files). Unfortunately IIRC porting HDF5 is often non-trivial so it might take more effort here...

[fingolfin]

See also #2763

[giordano]

Hdf5 can't be cross-compiled, we have to use prebuilt binaries from somewhere else. As a stopgap solution you can use https://github.com/giordano/HDF5_jll.jl