Installing HDF5.jl on ARM M1

[ryan-a-anderson]

Using Julia v1.8, having major issues using the HDF5 package. Using it as a dependency for another package, LearningAlgebraicVarieties.

Output of ] status:

(@v1.8) pkg> status
Status `~/.julia/environments/v1.8/Project.toml`
⌅ [f67ccb44] HDF5 v0.13.7
⌃ [033835bb] JLD2 v0.4.3
  [99c59dee] LearningAlgebraicVarieties v0.1.1
  [21216c6a] Preferences v1.4.1
⌅ [0234f1f7] HDF5_jll v1.10.5+7
Info Packages marked with ⌃ and ⌅ have new versions available, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated`

Output of ] status -m:

(@v1.8) pkg> status -m
Status `~/.julia/environments/v1.8/Manifest.toml`
⌅ [7d9fca2a] Arpack v0.4.0
  [bf4720bc] AssetRegistry v0.1.0
  [d1d4a3ce] BitFlags v0.1.8
  [ad839575] Blink v0.12.8
  [a74b3585] Blosc v0.7.3
⌅ [336ed68f] CSV v0.7.10
⌅ [324d7699] CategoricalArrays v0.8.3
  [d360d2e6] ChainRulesCore v1.19.0
  [9e997f8a] ChangesOfVariables v0.1.8
⌅ [aaaa29a8] Clustering v0.14.4
  [944b1d66] CodecZlib v0.7.3
  [35d6a980] ColorSchemes v3.24.0
  [3da002f7] ColorTypes v0.11.4
  [c3611d14] ColorVectorSpace v0.10.0
  [5ae59095] Colors v0.12.10
  [861a8166] Combinatorics v1.0.2
⌅ [34da2185] Compat v2.2.1
  [f0e56b4a] ConcurrentUtilities v2.3.0
  [187b0558] ConstructionBase v1.5.4
  [d38c429a] Contour v0.6.2
  [9a962f9c] DataAPI v1.15.0
⌅ [a93c6f00] DataFrames v0.21.8
⌃ [864edb3b] DataStructures v0.17.20
  [e2d170a0] DataValueInterfaces v1.0.0
⌅ [b4f34e82] Distances v0.9.2
⌅ [ffbed154] DocStringExtensions v0.8.6
⌅ [7c1d4256] DynamicPolynomials v0.3.21
⌃ [9c0f25c4] Eirene v1.3.5
  [460bff9d] ExceptionUnwrapping v0.1.9
  [c87230d0] FFMPEG v0.4.1
  [5789e2e9] FileIO v1.16.1
  [53c48c17] FixedPointNumbers v0.8.4
  [3dd14ad9] FixedPolynomials v0.4.0
  [59287772] Formatting v0.4.2
  [de31a74c] FunctionalCollections v0.5.0
⌅ [28b8d3ca] GR v0.72.10
  [42e2da0e] Grisu v1.0.2
⌅ [f67ccb44] HDF5 v0.13.7
  [cd3eb016] HTTP v1.10.1
  [9fb69e20] Hiccup v0.2.2
⌅ [e91730f6] Hungarian v0.6.0
  [3587e190] InverseFunctions v0.1.12
  [41ab1584] InvertedIndices v1.3.0
  [92d709cd] IrrationalConstants v0.2.2
  [82899510] IteratorInterfaceExtensions v1.0.0
⌅ [4138dd39] JLD v0.10.0
⌃ [033835bb] JLD2 v0.4.3
  [1019f520] JLFzf v0.1.7
  [692b3bcd] JLLWrappers v1.5.0
  [97c1335a] JSExpr v0.5.4
  [682c06a0] JSON v0.21.4
  [b964fa9f] LaTeXStrings v1.3.1
  [23fbe1c1] Latexify v0.16.1
  [50d2b5c4] Lazy v0.15.1
  [99c59dee] LearningAlgebraicVarieties v0.1.1
  [2ab3a3ac] LogExpFunctions v0.3.26
  [e6f89c97] LoggingExtras v1.0.3
  [1914dd2f] MacroTools v0.5.12
  [739be429] MbedTLS v1.1.9
  [442fdcdd] Measures v0.3.2
⌅ [e1d29d7a] Missings v0.4.5
⌅ [102ac46a] MultivariatePolynomials v0.3.18
⌅ [6f286f6a] MultivariateStats v0.7.0
  [ffc61752] Mustache v1.0.19
⌅ [d8a4904e] MutableArithmetics v0.2.22
  [a975b10e] Mux v1.0.1
  [77ba4419] NaNMath v1.0.2
  [b8a86587] NearestNeighbors v0.4.16
  [510215fc] Observables v0.5.5
  [4d8831e6] OpenSSL v1.4.1
  [bac558e1] OrderedCollections v1.6.3
⌅ [69de0a69] Parsers v1.1.2
  [fa939f87] Pidfile v1.3.0
  [b98c9c47] Pipe v1.3.0
  [ccf2f8ad] PlotThemes v3.1.0
⌃ [995b91a9] PlotUtils v1.2.0
⌅ [a03496cd] PlotlyBase v0.5.4
⌅ [f0f68f2c] PlotlyJS v0.14.1
  [91a5bcdd] Plots v1.39.0
⌅ [2dfb63ee] PooledArrays v0.5.3
  [aea7be01] PrecompileTools v1.2.0
  [21216c6a] Preferences v1.4.1
  [3cdcf5f2] RecipesBase v1.3.4
  [01d81517] RecipesPipeline v0.6.12
⌅ [189a3867] Reexport v0.2.0
  [05181044] RelocatableFolders v1.0.1
  [ae029012] Requires v1.3.0
⌅ [af85af4c] RowEchelon v0.1.1
  [6c6a2e73] Scratch v1.2.1
  [91c51154] SentinelArrays v1.4.1
  [992d4aef] Showoff v1.0.3
  [777ac1f9] SimpleBufferStream v1.1.0
⌅ [a2af1166] SortingAlgorithms v0.3.2
  [90137ffa] StaticArrays v1.8.1
  [1e83bf80] StaticArraysCore v1.4.2
  [82ae8749] StatsAPI v1.7.0
⌅ [2913bbd2] StatsBase v0.33.21
  [856f2bd8] StructTypes v1.10.0
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.11.1
  [62fd8b95] TensorCore v0.1.1
  [3bb67fe8] TranscodingStreams v0.10.2
  [5c2747f8] URIs v1.5.1
  [1cfade01] UnicodeFun v0.4.1
  [1986cc42] Unitful v1.19.0
  [45397f5d] UnitfulLatexify v1.6.3
  [41fe7b60] Unzip v0.2.0
  [0f1e0344] WebIO v0.8.21
  [104b5d7c] WebSockets v1.6.0
  [cc8bc4a8] Widgets v0.6.6
⌅ [68821587] Arpack_jll v3.5.1+1
  [0b7ba130] Blosc_jll v1.21.5+0
  [6e34b625] Bzip2_jll v1.0.8+0
  [83423d85] Cairo_jll v1.16.1+1
  [2702e6a9] EpollShim_jll v0.0.20230411+0
  [2e619515] Expat_jll v2.5.0+0
  [b22a6f82] FFMPEG_jll v4.4.4+1
  [a3f928ae] Fontconfig_jll v2.13.93+0
  [d7e528f0] FreeType2_jll v2.13.1+0
  [559328eb] FriBidi_jll v1.0.10+0
  [0656b61e] GLFW_jll v3.3.9+0
⌅ [d2c73de3] GR_jll v0.72.10+0
  [78b55507] Gettext_jll v0.21.0+0
  [7746bdde] Glib_jll v2.76.5+0
  [3b182d85] Graphite2_jll v1.3.14+0
⌅ [0234f1f7] HDF5_jll v1.10.5+7
  [2e76f6c2] HarfBuzz_jll v2.8.1+1
  [aacddb02] JpegTurbo_jll v3.0.1+0
⌅ [f7e6163d] Kaleido_jll v0.1.0+0
  [c1c5ebd0] LAME_jll v3.100.1+0
  [88015f11] LERC_jll v3.0.0+1
  [1d63c593] LLVMOpenMP_jll v15.0.7+0
  [dd4b983a] LZO_jll v2.10.1+0
⌅ [e9f186c6] Libffi_jll v3.2.2+1
  [d4300ac3] Libgcrypt_jll v1.8.7+0
  [7e76a0d4] Libglvnd_jll v1.6.0+0
  [7add5ba3] Libgpg_error_jll v1.42.0+0
  [94ce4f54] Libiconv_jll v1.17.0+0
  [4b2f31a3] Libmount_jll v2.35.0+0
⌅ [89763e89] Libtiff_jll v4.5.1+1
  [38a345b3] Libuuid_jll v2.36.0+0
  [5ced341a] Lz4_jll v1.9.4+0
  [e7412a2a] Ogg_jll v1.3.5+1
  [458c3c95] OpenSSL_jll v3.0.12+0
  [91d4177d] Opus_jll v1.3.2+0
  [30392449] Pixman_jll v0.42.2+0
  [c0090381] Qt6Base_jll v6.5.3+1
  [a44049a8] Vulkan_Loader_jll v1.3.243+0
  [a2964d1f] Wayland_jll v1.21.0+1
  [2381bf8a] Wayland_protocols_jll v1.25.0+0
  [02c8fc9c] XML2_jll v2.12.2+0
  [aed1982a] XSLT_jll v1.1.34+0
  [ffd25f8a] XZ_jll v5.4.5+0
  [f67eecfb] Xorg_libICE_jll v1.0.10+1
  [c834827a] Xorg_libSM_jll v1.2.3+0
  [4f6342f7] Xorg_libX11_jll v1.8.6+0
  [0c0b7dd1] Xorg_libXau_jll v1.0.11+0
  [935fb764] Xorg_libXcursor_jll v1.2.0+4
  [a3789734] Xorg_libXdmcp_jll v1.1.4+0
  [1082639a] Xorg_libXext_jll v1.3.4+4
  [d091e8ba] Xorg_libXfixes_jll v5.0.3+4
  [a51aa0fd] Xorg_libXi_jll v1.7.10+4
  [d1454406] Xorg_libXinerama_jll v1.1.4+4
  [ec84b674] Xorg_libXrandr_jll v1.5.2+4
  [ea2f1a96] Xorg_libXrender_jll v0.9.10+4
  [14d82f49] Xorg_libpthread_stubs_jll v0.1.1+0
  [c7cfdc94] Xorg_libxcb_jll v1.15.0+0
  [cc61e674] Xorg_libxkbfile_jll v1.1.2+0
  [e920d4aa] Xorg_xcb_util_cursor_jll v0.1.4+0
  [12413925] Xorg_xcb_util_image_jll v0.4.0+1
  [2def613f] Xorg_xcb_util_jll v0.4.0+1
  [975044d2] Xorg_xcb_util_keysyms_jll v0.4.0+1
  [0d47668e] Xorg_xcb_util_renderutil_jll v0.3.9+1
  [c22f9ab0] Xorg_xcb_util_wm_jll v0.4.1+1
  [35661453] Xorg_xkbcomp_jll v1.4.6+0
  [33bec58e] Xorg_xkeyboard_config_jll v2.39.0+0
  [c5fb5394] Xorg_xtrans_jll v1.5.0+0
  [3161d3a3] Zstd_jll v1.5.5+0
  [35ca27e7] eudev_jll v3.2.9+0
  [214eeab7] fzf_jll v0.43.0+0
  [1a1c6b14] gperf_jll v3.1.1+0
  [a4ae2306] libaom_jll v3.4.0+0
  [0ac62f75] libass_jll v0.15.1+0
  [2db6ffa8] libevdev_jll v1.11.0+0
  [f638f0a6] libfdk_aac_jll v2.0.2+0
  [36db933b] libinput_jll v1.18.0+0
  [b53b4c65] libpng_jll v1.6.40+0
  [f27f6e37] libvorbis_jll v1.3.7+1
  [009596ad] mtdev_jll v1.1.6+0
  [1270edf5] x264_jll v2021.5.5+0
  [dfaa095f] x265_jll v3.5.0+0
  [d8fb68d0] xkbcommon_jll v1.4.1+1
  [0dad84c5] ArgTools v1.1.1
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8bb1440f] DelimitedFiles
  [8ba89e20] Distributed
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching
  [9fa8497b] Future
  [b77e0a4c] InteractiveUtils
  [b27032c2] LibCURL v0.6.3
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.8.0
  [de0858da] Printf
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization
  [1a1011a3] SharedArrays
  [6462fe0b] Sockets
  [2f01184e] SparseArrays
  [10745b16] Statistics
  [fa267f1f] TOML v1.0.0
  [a4e569a6] Tar v1.10.1
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
  [e66e0078] CompilerSupportLibraries_jll v0.5.2+0
  [deac9b47] LibCURL_jll v7.84.0+0
  [29816b5a] LibSSH2_jll v1.10.2+0
  [c8ffd9c3] MbedTLS_jll v2.28.0+0
  [14a3606d] MozillaCACerts_jll v2022.2.1
  [4536629a] OpenBLAS_jll v0.3.20+0
  [05823500] OpenLibm_jll v0.8.1+0
  [efcefdf7] PCRE2_jll v10.40.0+0
  [83775a58] Zlib_jll v1.2.12+3
  [8e850b90] libblastrampoline_jll v5.1.1+0
  [8e850ede] nghttp2_jll v1.48.0+0
  [3f19e933] p7zip_jll v17.4.0+0
Info Packages marked with ⌃ and ⌅ have new versions available, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated -m`

[mkitti]

Is there a reason we are using old versions of the packages? The main problem here is that you are trying to use HDF5_jll 1.10.5, which is from 2019, before the commercial release of the M1.

[mkitti]

I Investigated this. There are some old and frankly esoteric package compatibility settings with the packages involved. I'm going to attempt to update them.

[mkitti]

I have created two requests to update dependency compatibility sections of LearningAlgebraicVarieties.jl and Eirene.jl

In the meantime, I recommend the following course of action.

julia> pkg"activate --temp" # optional, if you just want to test it first
  Activating new project at `/tmp/jl_Fm1Lg0`

julia> pkg"add https://github.com/mkitti/Eirene.jl#mkitti-update-1_4_0 https://github.com/mkitti/LearningAlgebraicVarieties.jl#master HDF5 HDF5_jll JLD JLD2 Preferences"

julia> pkg"status"
Status `/tmp/jl_Fm1Lg0/Project.toml`
  [9c0f25c4] Eirene v1.4.0 `https://github.com/mkitti/Eirene.jl#mkitti-update-1_4_0`
  [f67ccb44] HDF5 v0.17.1
  [4138dd39] JLD v0.13.4
  [033835bb] JLD2 v0.4.40
  [99c59dee] LearningAlgebraicVarieties v0.3.0 `https://github.com/mkitti/LearningAlgebraicVarieties.jl#master`
  [21216c6a] Preferences v1.4.1

julia> using HDF5_jll

julia> HDF5_jll.libhdf5
"/home/mkitti/.julia/artifacts/2829a1f6a9ca59e5b9b53f52fa6519da9c9fd7d3/lib/libhdf5.so"

julia> HDF5_jll.libhdf5_handle
Ptr{Nothing} @0x0000000008cecc10

Please let me know if this works for you. I have tested this on Julia 1.10.0.

[ryan-a-anderson]

Amazing, that worked perfectly! Thank you very much!!

[mkitti]

I would appreciate if you could comment on the following pull requests that these changes were helpful to you.

• Update compat PBrdng/LearningAlgebraicVarieties.jl#16
• Update code for new versions of dependencies henselman-petrusek/Eirene.jl#49