set JULIA_CPU_TARGET to avoid compilation overhead

[BioTurboNick]

If someone wishes to use this container and pre-load packages for their own, they may find that every time they run the container remotely, Julia spends time compiling before anything can happen.

This is because on an individual system Julia will just compile code for that specific architecture. A remote system with a slightly different architecture will trigger re-compilation.

To make the compilation that happens during container building stick, the JULIA_CPU_TARGET environment variable needs to be set to the generic string that is used to build Julia in the first place:

ENV JULIA_CPU_TARGET generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)

See: https://docs.julialang.org/en/v1/devdocs/sysimg/#Specifying-multiple-system-image-targets

[giordano]

For reference, these are the targets settings for all the architectures: https://github.com/JuliaCI/julia-buildkite/blob/9c9f7d324c94130f36e52a386274aa250baef495/utilities/build_envs.sh#L20-L73

[tianon]

Oh, that's really interesting, thanks for the useful links!

My biggest question here is whether this is something we should be setting to something sanely generic by default, or whether this is something we should instead document for users to choose for themselves?

(I imagine the default behavior wasn't chosen arbitrarily, but probably also wasn't chosen for the use case of sharing the built results like Docker images enable/encourage. 😅)

[frankier]

I think this string compiles fat binaries which contain generic, sandybridge and haswell versions. Since this is what Julia itself is compiled with, this is really just making it so that stuff used at build time is reused rather than invalidated because it doesn't have the exact same CPU as the system the user is eventually running the image on. Thus this is exactly what is wanted for the case of sharing build results as in the case of publicly available Docker images.

The alternative use case is someone deploying a Docker image to a large number of identical machines. In this case they would set a very specific arch for these machines. Someone doing this probably knows what they are doing and might even rebuild Julia themselves and so not even use a base image like this one.