Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pre-built architectural enclaves contain redundant NixOS ld-linux path #1040

Open
berrange opened this issue Jul 15, 2024 · 2 comments · May be fixed by #1061
Open

Pre-built architectural enclaves contain redundant NixOS ld-linux path #1040

berrange opened this issue Jul 15, 2024 · 2 comments · May be fixed by #1061

Comments

@berrange
Copy link

Attempting to reproduce the architectural enclaves on a non-NixOS platform fails, despite having the same toolchain versions. Debugging revealed that this is a result of the built enclaves containing a reference to the NixOS ld-linux path.

This can be easily seen by querying the INTERP section of the binaries

$ readelf -l libsgx_*.signed.so | grep --after 2 INTERP
  INTERP         0x00000000000002a8 0x00000000000002a8 0x00000000000002a8
                 0x0000000000000054 0x0000000000000054  R      0x1
      [Requesting program interpreter: /nix/store/scd5n7xsn0hh0lvhhnycr9gx0h8xfzsl-glibc-2.34-210/lib/ld-linux-x86-64.so.2]
--
  INTERP         0x00000000000002a8 0x00000000000002a8 0x00000000000002a8
                 0x0000000000000054 0x0000000000000054  R      0x1
      [Requesting program interpreter: /nix/store/scd5n7xsn0hh0lvhhnycr9gx0h8xfzsl-glibc-2.34-210/lib/ld-linux-x86-64.so.2]
--
  INTERP         0x00000000000002a8 0x00000000000002a8 0x00000000000002a8
                 0x0000000000000054 0x0000000000000054  R      0x1
      [Requesting program interpreter: /nix/store/scd5n7xsn0hh0lvhhnycr9gx0h8xfzsl-glibc-2.34-210/lib/ld-linux-x86-64.so.2]
--
  INTERP         0x00000000000002a8 0x00000000000002a8 0x00000000000002a8
                 0x0000000000000054 0x0000000000000054  R      0x1
      [Requesting program interpreter: /nix/store/scd5n7xsn0hh0lvhhnycr9gx0h8xfzsl-glibc-2.34-210/lib/ld-linux-x86-64.so.2]

IIUC, the ld-linux interpretor is irrelevant for SGX, since the SGX SDK contains native logic for loading code into the enclaves. If this is indeed the case, then I'd request that the reference to the NixOX ld-linux is removed from future pre-built enclave binaries.

Looking at the build system this could potentially be achieved by extending the 'strip' command usage to also purge the '.interp' section of the ELF binary.

$(STRIP) --strip-unneeded --remove-section=.comment --remove-section=.note --remove-section=.interp $(SONAME)

Alternatively the -Wl,-dynamic-linker arg could be used to set the .interp section contents to some dummy value to indicate it is redundant / unused e.g.

LDTFLAGS  = -L$(SGX_LIB_DIR) -Wl,--whole-archive $(TRTSLIB) -Wl,--no-whole-archive \
            -Wl,--start-group $(EXTERNAL_LIB) -Wl,--end-group -Wl,--build-id       \
            -Wl,--version-script=$(ROOT_DIR)/build-scripts/enclave.lds $(ENCLAVE_LDFLAGS) \
            -Wl,dynamic-linker,/bin/false

There are probably other options too - I don't mind as long as this (apparently) redundant NixOS path can be removed from future pre-built enclave binaries.

@lzha101
Copy link
Contributor

lzha101 commented Aug 1, 2024

@berrange The AE reproducibility build is based on the provided docker file with Nix env. So we assume the reproducibility verification is also based on the same environment from the provided docker.
May I know if the NixOS ld-path is the only issue that prevent you from reproducing the AEs in non Nix env?

@berrange
Copy link
Author

berrange commented Aug 1, 2024

@berrange The AE reproducibility build is based on the provided docker file with Nix env. So we assume the reproducibility verification is also based on the same environment from the provided docker. May I know if the NixOS ld-path is the only issue that prevent you from reproducing the AEs in non Nix env?

I'm aiming to distribute the pre-built AE's in Fedora, but as a pre-requisite for proposing that be allowed by Fedora, I need to be able to reproduce their build process within Fedora, not NixOS. This isn't easy, as obviously you've never expected/intended for this to be done. The three biggest issues that affect reproducibility which I'm having to patch

  • This (apparently redundant) ld-linux path gets added to the AE .so binaries
  • The asserts in the cpp library get generated with an absolute path which I have to rewrite with -fmacro-prefix-map GCC arg
  • The NASM created .o files in ipp-crypto get an absolute path included which I have to rewrite by editting the ELF file to rewrite a symbol

I've not had time to investigate whether there's an easy way to avoid the absolute files paths in the last 2 cases, so I've not filed any issues requesting changes for those yet.

FYI, I've currently got reproducibility succeeding for SGX 2.22 release, and now working on 2.24 release for the qve.so AE that was re-issued. https://gitlab.com/berrange/fedora-sgx-ng-copr/-/tree/main/linux-sgx-enclaves-2_22?ref_type=heads Of course this all relies on me packaging the same versions of GCC, binutils, nasm that your NixOS env uses, which I've done in that same git repo.

berrange added a commit to berrange/linux-sgx that referenced this issue Oct 8, 2024
The enclaves are getting built as ELF executables, and thus the linker
will embed the current ld-linux.so path for the host OS environment
in the binary:

 $ readelf -a libsgx_pce.signed.so | grep interpreter
      [Requesting program interpreter: /nix/store/xmprbk52mlcdsljz66m8yf7cf0xf36n1-glibc-2.38-44/lib/ld-linux-x86-64.so.2]

The SGX enclaves are never loaded using ld-linux.so, as SGX has custom
code for loading enclaves in the required manner.

This embedded ld-linux.so path thus serves no functional purpose, while
also making it harder to do a reproducible build of the enclaves outside
of the NixOS environment.

This patch blanks out the NixOX interpretor path, by setting it to the
empty string.

Fixes: intel#1040
Signed-off-by: Daniel P. Berrangé <[email protected]>
berrange added a commit to berrange/SGXDataCenterAttestationPrimitives that referenced this issue Oct 8, 2024
The enclaves are getting built as ELF executables, and thus the linker
will embed the current ld-linux.so path for the host OS environment
in the binary:

 $ readelf -a libsgx_tdqe.signed.so | grep interpreter
      [Requesting program interpreter: /nix/store/xmprbk52mlcdsljz66m8yf7cf0xf36n1-glibc-2.38-44/lib/ld-linux-x86-64.so.2]

The SGX enclaves are never loaded using ld-linux.so, as SGX has custom
code for loading enclaves in the required manner.

This embedded ld-linux.so path thus serves no functional purpose, while
also making it harder to do a reproducible build of the enclaves outside
of the NixOS environment.

This patch blanks out the NixOX interpretor path, by setting it to the
empty string.

Related: intel/linux-sgx#1040
Signed-off-by: Daniel P. Berrangé <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants