Insufficient documentation on trust anchors

[ranok]

There appears to be a lack of documentation in how the machine state is rolled into the attestation JWT. From within the CVM, there is no mention of AMD SEV-SNP in the kernel cpuinfo or debug logs, so it does not appear that the system is actually encrypted. Usually to use AMD SEV-SNP, you take the nonce and have that signed (along with an optional key digest) by the AMD-signed CPU key that attests the state of the CVM. There is no /dev/sev present, and no service providing access to the AMD firmware to get a signed attestation blob.

I am struggling to see the value of the Azure CVM architecture (SEP-SEV is designed to remove the hypervisor from the TCB) if from within the guest there is no proof that it's running in a SEV-SNP environment, and the only signatures are coming from Azure itself. Pointers to documentation, or an example VHD that shows that the memory is encrypted would be much appreciated.

[jepio]

Hi @ranok, i came across this issue while working on integrating CVMs with kata confidential-containers, so I might have some answers that you're looking for.

There appears to be a lack of documentation in how the machine state is rolled into the attestation JWT. From within the CVM, there is no mention of AMD SEV-SNP in the kernel cpuinfo or debug logs, so it does not appear that the system is actually encrypted.

CVMs use a sub-feature of SEV-SNP (vTOM) which does not require as much support from the kernel, and that's why you won't see it mentioned in dmesg. There's a patchset being worked on in the kernel that will result in you seeing more of the standard messages in the log: https://lore.kernel.org/lkml/[email protected]/T/#u.
You can verify that the kernel is detecting memory encryption by checking for the following lines:

$ dmesg | grep -i isolation
[    0.000000] Hyper-V: Isolation Config: Group A 0x1, Group B 0xba2

Usually to use AMD SEV-SNP, you take the nonce and have that signed (along with an optional key digest) by the AMD-signed CPU key that attests the state of the CVM. There is no /dev/sev present, and no service providing access to the AMD firmware to get a signed attestation blob.

There is no /dev/sev-guest because the PSP communication is handled in the HCL, which runs at VMPL0 inside the enclave. You can fetch an SNP report from the HCL that includes a key held by the HCL (AK), you can also ask the HCL to generate TPM PCR quotes signed using the AK that include a nonce (and the nonce can include a key digest). You can validate the chain all the way up to AMD's root keys this way: nonce -> TPM quote -> AK -> SNP report -> VCEK -> ASK -> ARK.

I've tried to assemble an example script that illustrates this:

#!/bin/bash

set -xe
NONCE=cafe
# fetch VCEK/ASK/ARK
curl -H Metadata:true http://169.254.169.254/metadata/THIM/amd/certification > vcek
jq -r '.vcekCert , .certificateChain' >vcek.pem <vcek

# fetch HCL report
tpm2_nvread -C o 0x01400001 >snp_report.bin
# extract static SNP report
dd skip=32 bs=1 count=1184 if=snp_report.bin of=guest_report.bin
# extract runtime data (includes akpub.pem)
dd skip=1236 bs=1 if=snp_report.bin | tr -d '\0' >runtimedata
# read akpub.pem from TPM
tpm2_readpublic -c 0x81000003 -f pem -o akpub.pem
# generate TPM quote with nonce
tpm2_quote -c 0x81000003 -l sha256:4,7 -q "${NONCE}" -m quote_msg -s quote_sig
# verify TPM quote with nonce
tpm2_checkquote -m quote_msg  -s quote_sig -u akpub.pem -q "${NONCE}"

# validate SNP report
if command sevtool; then
  sevtool --validate_guest_report
fi

# check link between akpub.pem and snp
# extract report_data field
reportdata=$(dd skip=$(( 0x50 )) bs=1 count=32 if=guest_report.bin | xxd -ps -c 32)
# verify that runtimedata hash is included in SNP report
echo "${reportdata} runtimedata" | sha256sum -c -

# convert AK key in runtimedata to openssl text representation
python3 >runtimedata.pub.txt <<EOF
import json, base64

rundata = json.load(open('runtimedata'))
key = rundata['keys'][0]

f = lambda x: base64.urlsafe_b64decode(x + '===')
e = f(key['e'])
n = f(key['n'])

n2 = [0] + list(n)
n2 = ':'.join('{:02x}'.format(b) for b in n2)
grp = 15 * 3
n2 = [n2[i:i+grp] for i in range(0, len(n2), grp)]

e = '0x' + ''.join('{:02x}'.format(b) for b in e)
e = int(e, 16)
e_hex = '0x{:x}'.format(e)

print(f'RSA Public-Key: ({len(n) * 8} bit)')
print(f'Modulus:')
for line in n2:
    print(f'    {line}')
print(f'Exponent: {e} ({e_hex})')
EOF

openssl rsa -pubin -in akpub.pem -text -noout >akpub.txt

# compare key used to sign/verify TPM quote with key that
# is included in SNP report
diff -q runtimedata.pub.txt akpub.txt

I can also point you to this repository that I am working on for kata: https://github.com/kinvolk/azure-cvm-tooling/.

I am struggling to see the value of the Azure CVM architecture (SEP-SEV is designed to remove the hypervisor from the TCB) if from within the guest there is no proof that it's running in a SEV-SNP environment, and the only signatures are coming from Azure itself.

The hypervisor is indeed out of the TCB, but with CVMs you do have the HCL (/vTPM) in the TCB. So you have to trust the HCL, but you do get signatures that come from the CPU manufacturer to back all this up.

[fabienpe]

@jepio This is very useful! Thank you! However I think this is missing an important part: comparing the measurement in the signed attestation with a measurement that the guest can compute themselves. How to you get the UEFI image that is required to compute such measurement?

[jepio]

Unfortunately I don't have a good answer for you here. In this model the UEFI image is part of the guest TCB and not in the users hands, so you have to trust it. So you can't compute the UEFI measurement yourself.

You can either rely on MAA to validate the trustworthiness of a specific UEFI measurement, or take a look at idkeydigest in the raw SNP report. Idkeydigest contains the hash of a key used to sign the launch measurement before it is booted, and the PSP will validate the measurement signature before launching the guest. Idkeydigest would be a more stable trust anchor for the launch measurement.

The statements here provide some guidance: https://github.com/Azure/confidential-computing-cvm-guest-attestation/blob/main/cvm-guest-attestation.md#with-cvms-can-i-use-the-claim-x-ms-sevsnpvm-launchmeasurement-to-prove-code-integrity-over-the-initial-state-of-a-cvm.

[fabienpe]

Thanks.

I was not able to go much further than what you described. This confirms the limit of the attestation system.

[gram-signal]

@jepio Is the key for which idkeydigest is the digest available somewhere, and is there a certificate chain from it up to a key that has a longer lifetime than ~1y?