change-key docs clarification

[InQuize]

Reading man page on zfs change-key and watching ZFS Native Encryption presentation, honestly brings a bunch of questions that I am struggling to search for on the web.

This docs quote:

If the user's key is compromised, zfs change-key does not necessarily protect existing or newly-written data from attack. Newly-written data will continue to be encrypted with the same master key as the existing data. The master key is compromised if an attacker obtains a user key and the corresponding wrapped master key. Currently, zfs change-key does not overwrite the previous wrapped master key on disk, so it is accessible via forensic analysis for an indeterminate length of time.

leaves an impression of change-key as some kind of placeholder function that brings no value until underlying mechanism is refined.
Trying to understand the threat surface here, assuming that user's wrapping key is compromised.

How exactly it is possible to further compromise wrapped master key as well?
If change-key does not overwrite the previous wrapped master key on disk, what is its purpose at all and where the new wrapped key goes?
Will there be accumulated a log of sorts of all the wrapped keys that were in use?
Why it behaves this way in the first place?

[rincebrain]

The dance native encryption does is that each dataset gets a key for its lifetime, and then that is encrypted with whatever you initially gave it as key material (a passphrase + pbkdf, or some raw bytes).

That is used, to my understanding, to square the circle around the fact that ZFS can't really go retroactively rewrite your data if you change the key out, so you do this dance to swap the key that decrypts the key that the tooling doesn't give you an easy way to get at to give you some level of key changing functionality on something that assumes your data is immutable until deleted once written.

The man page is basically saying if you have the previously encrypted version of the master key and know what was used to decrypt that, you can still derive the master key, since the actual master key doesn't change. I believe the reason it doesn't make any promises about when that goes away on disk is that there's not really any logic in the ZFS code to control when that happens, since there's not really a good way for the "FS" part of the code to say "overwrite these specific parts of the object, please, once it's no longer in use", since once it's no longer in use, you no longer have anything describing where it used to be.

I think it'd be a little exciting to convince the code to do it, as well, since you'd basically want a background thread that waits until nobody else is looking at the object, then scribbles over it and frees it, while also avoiding something like zpool scrub thinking the scribbled over bits are an error. I don't think you can't do it, but I think it'd be invasive to try and do "correctly", versus just suggesting you use something like zpool initialize to minimize that risk if that's a thing you're concerned about.

[rincebrain]

That is my understanding, yes.

[InQuize]

Both your first reply and docs quote make a lot more sense now. Much appreciated!

So, in the event of compromising KEK, but being certain that encrypted DEK was not leaked, we simply change KEK.

If there is a chance, than the only option to be sure is to rotate DEK, which is only possible by creating new encryption root (dataset) from scratch with subsequent data migration. Which won't be an option in-place in case of root dataset, and result in a new pool or move all encrypted data to new child dataset(s).

[rincebrain]

Yes, though for anyone reading, I would like to make explicit that AIUI each dataset gets its own DEK even if they have a shared encryptionroot, encryptionroot just determines how you derive the KEK - otherwise you couldn't ever change the hierarchy of encryption roots afterward, among other problems.

One of a number of reasons that I would not suggest keeping data in the root dataset.

[InQuize]

Does not seem right. On the same man page:

After entering an encryption key, the created dataset will become an encryption root. Any descendant datasets will inherit their encryption key from the encryption root by default, meaning that loading, unloading, or changing the key for the encryption root will implicitly do the same for all inheriting datasets. If this inheritance is not desired, simply supply a keyformat when creating the child dataset or use zfs change-key to break an existing relationship, creating a new encryption root on the child. Note that ...

[rincebrain]

Yes, that's correct. I think the language there is ambiguous because it doesn't really want to explain that it's encrypting/decrypting a second key, but.

encryptionroot means that they share how they derive the KEK from their DEK, and where you get the material to decrypt the KEKs. But if they didn't have independent DEKs, you couldn't zfs change-key and sever that parent-child relationship later, since the DEKs are what you actually use to do the encrypting, and you run into basically the same problem as why the DEK doesn't rotate when you change-key.

Phrased differently, encryptionroot specifies which dataset has the settings for turning DEKs into KEKs and vice-versa, but they still all get their own DEK.

Does that make sense?