Certificate usage support client side (#912 with small modification)

[sbernard31]

This is mainly #912 with small cosmetic modification.

Most changes are in e786afa.
If PR is approved, I will squash this commit with 4601d1e

[sbernard31]

This code sounds not so easy to read.
Maybe it deserves to be rewritten. (lot of break, lot of hidden exception)
Is it something you copy/paste ?

Do we really need this maxDepth check ?

[dachaac]

I wrote it my self.

Logic is kinda simple -- just find previous cert from trust store and make sure it is the right cert (verify check) and so that it is valid.

Verify check is important when new CA is issued with same name or CA certificate is re-keyed and/or re-issued. It is quite common to start transition period to new CA in controlled fashion. Eg. first distributing new CA's and then when distributed then start changing server certs.

Stop when we found root ca or if some-one made circular chain then protect against that with depth limit.

If everything went well and something was modified -> return modified chain.

If there was an error or no modifications return original chain.

Java crypto library has done most heavy lifting on certificate checks so we can do simple checks here.

[sbernard31]

OK I think this is the last point to address. 🙂

So this code do both thing :

• check for chain loop
• verify certificate signature

(maybe this should be separated in 2 functions)

Tell me if I'm wrong :

1. we check loop only for certificate in truststore ?
2. About verification this should not be done in CertPathUtil.validateCertificatePath(truncateCertificatePath, certPath, trustedCertificates); ?

(I'm pretty sure I missed something 🤔)

[dachaac]

It reconstructs the missing chain elements from trust store. And part of that process is verifying that correct certificate is picked from trust store.

[dachaac]

CertPathUtil does not pick certificates properly from trust store if there are multiple CA certs there with same subject DN.

If we make sure that it is correct before going there then it hopefully copes with that.

But that function has:

		CertPathValidator validator = CertPathValidator.getInstance("PKIX");
		PKIXParameters params = new PKIXParameters(trustAnchors);
		// TODO: implement alternative means of revocation checking
		params.setRevocationEnabled(false);
		validator.validate(verifyCertPath, params);

Which does the PKI validation magic within Java JRE which is what we want to do as that is rather complex beast.

And this validator is also a picky -- eg. there should not be same certificates in trust store vs. in chain.

[sbernard31]

You didn't answer about :

1. we check loop only for certificate in truststore ?

[dachaac]

Hmm? Perhaps I didn't understood the question.

Let me try to describe it differently.

1. We start by getting list of certificates from actual DTLS connection:

         List<? extends Certificate> certificates = certPath.getCertificates();

Note: first certificate in the list is end entity (server) certificate and then next certificates are possibly the chain certificates if the server is sending them.

2. We setup an chain as ArrayList and populate it with certificates so that we can add more certs there

Now at this point chainand certificates are equals assuming all certificates in certificateswere X509Certificates.

3. Then we have while loop with max depth check

3a. We get last certificate in array list. eg. the certificate which might be missing next in chain.

X509Certificate cert = chain.get(chain.size() - 1);

3b. We check that if the certificate is root CA or not (issuer equals to subject)

X500Principal issuer = cert.getIssuerX500Principal();

                // Check if we found the root CA
                if (issuer.equals(cert.getSubjectX500Principal()))
                    break;

If it is then we have completed our job. This should be the normal exit path.

3c. Now we know that we do not have full chain.

3d. So let's try to find missing certificates from trust store (trustedCertificates):

                for (X509Certificate caCert : trustedCertificates) {
                    X500Principal subject = caCert.getSubjectX500Principal();
                    if (subject.equals(issuer)) {
                        try {
                            cert.verify(caCert.getPublicKey());
                            caCert.checkValidity();
                            chain.add(caCert);
                            modified = true;
                            found = true;
                            break;
                        } catch (NoSuchAlgorithmException | InvalidKeyException | NoSuchProviderException
                                | SignatureException e) {
                            // Skip invalid certificates
                        }
                    }
                }

Now this code here is searching all trust store entries and getting subject DN.

If subject DN is the same as issuer's subject DN (3b) then we might have located the missing CA certificate.

In order to make sure that it is:

• We verify that issuer has actually signed the cert with Certificate.verify():

cert.verify(caCert.getPublicKey());

• and then also make sure that it has not expired X509Certificate.checkValidity().

caCert.checkValidity();

If checks pass we add the newly discovered issuer certificate to the end of certificate chain (chain):

chain.add(caCert);

In same go we mark that we have modified the list and that we have found the certificate.

3e. If we didn't find certificate we know that chain is incomplete and there is no reason to try another round.

                if (!found)
                    break;

3f. If we did find the certificate just loop again in while

4. Now we are out of while loop

5. If we did not modify the chain at all or we detected that there were too many loop in the while we return with original certificate path.

            if (!modified || maxDepth == 0)
                return certPath;

6. If we did modify then it construct a new certpath and returns it

            CertificateFactory factory = CertificateFactory.getInstance("X.509");
            return factory.generateCertPath(chain);

7. If there were any exceptions or so hide them and just return original certPath

Does this help to understand what happens there?

[sbernard31]

Thx for the detailed explanation.

Let me try to explain what I try to say by : "we check loop only for certificate in truststore ?"

I'm talking about depth guard.
I understand that :

1. we start to check by the end of the initial certpath. X509Certificate cert = chain.get(chain.size() - 1);
2. then we only add certificate from trustedCertificate (chain.add(caCert);)

So if we discover a loop, it seems to me that it was in the truststore?

[boaks]

So, the strategy seems to complete the path with certificates from the truststore.

For trusting intermediate CAs (one within the original path), that requires still to truncate that path before the PKIX CertPathValidator.

Also, AFAIK, the PKIX CertPathValidator searches for the issuer of the last certificate. If that is not a self-signed root, I wonder, if that algo doesn't add one trusted certificate too much?

Assuming multiple CA-Certificates with the same DN requires then a redesign in CertPathUtil.validateCertificatePath. So, wait for more info about that ...

[dachaac]

So if we discover a loop, it seems to me that it was in the truststore?

I was probably thinking originally the whole chain in here.

But you are true -- the certificate loop would be in trust store.

Now the question would be how it would end up there and shall we have a safe guard in place.

Thinking aloud for possible scenarios for certificate loop:

• If you would have trusted certs from Mozilla or so I would expect them to do also good job in this sense so that would not be an vector.
• One could inject corporate CA's to servers -- not a problem for (embedded) client.
• One could use UI of device to configure additional trusts -- out-of-scope.
• When performing bootstrap one can inject new Server Identity and that would be dropped to trust store -> compromised server -- In this case it probably gets sorted out and clients need to have normal ways to get back in (eg. not get stuck).
• When performing CoAP-EST's CA certs request -- same compromise scenario as above.
• Someone hacked the device -- bigger problem already -- out-of-scope.
• Some security tester proofing a point

So I suppose only scenarios we have here is:

• sanity check that we don't accidentally go to infinite loop. I have never seen 32 deep or even 8 deep CA chains. So value should be safe bet.
• server compromise and making sure that feet of devices can automatically get back in. You probably need to issue new certs in this kind of scenario triggering clients to connect again to bootstrap server and hope that you didn't need to change trust point for bootstrap.

[boaks]

I tried to have a fast look on it.

If CAs a rolled out before the node's certificates are update, then the test in Californium about not to have duplicates in the truststore based on the DN will fail.

@dachaac any links for that practice? if so, I guess, I need to adapt Californium as well.

[dachaac]

I tried to have a fast look on it.

If CAs a rolled out before the node's certificates are update, then the test in Californium about not to have duplicates in the truststore based on the DN will fail.

@dachaac any links for that practice? if so, I guess, I need to adapt Californium as well.

I have seen in one company's Windows trust store their Root CA and then two intermediate CA's with same name. How they did end up in there I cannot say.

Reason probably for using the same name might come from fact that many software only care about trust store as transparent thing and then subject DNs. Eg. they may say that it is OK to have this issuer's subject DN.

But possible scenarios why one could have duplicate subject DNs.

• Re-issuance of server certificate (end entity)
  • broken hardware
  • lost key or so...
  • however not really a scenario for this code
• Re-issueance if intermediate CA
  • server could have been compromised -> new key to be generated
  • server moved to new hardware and hardware protected token could not be moved
  • hardware token broken and there was no backup procedure
  • security audit happened -- and new measures had to be implemented
• Re-issuance of root CA
  • company started with light weight setup and understood that it was not good. Graceful transition from old world to new world
  • server could have been compromised -> new key to be generated
  • hardware token broken and there was no backup procedure
  • security audit happened -- new offline root with HSM implemented
• Key rollover
  • RFC 4210 - 4.4. Root CA Key Update
  • RFC 6489 - Certification Authority (CA) Key Rollover in the Resource Public Key Infrastructure (RPKI)
  • There may be a policy to perform key rollovers like every 10 years or so.
  • If you don't have access anymore to (old) private key then you just need to distribute new CA certs.
  • I have seen somewhere a table which indicated that not all software support the cross-signing rollover as defined in those RFCs. (if I recall correctly IIS was on the list) -> if you happen to have such in use then you just need to issue new CA without above rollover procedure.

[boaks]

@dachaac

Thanks for the links! A first web-search yesterday points to the same.
What I not fully get, why is it important to complete the "path" from the certificates of the trust store?
I understand, that it is important to "chose" the right certificates (key), when multiple certificates are available for the same subject (DN), but beside of that, I wonder, why that completion is relevant. Can you help me out?

[dachaac]

@dachaac

Thanks for the links! A first web-search yesterday points to the same.
What I not fully get, why is it important to complete the "path" from the certificates of the trust store?
I understand, that it is important to "chose" the right certificates (key), when multiple certificates are available for the same subject (DN), but beside of that, I wonder, why that completion is relevant. Can you help me out?

When everything would be set up properly and both servers and clients would be sending their full chain (-root) nicely then there should not be the need.

Scenarios where it could be optimized out is LTE-NB1 (NB-IoT) or so networks where you want to save sent bytes ('old world problems'). Reason could also be to save battery energy or so. In here the client has been pre-deployed part of the chain out-of-bound. This becomes larger problem if you use RSA instead of ECDSA. Some server software deployment models lose the DTLS session cache on updates which requires one to re-initiate DTLS session more often. Some LTE-NB1 SIM plans have quite small byte budgets / some interval.

AWS IoT Core works in a way that you cannot even register your root CA in their systems instead you need to register intermediate CA. In this kind of scenario you might have root CA and intermediate CA installed. (this I believe is more on category verifying client when they connect).

LWM2M 1.0 specification (and partly even 1.1) have a chain problem so only way to workaround that is to not send chains and have intermediate CA also distributed so that you can do chain validation (this is more client side issue).

And then like in the above one company scenario intermediate CA was deployed in the Windows certificate store and the servers were improperly configured not to send their chains. This scenario could actually quite easily happen as it is not "straight forward" to use eg. openssl to send the chain. Also people seem to just hack their way and just distribute the intermediate CA as they were not wholly aware how it should be properly configured in servers.

[boaks]

I still don't understand, why the "full chain" is required, if a trust for a intermediate certificate is available.

Sure, if the paths are truncated you generally may fill them up again, but why?

[dachaac]

I still don't understand, why the "full chain" is required, if a trust for a intermediate certificate is available.

Sure, if the paths are truncated you generally may fill them up again, but why?

The test code:

    /**
     * Test scenario:
     * - Certificate Usage = CA constraint
     * - Server Certificate = root CA certificate (not end-entity certificate)
     * - Server's TLS Server Certificate = intermediate signed certificate (with SAN DNS entry)
     * - Server accepts client
     * - Client Trust Store = root CA
     *
     * Expected outcome:
     * - Client is able to connect (root CA cert is part of the chain)
     */
    @Test
    public void registered_device_with_x509cert_to_server_with_x509cert_rootca_certificate_ca_domain_root_ca_given()

In here if you change expandCertPath() to just return input (eg. NOP operation).

In this case we have in certPath (eg. messageChain):

• Server's certificate
• Intermediate CA

Now verification is for Root CA and it cannot be found from the certPath and connection is not allowed.

If we do the expandCertPath() normally then we have certPath:

• Server's certificate
• Intermediate CA
• Root CA

And now the matcher finds the match and allows connection to happen.

[boaks]

If the Intermediate CA is "issued" by the Root CA, it's not required to add the issuer's certificate (Root CA) to the path. At least, in my experience.

Assuming, there are more than 1 Root CA with the same DN (or at any other trusted CA level), then sure, the right certificate must be selected as trust-anchor. I'm not sure, if the PKIX does it on it's own, or a "preselection" is required.

But this is about the right trust-anchor, not about completion of the certificate path.

[dachaac]

If the Intermediate CA is "issued" by the Root CA, it's not required to add the issuer's certificate (Root CA) to the path. At least, in my experience.

Assuming, there are more than 1 Root CA with the same DN (or at any other trusted CA level), then sure, the right certificate must be selected as trust-anchor. I'm not sure, if the PKIX does it on it's own, or a "preselection" is required.

But this is about the right trust-anchor, not about completion of the certificate path.

That's the reason PKIX path validation is separate from CA constraint path matcher.

In CA constraint trust store consist of:

• System/Application trust store
• (Server Public Identity) Certificate provided by LWM2M

It is specified as:

 * 0 -- Certificate usage 0 is used to specify a CA certificate, or
 *       the public key of such a certificate, that MUST be found in any of
 *       the PKIX certification paths for the end entity certificate given
 *       by the server in TLS.  This certificate usage is sometimes
 *       referred to as "CA constraint" because it limits which CA can be
 *       used to issue certificates for a given service on a host.  The
 *       presented certificate MUST pass PKIX certification path
 *       validation, and a CA certificate that matches the TLSA record MUST
 *       be included as part of a valid certification path.  Because this
 *       certificate usage allows both trust anchors and CA certificates,
 *       the certificate might or might not have the basicConstraints
 *       extension present.

Scenario A:

If server sends:

• TLS Server Certificate

In trust store:

• Intermediate CA
• Root CA

CA constraint certificate:

• Root CA

Then CertPathUtil.validateCertificatePath() would just have TLS Server Certificate in the chain and then trust store is provided to JRE's PKIX validation and that would pass.

Without path completion we cannot match for Root CA at all in this scenario.

Scenario B:

If server sends:

• TLS Server Certificate

In trust store:

• Intermediate CA
• Root CA

CA constraint certificate:

• Intermediate CA

Then CertPathUtil.validateCertificatePath() would just have TLS Server Certificate in the chain and then trust store is provided to JRE's PKIX validation and that would pass.

In order to match Intermediate CA we would need to search for intermediate CA from trust store / CA constraint certificate. So this becomes more or less the path expansion case.

Scenario C:

If server sends:

• TLS Server Certificate
• Intermediate CA

In trust store:

• Root CA

CA constraint certificate:

• Root CA

Then CertPathUtil.validateCertificatePath() would have TLS Server Certificate and Intermediate CA in the chain and then trust store is provided to JRE's PKIX validation and that would pass.

In order to match Root CA we would need to search for Root CA from trust store / CA constraint certificate. So this becomes more or less the path expansion case.

Scenario D:

If server sends:

• TLS Server Certificate
• Intermediate CA

In trust store:

• Root CA

CA constraint certificate:

• Intermediate CA

Then CertPathUtil.validateCertificatePath() would have TLS Server Certificate and Intermediate CA in the chain and then trust store is provided to JRE's PKIX validation and that would pass.

In this case the Intermediate CA would be part of cert chain and it would pass.

Scenario E:

If server sends:

• TLS Server Certificate
• Intermediate CA

In trust store:

• Root CA
• Intermediate CA

CA constraint certificate:

• Intermediate CA

Then CertPathUtil.validateCertificatePath() would have TLS Server Certificate and Intermediate CA in the chain, it would drop intermediate CA from the list and send it to JRE's PKIX validation and that would pass.

In order to match Intermediate CA we would need to search for intermediate CA from trust store / CA constraint certificate. So this becomes more or less the path expansion case.

That being said --

In verifyCertificate we can always return DTLS certificate chain (messageChain) in LWM2M Client case. In LWM2M Bootstrap server case we need to store whole chain for later.

We can also drop CertPath generation from expandCertPath() and just return list of certificates if that makes life easier.

[boaks]

At least in my experience, a server may only remove the "root CA" (self signed). Removing more requires Information in a hello_extension from a client (AFAIK, that's not implemented in Californium).
With that scenario A) fails, because the server seems to me to be not compliant.

B) to E) don't require to expand the path, or?

In LWM2M Bootstrap server case we need to store whole chain for later.

If the path completion is for the bootstrapping/provisioning, sure, there it makes sense.
I was focused on the use in the handshake.

[dachaac]

At least in my experience, a server may only remove the "root CA" (self signed). Removing more requires Information in a hello_extension from a client (AFAIK, that's not implemented in Californium).
With that scenario A) fails, because the server seems to me to be not compliant.

I don't see that this is about TLS handshake phase thing. It is side operation.

What would render it not compliant? Client anyway needs to have at least Root CA there so why it couldn't also have Intermediate CA in its trust store? Not the wisest thing to do but...

Only systems so far that I have seen that enforces full chain for server seems to be Java or then just some libraries on top of that.

As an example with any openssl built software you just specify certificate or certificates and they appear there as 1:1 what is in file. I have seen this problem many times as people don't always recognize how it should be configured properly on server and as it doesn't work then they just import also the intermediate CA in clients trust store and be happy after that as everything seems to work. (and then they share the tale what needs to be done.)

For this reason there has been "incorrectly" configured servers at least in company networks.

Why company network is important in here is that it is somewhat reasonably not to utilize public CA with LWM2M meaning that private CA would be utilized thus not everything is perfect in those setups always.

B) to E) don't require to expand the path, or?

How would you then compare the certificate match?

Also assume that system trust store would be there,

[boaks]

I don't see that this is about TLS handshake phase thing. It is side operation.

certificate_list

This is a sequence (chain) of certificates. The sender's
certificate MUST come first in the list. Each following
certificate MUST directly certify the one preceding it. Because
certificate validation requires that root keys be distributed
independently, the self-signed certificate that specifies the root
certificate authority MAY be omitted from the chain, under the
assumption that the remote end must already possess it in order to
validate it in any case.

FMPOV, it is specified for TLS 1.2, and so for DTLS 1.2. Sure, if other local setups are common, the local operators may do what they want. And use then a "custom verifier".

How would you then compare the certificate match?

Isn't hat just the "signing thing"? Either one of the trusted-certificates is in the chain, or the issuer of that last certificate in the chain, that signed it, is a trusted-certificate. If there are more than one trusted certificate with that subject, then it may be required to select the right one.

[dachaac]

I don't see that this is about TLS handshake phase thing. It is side operation.

certificate_list

This is a sequence (chain) of certificates. The sender's
certificate MUST come first in the list. Each following
certificate MUST directly certify the one preceding it. Because
certificate validation requires that root keys be distributed
independently, the self-signed certificate that specifies the root
certificate authority MAY be omitted from the chain, under the
assumption that the remote end must already possess it in order to
validate it in any case.

FMPOV, it is specified for TLS 1.2, and so for DTLS 1.2. Sure, if other local setups are common, the local operators may do what they want. And use then a "custom verifier".

From TLS connectivity verification point of view I do not see a problem.

As noted above PKIX validation would pass.

But we do have Certificate Usage rules that need to be applied and for this there needs to be compliant certificate verifier. And this is where this PR comes in with custom certificate verifiers. Whether the logic is implemented in Leshan or in Californium for those operations you can agree with @sbernard31 😁.

What should be fixed in Californium is the certificate search logic so that it does not accidently construct failing path for PKIX validation.

How would you then compare the certificate match?

Isn't hat just the "signing thing"? Either one of the trusted-certificates is in the chain, or the issuer of that last certificate in the chain, that signed it, is a trusted-certificate. If there are more than one trusted certificate with that subject, then it may be required to select the right one.

Yes, and this is what expandCertPath() is doing.

Selecting the certificates that should be there.

Once the list is complete. Then do the comparison for the list.

[boaks]

Seems you have a different understanding (or experience).

In my opinion, it's not required to complete the path, at least in general. If there are some LwM2M specific ideas, then I think, leshan is the right place to implement it. If there are ideas about relaxing the rules in order to help out "local operating", that's also ok, but I would not assume that to be the basic implementation.

Completing the path doesn't "harm" (beside of the CPU usage). But for now, I don't see, that this is a common requirement.

I will try to adapt Californium to remove the check for DN duplicates this week.

[boaks]

Let me add:

At least for midterm, I think, addressing the concerns about the requirements for server's certificate path and the idea to therefore "complete the path based on the system's trust store" to the OMA/LwM2M will payoff. If that "path completion" is considered to be valid, then that gets clear. If sending "complete server certificate paths" is required, that may get clear as well.

[boaks]

The support for "multiple trusted certificates sharing same DN" is in Californium - PR1442.

About the idea of completing the server certificate chain, I'm still skeptical. That maybe "the realistic way", but without clarification from the OMA, I would prefer to leave this to custom implementations.

[sbernard31]

It seems to me expendCertPath tries to address several issue and I'm not smart enough to solve all of this at the same time 😅.

A list (Maybe not exhaustive) about what expandCertPath address :

1. Extend certpath with trusted certificate
2. Avoid loop in truststore
3. support multiple trusted certificates sharing same DN
4. support "incomplete chain" (more than root CA is missing)

For now, I propose to let aside the use case where certchain is incomplete (4.), because this sounds not the specified way. (If this is a real use case, we can add it later in specific PR with dedicated tests and dedicated discussion)

With this restriction this means that truststore only contains root anchor and so there is no more need to check loop in truststore (2.), correct ?

(3.) will be supported by californium 2.5.0.

For (1.) this is needed for CA Constraint usage from rfc6698

 * 0 -- Certificate usage 0 is used to specify a CA certificate, or
 *       the public key of such a certificate, that MUST be found in any of
 *       the PKIX certification paths for the end entity certificate given
 *       by the server in TLS.  This certificate usage is sometimes
 *       referred to as "CA constraint" because it limits which CA can be
 *       used to issue certificates for a given service on a host.  The
 *       presented certificate MUST pass PKIX certification path
 *       validation, and a CA certificate that matches the TLSA record MUST
 *       be included as part of a valid certification path.  Because this
 *       certificate usage allows both trust anchors and CA certificates,
 *       the certificate might or might not have the basicConstraints
 *       extension present.

My guess is the method which is responsible to do PKIX validation SHOULD return the TrustAnchor selected.
Java API seems to provides this information via PKIXCertPathValidatorResult.getTrustAnchor() maybe CertPathUtil.validateCertificatePath() should be adapted, but let discuss about that in a dedicated issue. (eclipse-californium/californium#1443)

So for this PR, I will rewrite the code to address the (1.) issue only.

[boaks]

@dachaac

[ERROR]   SecurityTest.registered_device_with_x509cert_to_server_with_x509cert_rootca_certificate_usage_taa:1176 No registration expected
[ERROR]   SecurityTest.registered_device_with_x509cert_to_server_with_x509cert_server_certificate_usage_ca_server_cert_wo_chain_given:935 No registration expected
[ERROR]   SecurityTest.registered_device_with_x509cert_to_server_with_x509cert_server_certificate_usage_taa_server_cert_wo_chain_given:1380 No registration expected

Beside of a bug in Californium 2.4.1, is there any reason, why

Expected outcome:

• Client denied the connection (missing intermediate CA aka. "server chain configuration problem")

Using Californium 2.5.0 these unit tests are failing, because the validation now succeeds with the fix in Californium. And with that the handshakes are also succeeding and not longer failing.

The current PR in Californium offers now also the possibility to truncate/amend the received certpath with the trusted CA certificate (before this Californium PR that CA trusted certificate was cut off :-) ). But only the one certificate, not more.
I'm not sure, maybe that makes the one or other function easier.

[dachaac]

@boaks

[ERROR]   SecurityTest.registered_device_with_x509cert_to_server_with_x509cert_rootca_certificate_usage_taa:1176 No registration expected

   /**
     * Test scenario:
     * - Certificate Usage = trust anchor assertion
     * - Server Certificate = server certificate
     * - Server's TLS Server Certificate = intermediate signed certificate (with SAN DNS entry)
     * - Server accepts client
     * - Client Trust Store = root CA
     *
     * Expected outcome:
     * - Client denied the connection (not pkix chainable, client trust store not in use, server not sending root CA and TAA is not root CA)
     */
    @Test
    public void registered_device_with_x509cert_to_server_with_x509cert_rootca_certificate_usage_taa()

With:

        helper.createServerWithX509Cert(helper.serverIntX509CertChain, helper.serverIntPrivateKeyFromCert,
                helper.trustedCertificates, true);

Server is sending [Server] + [Intermediate CA].

And with:

        helper.createX509CertClient(new X509Certificate[] { helper.clientX509Cert }, helper.clientPrivateKeyFromCert,
                helper.clientTrustStore, helper.serverIntX509CertChain[0], CertificateUsage.TRUST_ANCHOR_ASSERTION);

Clients trust store is [Server].

In LWM2M transport specification the wording is:

2: Certificate usage 2 ("trust anchor assertion")
This mode is used to specify a certificate, or the public key of such a certificate, that MUST be used as the trust
anchor when validating the end entity certificate given by the server in TLS. This certificate usage is sometimes
referred to as "trust anchor assertion" and allows a domain name administrator to specify a new trust anchor --
for example, if the domain issues its own certificates under its own CA that is not expected to be in the end users'
collection of trust anchors. The target certificate MUST pass PKIX certification path validation, with any certificate
matching the Server Public Key Resource content considered to be a trust anchor for this certification path
validation.

The test case here is actually a bit stupid in a way as it could match right away in server certificate. What I was thinking in here with PKIX validation was that there must be at least two certificates before it can match and it must end in Root CA.

As client's trust store is not used in this mode at all there are no CA checks at all. I believe Java's PKIX validation would fail in here. Or you would end up with empty chain for PKIX validation or no call at all. What needs to happen is that certificate is checked for validity and other requirements present in the certificate. This is best left as a job of PKIX validation code as the code might be a bit challenging to write.

I am wondering if it is OK to leave chain open in order for PKIX validation to pass.

---

[ERROR]   SecurityTest.registered_device_with_x509cert_to_server_with_x509cert_server_certificate_usage_ca_server_cert_wo_chain_given:935 No registration expected

   /**
     * Test scenario:
     * - Certificate Usage = CA constraint
     * - Server Certificate = intermediate signed certificate/wo chain
     * - Server's TLS Server Certificate = intermediate signed certificate/wo chain (with SAN DNS entry)
     * - Server accepts client
     * - Client Trust Store = root CA
     *
     * Expected outcome:
     * - Client denied the connection (missing intermediate CA aka. "server chain configuration problem")
     */
    @Test
    public void registered_device_with_x509cert_to_server_with_x509cert_server_certificate_usage_ca_server_cert_wo_chain_given()

In this test we have:

        helper.createServerWithX509Cert(new X509Certificate[] { helper.serverIntX509CertChain[0] },
                helper.serverIntPrivateKeyFromCert, helper.trustedCertificates, true);

Which causes server to only send server certificate and not intermediate nor root CA.

Now in:

        helper.createX509CertClient(new X509Certificate[] { helper.clientX509Cert }, helper.clientPrivateKeyFromCert,
                helper.clientTrustStore, helper.serverIntX509CertChain[0], CertificateUsage.CA_CONSTRAINT);

We do have helper.clientTrustStore which is:

clientTrustStore = Arrays.asList(new Certificate[] {rootCAX509Cert});

Which is only root CA.

Then as a CA constraint rule to check for helper.serverIntX509CertChain[0].

Now for Certificate Constraint we need to do PKIX chain verification.

In clientTrustStore we have Root CA and as server is not sending intermediate CA then we have:

[Server] [missing: intermediate CA] [Root CA]

Aka. we have chain that is not terminated to known Root CA. -> not PKIX chain validatable.

Now I do not know if it gets confused or not but:

We have internal trust store for CA constraint configured as:

[trust store] + [server cert]

So we have:
{ [Root CA], [Server] }

In LWM2M transport specification the wording is:

0: Certificate usage 0 ("CA constraint")
This mode is used to specify a CA certificate, or the public key of such a certificate, that MUST be found in any of
the PKIX certification paths for the end entity certificate given by the server in TLS or DTLS. This certificate usage
is sometimes referred to as "CA constraint" because it limits which CA can be used to issue certificates for a given
service on a host. The presented certificate MUST pass PKIX certification path validation, and a CA certificate that
matches the Server Public Key Resource content MUST be included as part of a valid certification path. Because this
certificate usage allows both trust anchors and CA certificates, the certificate might or might not have the
basicConstraints extension present.

Depending a bit CA constraint can be understood in two ways:

• PKIX validation with [trust store] & then in PKIX chain the Public Key Resource (certificate) must be present in chain.
  OR:
• PKIX validation with [trust store+certificate] & then in PKIX chain the Public Key Resource (certificate) must be present in chain.

I took the more liberal path in here but the exact path might be better.

So if you remove in CaliforniumEndpointsManager.createEndpoint() the adding of server certificate to trust store:

if (certificateUsage == CertificateUsage.CA_CONSTRAINT) {
...
                    if (serverInfo.serverCertificate instanceof X509Certificate) {
                        newTrustedCertificatesList.add((X509Certificate) serverInfo.serverCertificate);
                    }

Does it now fail -> eg test pass?

I am OK to take more restrictive interpretation in here.

---

[ERROR]   SecurityTest.registered_device_with_x509cert_to_server_with_x509cert_server_certificate_usage_taa_server_cert_wo_chain_given:1380 No registration expected

    /**
     * Test scenario:
     * - Certificate Usage = trust anchor assertion
     * - Server Certificate = intermediate signed certificate/wo chain
     * - Server's TLS Server Certificate = intermediate signed certificate/wo chain (with SAN DNS entry)
     * - Server accepts client
     * - Client Trust Store = root CA
     *
     * Expected outcome:
     * - Client denied the connection (not pkix chainable, missing root CA and intermediate CA)
     */
    @Test
    public void registered_device_with_x509cert_to_server_with_x509cert_server_certificate_usage_taa_server_cert_wo_chain_given()

With:

        helper.createServerWithX509Cert(new X509Certificate[] { helper.serverIntX509CertChain[0] },
                helper.serverIntPrivateKeyFromCert, helper.trustedCertificates, true);

Server is sending [Server].

And with:

        helper.createX509CertClient(new X509Certificate[] { helper.clientX509Cert }, helper.clientPrivateKeyFromCert,
                helper.clientTrustStore, helper.serverIntX509CertChain[0], CertificateUsage.TRUST_ANCHOR_ASSERTION);

Clients trust store is [Server].

Even thou what the server is sending is different in here this is more or less same as SecurityTest.registered_device_with_x509cert_to_server_with_x509cert_rootca_certificate_usage_taa().

[boaks]

@dachaac
@sbernard31

I'm still not sure, what exactly for LwM2M should be used to trust or not.
So, FMPOV, I guess LwM2M will be better off using a own custom verifier. The idea, implemented in Californium, may be too simple (pass, if one certificate of the presented path is in the trusts or signed by one certificate of the trusts), but therefore it's customizable.

[sbernard31]

@boaks, just to be clear what we try to implement here is not specific to LWM2M, it's about what is specified in rfc6698. So our specification is just the 2.1.1. The Certificate Usage Field paragraph.

[boaks]

@sbernard31

My impression is, that this requires discussion and clarifications.
And my experience of the past years is, that such discussion may take longer until there is a common understanding, what is to be implemented. That discussion may even span the IETF and/or OMA.

So my preference is still
Release Californium 2.5.0 today.

If afterwards it gets clear, how that should be done, then adapt californium again, either 2.5.1 or 2.6.0.

[sbernard31]

So my preference is still
Release Californium 2.5.0 today.
f afterwards it gets clear, how that should be done, then adapt californium again, either 2.5.1 or 2.6.0.

I have no problem with this 👍 (There is no urgency to me to bring this in 2.5.0)

I just said this just to let you know that :

• you don't need to read the LWM2M specification but just this paragraph to give us your understanding.
• as this specification is not specific to LWM2M and this is maybe more a TLS/DTLS stuff and so probably more a Scandium discussion than a Leshan one. (But I suppose this point is debatable)

[boaks]

Reading RFC6698, 2.1.1 leaves many questions to me.

What I understand:

It differentiate, where the provided "trust certificate" is located in the path and if a PKIX validation must be passed.

• 0 middle or root, not end node, PKIX validation
• 1 end node, PKIX validation
• 2 root, PKIX validation
• 3 end node, no validation

What is unclear:

Is there a difference between mentioned "target" and "presented" certificate?

Where does the PKIX validation start?

• With the issuer of the last in the path? (That's java's PKIX validator).
• With the first on of the trusted certificates in the path? (That's Californiums implementation)

I don't know, why in RFC6698 the position is that important. I personally consider more to know precise the certificate before I use this as trust. With keyusage/extkeyusage I really wondering about that.

[dachaac]

Reading RFC6698, 2.1.1 leaves many questions to me.

What I understand:

It differentiate, where the provided "trust certificate" is located in the path and if a PKIX validation must be passed.

• 0 middle or root, not end node, PKIX validation
• 1 end node, PKIX validation
• 2 root, PKIX validation
• 3 end node, no validation

I assume no validation here means "no PKIX validation".

Trust anchor can also be intermediate CA. I believe this is used in cases where you want to make sure that trust is from one specific intermediate CA.

How I see the difference between '0' and '2' is that '0' would have system/app trust store available where as '2' would not.

In here there is some discussion:
https://tools.ietf.org/html/rfc6698#section-8

   Certificates that are delivered in TLSA with certificate usage 2
   fundamentally change the way the TLS server's end entity certificate
   is evaluated.  For example, the server's certificate might chain to
   an existing CA through an intermediate CA that has certain policy
   restrictions, and the certificate would not pass those restrictions
   and thus normally be rejected.  That intermediate CA could issue
   itself a new certificate without the policy restrictions and tell its
   customers to use that certificate with certificate usage 2.  This in
   essence allows an intermediate CA to become a trust anchor for
   certificates that the end user might have expected to chain to an
   existing trust anchor.

With this text I would translate your list:

• 0 middle or root, not end node, PKIX validation (where as trust store is system/app trust store)
• 1 end node, PKIX validation (where as trust store is system/app trust store)
• 2 middle or root, not end node, PKIX validation (where as trust store is only the provided expected certificate)
• 3 end node, no PKIX validation

If we forget DNSSEC for a second in this pseudo code:
https://tools.ietf.org/html/rfc6698#appendix-B.2

There is actually logic presented for implementation. A bit challenging to interpret -- I would say there is something wrong in the example there. But the idea is hopefully is correct.

In LWM2M case I believe parameters would be:

• R.matchingType = 0 (Exact match on selected content)
• R.selectorType = 0 (X.509 as DER) (or 1 for SubjectPublicKeyInfo as DER)

Differences I believe are:

• In the code there is no support for R.selectorType=1 only 0 is supported.
  • Not sure if this even needs to be implemented. Don't understand why someone would use SubjectPublicKeyInfo in the X.509 world.
• In cert usage=0 path we do not make sure that it is CA certificate.
  • This would be basic constraints CA:TRUE check
  • Wondering shouldn't this be PKIX validation job -- but can be added for completeness.

                // Make sure that certificate is a CA certificate
                if (x509Certificate.getBasicConstraints() != -1) {
                    found = true;
                    break;
                }

• in cert usage=0 -- expected certificate was also included in trust store -> we could drop this addition and make it more strict.
  • use case would be mainly to limit which CA is trusted from public/system/app CA's.

I have updated original PR #912 with the changes above:

• CA type check
• Not to include expected server certificate into the CA constraint trust store
• Updated test cases to match above

What is unclear:

Is there a difference between mentioned "target" and "presented" certificate?

I believe the they are the "same".

Target seems to be describing server certificate:

The target certificate MUST pass PKIX certification path validation and MUST match the TLSA record.

The target certificate MUST pass PKIX certification path validation, with any certificate matching the TLSA record considered to be a trust anchor for this certification path validation.

The target certificate MUST match the TLSA record.

In here I would interpret this that target certificate is coming from server and TLSA record in our case is LWM2M Server Public Key identity.

And the presented part:

The presented certificate MUST pass PKIX certification path validation, and a CA certificate that matches the TLSA record MUST be included as part of a valid certification path.

Why different wording is then used here -- perhaps it wants to illustrate that TLSA record processing is not involved in the PKIX validation process at all in here?

But I would interpret it as presented certificate is coming from server. Eg. the same as 'target'.

Where does the PKIX validation start?

• With the issuer of the last in the path? (That's java's PKIX validator).
• With the first on of the trusted certificates in the path? (That's Californiums implementation)

My understanding is that PKIX validation starts from server certificate and ends in trusted CA certificate from trust store. In last ''era' in browsers you could have added implicit trust for server certificate but this seems to be gone and only allowed for self signed server certificates. Otherwise issuer (or Root CA) needs to be in trust store.

Where this is in RFCs is then a good question.

I don't know, why in RFC6698 the position is that important. I personally consider more to know precise the certificate before I use this as trust. With keyusage/extkeyusage I really wondering about that.

I agree that in installation that I would do the configuration would be very precise -- but that is kinda different what kind of possibilities they want to allow for configuration. Different companies are in different situations and with different policies.

That quote above kinda illustrates one special case that they have been thinking when crafting the RFC.

LWM2M implementors probably have been thinking that this is good flexibility to allow.

Your comment about key usage/extended key usage was a bit short -- care to elaborate?

[sbernard31]

My understanding is that PKIX validation starts from server certificate and ends in trusted CA certificate from trust store.

RFC3280 seems to confirm that. (see eclipse-californium/californium#1446)

[sbernard31]

I think this PR is now in a mergeable state.
@dachaac, I would appreciate that you have a look at the 2 last commit.

@boaks, I finally don't reuse CertPathUtil from scandium directly. 😬
You could have a look to CertPathUtilExt in this PR, if you are curious.
We will polish all this x509 stuff and once all of this will be clearer for us, maybe we will come back to you with some proposal for scandium 3.0.0.

@dachaac, I didn't integrate the last change you did in your #912 PR. (or least not intentionally)
We can discuss about this in smaller dedicated PR.

@boaks, @dachaac about :

[ERROR]   SecurityTest.registered_device_with_x509cert_to_server_with_x509cert_rootca_certificate_usage_taa:1176 No registration expected
[ERROR]   SecurityTest.registered_device_with_x509cert_to_server_with_x509cert_server_certificate_usage_ca_server_cert_wo_chain_given:935 No registration expected
[ERROR]   SecurityTest.registered_device_with_x509cert_to_server_with_x509cert_server_certificate_usage_taa_server_cert_wo_chain_given:1380 No registration expected

I think failure is expected mainly because direct trust is not OK for "trust anchor" and "CA constraint" usage. (At least this is my current understanding)

Working on this make me ask more questions about some corner case I prefer to not discuss in this PR.
I will create dedicate topic for those. @boaks would you like to participate or at least follow this discussion ? should I create issue in Californium or is it more appropriate to do this in Leshan repository ?

[boaks]

You could have a look to CertPathUtilExt in this PR, if you are curious.
We will polish all this x509 stuff and once all of this will be clearer for us.

Thanks a lot! I think, if this is stable for some time in leshan, then it makes sense to adapt californium accordingly.

would you like to participate or at least follow this discussion ? should I create issue in Californium or is it more appropriate to do this in Leshan repository ?

Seems that mainly @dachaac and you are interested. So I prefer an issue in leshan.
I will follow it, but mainly passive and with some delay.

[sbernard31]

Before to merge, I need to change Keystore format to be java7 compliant : see #855 (comment)

(I will add a TODO to change it again if we decide that finally Leshan 2.0.0 target java8 : see #924)

[boaks]

@dachaac

Your comment about key usage/extended key usage was a bit short -- care to elaborate?

These options also define the position of the trust in the chain:

• CA, not end node, keyusage = KEY_USAGE_CERTIFICATE_SIGNING
• end node, keyusage = SERVER_AUTHENTICATION

It's not equivalent nor is it required at all to provide them. But if, then they describe the position as well.

[dachaac]

@dachaac

Your comment about key usage/extended key usage was a bit short -- care to elaborate?

These options also define the position of the trust in the chain:

• CA, not end node, keyusage = KEY_USAGE_CERTIFICATE_SIGNING
• end node, keyusage = SERVER_AUTHENTICATION

It's not equivalent nor is it required at all to provide them. But if, then they describe the position as well.

I have always thought that KEY_USAGE_CERTIFICATE_SIGNING is job of PKIX validation. Eg. all things that are part of integrity and validity of the chain is part of PKIX validation.

And as PKIX validation cannot know purpose what is being done then either TLS library or application code needs to check for SERVER_AUTHENTICATION.

If I am not mistaken latter was already happening.

[dachaac]

@sbernard31 about test change logic in registered_device_with_x509cert_to_server_with_x509cert_selfsigned_certificate_usage_taa_selfsigned_server_cert_given() - I am OK.

[dachaac]

@sbernard31 splitting functionality to BaseCertificateVerifier looks OK.

applyPKIXValidation() looks OKish.

If the tests pass then I suppose this is good to go. At least there is base to continue if needed.

Note: one could consider in future using Java's trust store as a memory object instead of lists.

[sbernard31]

@dachaac I just discover "OKish". 😁

It seems that means something like : "This word is a slang term merging OK with the suffix -ish, thus downgradeing the defined subject to less than OK."

So I suppose there are some points which disturb you, please do not hesitate to share. 🙏

[dachaac]

@dachaac I just discover "OKish".

It seems that means something like : "This word is a slang term merging OK with the suffix -ish, thus downgradeing the defined subject to less than OK."

So I suppose there are some points which disturb you, please do not hesitate to share.

Never actually looked for the definition ;).

I suppose the only problem there is in CaConstraintCertificateVerifier and if the trust store has the Root CA and Intermediate CA and then caCertificate is configured to be Root CA then the match would fail as there would be more than one certificate that needs to be added.

How one could get to that situation:

• Using EST and cacerts requests get imported to trust store.
• Two corporate CA's imported in trust store

Then the question is why cacerts would cause that to happen is:

• EST's cacerts contains root ca and intermediate CA's and simple enroll request then provides device certificate without chain.

It is then job of the EST client to reconstruct the device certificate chain with this information.

At least we were planning to do so that cacerts also contains some other CA's for the system so that they get delivered during bootstrapping and gets imported to different trust stores for other communication protocols.

[sbernard31]

I suppose the only problem there is in CaConstraintCertificateVerifier and if the trust store has the Root CA and Intermediate CA and then caCertificate is configured to be Root CA then the match would fail as there would be more than one certificate that needs to be added.

I agree there is something not clear here.
This is one of the topic I have in mind when I said :

Working on this make me ask more questions about some corner case I prefer to not discuss in this PR.
I will create dedicate topic for those

I will create issues for those topics where we can talk about this.

[sbernard31]

#935 and #936 are some of those topics. (Maybe more is coming)

[sbernard31]

The last one for now #938.

(@boaks, note that #935 and #938 concern a bit more californium, #936 is currently more a leshan one as certificate usage is currently implemented in Leshan)