Add support for RSA-PSS Keys

[Jakuje]

Description

This is not as straightforward as it looked like initially and consists of several changes that surface on different occasions:

• The private RSA keys can have CKA_ALLOWED_MECHANISMS which can be used to determine if a key is generic key (can be used for any operation) or RSA-PSS key, so that it could be used only for any or specific RSA-PSS operations.

• These keys get different identifier on OpenSSL level. They also get different OIDs on the ASN.1 in various places of the X.509 certificates:

  • The signatures in X.509 has a OID + parameters describing the hashes, mgf and salt length used. This is mandatory.

  • The public key encoding can contain the RSA-PSS restrictions. These are not mandatory so it allows us to indicate the key is restricted to PSS operations without the need to stick to specific combination of parameters.

• When we force all operation in pkcs11 provider, the rsapss table was missing the match() callback, making key comparison fail when using RSA-PSS keys.

Given that the certtool we use for signing certificates during setup can not distinguish RSA-PSS restricted keys and therefore generates unrestricted certificates, we need to generate the certificate later using openssl, making the tests a bit more ugly.

Fixes: #489

Checklist

• Code modified for feature
• Test suite updated with functionality tests
• Test suite updated with negative tests
• Documentation updated

Reviewer's checklist:

• Any issues marked for closing are addressed
• There is a test suite reasonably covering new functionality or modifications
• This feature/change has adequate documentation added
• Code conform to coding style that today cannot yet be enforced via the check style test
• Commits have short titles and sensible commit messages
• Coverity Scan has run if needed (code PR) and no new defects were found

[Jakuje]

(sorry for huge one-commit, but it was pile of different interleaving changes I did not manage to separate to sensible smaller chunks yet)

The failed test on macos times out in uri test as it is likely much slower with the new keys -- I can probably increase the timeout if we will agree that this is the right direction.

[simo5]

I wish the ASN.1 code was more straightforward and weren't obfuscated by all these OpenSSL Opaque structures.
For example all of the MD and MFG1 EVP_md stuff is ultimately only need to translate from the Hash name to the corresponding OID used in X509 ... I wonder if we shouldn't just store ASN1_OBJECT * pointers instead of the EVP_md ones in our structures to avoid all the churn. Even better if there was a way to just store the serialized ASN.1 once instead of reconstructing it from scratch at every invocation ...
After all there is nothing really dynamic in this information from our POV, the same fixed algorithm will always generate a very defined ASN.1 output.
Maybe we should just generate it once via some ASN.1 utility and store the pre-generated ASN.1 in the rsa-pss_map ?
Is there any need for runtime generation ?

[Jakuje]

I have the signature parameters rewritten to use static structures, but X509_PUBKEY_set0_param() requires again the ASN1_STRING structure as an input so we will have to convert it from the static structures again (with d2i_ASN1_STRING function likely), which makes it a bit defeating the point of having static structures. But I think we can live with it.

Edit: The ASN1_STRING is just a wrapper for any string so I can just feed it with the static buffer using ASN1_STRING_set.

[simo5]

You already amended your comment, yeah an ASN1_STRING is a lightweight wrapper for a "bunch of bytes", so is not a big deal.

[simo5]

LGTM

[simo5]

URI test still very slow on Mac + softhsm, but URI+softhsm is v. slow on other platforms too, pushing for now, and we'll investigate later