Enable RSA-PSS with ALLOWED_MECHANISMS tests for kryoptic

[Jakuje]

Description

For some reason, with the kryoptic backend we are not getting all the same
errors, but this one is common.

There is also one warning that I missed in the previous PR

Checklist

• Test suite updated with functionality tests
• Test suite updated with negative tests

Reviewer's checklist:

• Any issues marked for closing are addressed
• There is a test suite reasonably covering new functionality or modifications
• This feature/change has adequate documentation added
• Code conform to coding style that today cannot yet be enforced via the check style test
• Commits have short titles and sensible commit messages
• Coverity Scan has run if needed (code PR) and no new defects were found

[simo5]

Do you know why we get a different message? It could indicate a bug in kryoptic I assume.

[Jakuje]

Do you know why we get a different message? It could indicate a bug in kryoptic I assume.

I stepped through the code in pkcs11-provider to make sure it really goes through the same code and gets through the message mechanism not allowed with this key, but for some reason it was not propagated to the output. I assume the error stack is reset somewhere differently than in softhsm.

Not sure why. I will try to investigate it further tomorrow.

[Jakuje]

Tracing down from the error with kryoptic and softhsm leads to the same steps, but ERR_print_errors() in apps/pkeyutl.c:506 prints the pkcs11 provider errors in softhsm, but does not with kryoptic:

476	        P11PROV_raise(sigctx->provctx, CKR_ACTION_PROHIBITED,
477	                      "mechanism not allowed with this key");
478	        return CKR_ACTION_PROHIBITED;
479	    }
480	
(gdb) n
478	        return CKR_ACTION_PROHIBITED;
(gdb) 
483	}
(gdb) 
p11prov_sig_set_mechanism (sigctx=0x5555556ca220, digest_sign=true, mechanism=0x7fffffffacc0)
    at ../src/signature.c:548
548	        break;
(gdb) 
568	    if (result == CKR_OK) {
(gdb) 
573	    return result;
(gdb) 
574	}
(gdb) 
p11prov_sig_operate_init (sigctx=0x5555556ca220, digest_op=true, _session=0x5555556ca250) at ../src/signature.c:822
822	    if (ret != CKR_OK) {
(gdb) 
823	        return ret;
(gdb) 
879	}
(gdb) 
p11prov_sig_digest_update (sigctx=0x5555556ca220, 
    data=0x7fffffffadb0 "\306ۓɢH\251\301\324i۷\350M>\320*\030\367\t\376 \037C\256x\221\264\b\bx\025\022\3146\334\026o\002\347'\304\366\212\246FB\331%\246D\272\364^//2\003A\267{̗\203X\303C\227\242R\bs\037*\306]\254\265\322y\202\377!\372+\364]H6\271\221\320\030;\322\026\266l\225<;\345\314\375 \246\257U\\M\341h\275\231E\242\373\302b\216;\b\361\301^o\367\245L\r\350\025\301\206\261\f\263f\204\033.\t\364O\262\003:{8\361\223\f\235u\027]\200/ai\340\300\234\253K`q\021\v\024W\254\243\347\316\362\324N\313\r;\370-\312@m\266\3565e2\222\016\026w`̬\207\202"..., datalen=2048)
    at ../src/signature.c:970
970	        if (ret != CKR_OK) {
(gdb) 
971	            return RET_OSSL_ERR;
(gdb) 
997	}
(gdb) 
p11prov_rsasig_digest_sign_update (ctx=0x5555556ca220, 
    data=0x7fffffffadb0 "\306ۓɢH\251\301\324i۷\350M>\320*\030\367\t\376 \037C\256x\221\264\b\bx\025\022\3146\334\026o\002\347'\304\366\212\246FB\331%\246D\272\364^//2\003A\267{̗\203X\303C\227\242R\bs\037*\306]\254\265\322y\202\377!\372+\364]H6\271\221\320\030;\322\026\266l\225<;\345\314\375 \246\257U\\M\341h\275\231E\242\373\302b\216;\b\361\301^o\367\245L\r\350\025\301\206\261\f\263f\204\033.\t\364O\262\003:{8\361\223\f\235u\027]\200/ai\340\300\234\253K`q\021\v\024W\254\243\347\316\362\324N\313\r;\370-\312@m\266\3565e2\222\016\026w`̬\207\202"..., datalen=2048)
    at ../src/signature.c:1245
1245	}
(gdb) 
do_raw_keyop (pkey_op=pkey_op@entry=16, mctx=mctx@entry=0x555555652330, pkey=pkey@entry=0x555555664870, 
    in=in@entry=0x555555716f80, filesize=filesize@entry=65536, sig=sig@entry=0x0, siglen=0, out=0x7fffffffb6d0, 
    poutlen=0x7fffffffb6c0) at apps/pkeyutl.c:784
784	            if (rv != 1) {
(gdb) 
785	                BIO_printf(bio_err, "Error signing raw input data\n");
(gdb) 
Error signing raw input data
786	                goto end;
(gdb) 
798	    OPENSSL_free(mbuf);
(gdb) 
799	    return rv;
(gdb) 
pkeyutl_main (argc=<optimized out>, argv=<optimized out>) at apps/pkeyutl.c:485
485	    if (rv <= 0) {
(gdb) 
486	        if (pkey_op != EVP_PKEY_OP_DERIVE) {
(gdb) 
487	            BIO_puts(bio_err, "Public Key operation error\n");
(gdb) 
Public Key operation error
506	        ERR_print_errors(bio_err);

while this line in softhsm prints:

8012EDF7FF7F0000:error:4080001B:pkcs11:p11prov_sig_pss_restrictions:reason(27):../src/signature.c:476:mechanism not allowed with this key

I am really not sure how the error stack works here or if some call to the softhsm messes it up, if it is realated to the no-deinit quirk or blocking of the operation state.

The error is visilble in both cases in the p11prov-debug.log so alternative might be to log this operation to separate file and grep the log file. Would this work for you?

[simo5]

I just wanted to know why there is a difference.
It is very likely that softhsm2 mangles the error stack given they do not use a separate libctx or anything.
I think I am ok with the change you made, sounds like this is not a problem in kryoptic and divert the debug file would make it hard to debug other issues.

[simo5]

LGTM

[Jakuje]

I just wanted to know why there is a difference. It is very likely that softhsm2 mangles the error stack given they do not use a separate libctx or anything. I think I am ok with the change you made, sounds like this is not a problem in kryoptic and divert the debug file would make it hard to debug other issues.

Right. From my observation, the kryoptic worked well. I did even extend tests for that in latchset/kryoptic#142 as I saw this was not well covered.