intermittent invalid signature with tz2 in aws kms

[stephengaudet]

tz3 seems ok, but tz2 produces signature that octez-client reports as invalid about 50% of the time.

steps to reproduce:

1. configure the Signatory aws-kms vault
2. create a tz2 (Secp256k1) and a tz3 (P-256) keys for signing in aws kms
3. make active these 2 keys in signatory.yaml
4. import the Signatory URI for both keys into octez-client use aliases awstz2 and awstz3
5. fund both the tz2 and tz3 with some tez. 100 for example
6. make repeated calls to: octez-client transfer 1 from awstz2 to alice --burn-cap 0.06425

expected:
each transfer is successful. (it is with tz3 but not tz2)

actual:
Signatory logs each one as successful:
time="2023-05-23T20:58:43Z" level=info msg="Requesting signing operation" ops="map[transaction:1]" ops_total=1 pkh=tz2SctcTKRRyogSTC8BZcAyxjw9XLUuyjvgM request=generic vault=AWSKMS vault_name=aws
time="2023-05-23T20:58:43Z" level=info msg="About to sign raw bytes" ops="map[transaction:1]" ops_total=1 pkh=tz2SctcTKRRyogSTC8BZcAyxjw9XLUuyjvgM raw=0349a3452d2701444118c24aa6dc0ee16f234797af321c95693be265f135cc0a3b6c01c8c903c8f37924fb8c158b58b78f5cebdf54d4028a02089b0100c0843d00006b82198cb179e8306c1bedd08f12dc863f32888600 request=generic vault=AWSKMS vault_name=aws
time="2023-05-23T20:58:43Z" level=info msg="Signed generic successfully" ops="map[transaction:1]" ops_total=1 pkh=tz2SctcTKRRyogSTC8BZcAyxjw9XLUuyjvgM request=generic vault=AWSKMS vault_name=aws
time="2023-05-23T20:58:43Z" level=info msg="POST /keys/tz2SctcTKRRyogSTC8BZcAyxjw9XLUuyjvgM" duration=183.171042ms hostname="signatory:6732" method=POST path=/keys/tz2SctcTKRRyogSTC8BZcAyxjw9XLUuyjvgM start_time="2023-05-23T20:58:43Z" status=200

but half of them are invalid from octez-client perspective:
The signer for http://signatory:6732/tz2SctcTKRRyogSTC8BZcAyxjw9XLUuyjvgM produced an invalid signature
Fatal error:
transfer simulation failed

[stephengaudet]

failed validation:

13: http://signatory:6732/keys/tz2SctcTKRRyogSTC8BZcAyxjw9XLUuyjvgM
"032c305d6f5abe669dcb122008353a585ac26f0f24c61339b419f7f007ac9adbfa6c01c8c903c8f37924fb8c158b58b78f5cebdf54d4028a020e9b0100c0843d00006b82198cb179e8306c1bedd08f12dc863f32888600"
<<<<13: 200 OK

{ "signature":
"spsig15wjAnArhsnWM7jt7PRCeqq3JVVhey6qDCZHSVr4hbi1ir1wkQFGvAp8VKN4VFTC3dGHXo6YhSxyBXazwLQsgRggbZ33CR" }
The signer for http://signatory:6732/tz2SctcTKRRyogSTC8BZcAyxjw9XLUuyjvgM produced an invalid signature

passed validation:

13: http://signatory:6732/keys/tz2SctcTKRRyogSTC8BZcAyxjw9XLUuyjvgM
"039c6bbca75e6863101a5e1849d830b085d4e721bcf94487d4311a457fee67593f6c01c8c903c8f37924fb8c158b58b78f5cebdf54d4028a020e9b0100c0843d00006b82198cb179e8306c1bedd08f12dc863f32888600"
<<<<13: 200 OK
{ "signature":
"spsig1MKbCc5VXVYzQntnRKvxyLYDajX4S6XFGt9R1vGGzDhXXsXTnQ7GzB2nBP1VDHVkrcyx4wTwWGyFTC5DtaY2kW3axE1o2b" }
14: http://flextesa:20000/chains/main/blocks/head/helpers/preapply/operations

{ "name": "awstz2",
"value":
{ "locator":
"http://signatory:6732/tz2SctcTKRRyogSTC8BZcAyxjw9XLUuyjvgM",
"key": "sppk7cqbfSsBjf6ixegAbkBz9YP4TBM2dSsosfFzrurWShQRqvAo3YH" } },