Add ECDSA over secp256k1 signatures and verification #490

Vindaar · 2024-12-12T19:03:45Z

This adds support for the Elliptic Curve Digital Signature Algorithm (ECDSA) defined over secp256k1.

The current API supports:

Currently still missing:

batch verification
public key recovery
more test cases (currently: script to generate signatures using OpenSSL and verify those with Constantine). Maybe sign messages with CTT & verify using OpenSSL

In addition, there are a few minor things I'm unsure about. Will highlight them below.

(We might want to take this out before we merge?)

Adds a test vector generator and a test case (and test vectors) for verification of OpenSSL generated signatures.

Vindaar · 2024-12-12T19:04:23Z

constantine/ecdsa/ecdsa.nim

+    ## XXX: raise / what else to do if `sysrand` call fails?
+    doAssert b.limbs.sysrand()


In case this were to fail, do we want a different failure mode?

For now it's fine not to address this, AFAIK most protocols use RFC6979 to avoid adding the RNG to the attack surface. If the RNG fails to produce data, I'm not even sure what it means at the OS level, no more entropy maybe?

Not sure either to be honest, but given that we do return a bool I didn't just want to discard it.

Vindaar · 2024-12-12T19:05:23Z

constantine/ecdsa/ecdsa.nim

+    ## XXX: smarter way for this?
+    let r = Fr[C].fromBig(rx.toBig())


We need to convert the coordinate in Fp to a scalar in Fr. The "best" way I could come up with is this to big / from big conversion. Is there a cleaner way?

This is the best generic conversion.

In theory for secp256k1, you could take mod r because Fp/Fr can be both use canonical representation and still have a fast reduction (#445) however at the moment only Fp uses canonical repr and fast reduction while Fr uses Montgomery representation and Montgomery reduction.

mratsim

Note that for Ethereum the hash function is Keccak. But it's good to have both and be hash agnostic to help test.

The out-of-place function will need to be replaced by in-place.

You can check constantine/signatures/bls_signatures.nim in general for API guidance.
Also https://github.com/bitcoin-core/secp256k1 which is the reference implementation for Bitcoin and Ethereum to implement serializer without dynamic allocation.
And https://github.com/status-im/nim-eth/blob/master/eth/common/keys.nim for how secp256k1 is actually used in Ethereum

The PEM/DER part is unused in blockchain, but it's helpful for testing against OpenSSL, and if we ever want to add TLS/QUIC support to Constantine, it wil be necessary.

Regarding testing vs openssl if the API for ECDSA is less messy than hashes, it can be used as a library like so

constantine/tests/t_hash_sha256_vs_openssl.nim

Lines 8 to 54 in 13d5bb7

    
           # Deal with platform mess 
        
           # -------------------------------------------------------------------- 
        
           when defined(windows): 
        
             when sizeof(int) == 8: 
        
               const DLLSSLName* = "(libssl-1_1-x64|ssleay64|libssl64).dll" 
        
             else: 
        
               const DLLSSLName* = "(libssl-1_1|ssleay32|libssl32).dll" 
        
           else: 
        
             when defined(macosx) or defined(macos) or defined(ios): 
        
               const versions = "(.1.1|.38|.39|.41|.43|.44|.45|.46|.47|.48|.10|.1.0.2|.1.0.1|.1.0.0|.0.9.9|.0.9.8|)" 
        
             else: 
        
               const versions = "(.1.1|.1.0.2|.1.0.1|.1.0.0|.0.9.9|.0.9.8|.48|.47|.46|.45|.44|.43|.41|.39|.38|.10|)" 
        
             when defined(macosx) or defined(macos) or defined(ios): 
        
               const DLLSSLName* = "libssl" & versions & ".dylib" 
        
             elif defined(genode): 
        
               const DLLSSLName* = "libssl.lib.so" 
        
             else: 
        
               const DLLSSLName* = "libssl.so" & versions 
        
           # OpenSSL wrapper 
        
           # -------------------------------------------------------------------- 
        
           # OpenSSL removed direct use of their SHA256 function. https://github.com/openssl/openssl/commit/4d49b68504cc494e552bce8e0b82ec8b501d5abe 
        
           # It isn't accessible anymore in Windows CI on Github Action. 
        
           # But the new API isn't expose on Linux :/ 
        
           # TODO: fix Windows 
        
           when not defined(windows): 
        
             proc SHA256[T: byte|char]( 
        
                   msg: openarray[T], 
        
                   digest: ptr array[32, byte] = nil 
        
                 ): ptr array[32, byte] {.noconv, dynlib: DLLSSLName, importc.} 
        
             # proc EVP_Q_digest[T: byte|char]( 
        
             #                 ossl_libctx: pointer, 
        
             #                 algoName: cstring, 
        
             #                 propq: cstring, 
        
             #                 data: openArray[T], 
        
             #                 digest: var array[32, byte], 
        
             #                 size: ptr uint): int32 {.noconv, dynlib: DLLSSLName, importc.} 
        
             proc SHA256_OpenSSL[T: byte|char]( 
        
                   digest: var array[32, byte], 
        
                   s: openArray[T]) = 
        
               discard SHA256(s, digest.addr) 
        
               # discard EVP_Q_digest(nil, "SHA256", nil, s, digest, nil)

and we can remove the random messages json

mratsim · 2024-12-12T19:49:27Z

constantine/ecdsa/ecdsa.nim

@@ -0,0 +1,356 @@
+import


ECDSA should be in the constantine/signatures folder

mratsim · 2024-12-12T19:51:30Z

constantine/ecdsa/ecdsa.nim

@@ -0,0 +1,356 @@
+import
+  ../hashes/h_sha256,


It should be hash agnostic, with it's actual instantiation using a predefined hash.

See the function signature of BLS signatures

constantine/constantine/signatures/bls_signatures.nim

Lines 51 to 88 in 13d5bb7

func coreSign*[Sig, SecKey](

signature: var Sig,

secretKey: SecKey,

message: openArray[byte],

H: type CryptoHash,

k: static int,

augmentation: openArray[byte],

domainSepTag: openArray[byte]) {.genCharAPI.} =

## Computes a signature for the message from the specified secret key.

##

## Output:

## - `signature` is overwritten with `message` signed with `secretKey`

##

## Inputs:

## - `Hash` a cryptographic hash function.

## - `Hash` MAY be a Merkle-Damgaard hash function like SHA-2

## - `Hash` MAY be a sponge-based hash function like SHA-3 or BLAKE2

## - Otherwise, H MUST be a hash function that has been proved

## indifferentiable from a random oracle [MRH04] under a reasonable

## cryptographic assumption.

## - k the security parameter of the suite in bits (for example 128)

## - `output`, an elliptic curve point that will be overwritten.

## - `augmentation`, an optional augmentation to the message. This will be prepended,

## prior to hashing.

## This is used for building the "message augmentation" variant of BLS signatures

## https://www.ietf.org/archive/id/draft-irtf-cfrg-bls-signature-05.html#section-3.2

## which requires `CoreSign(SK, PK || message)`

## and `CoreVerify(PK, PK || message, signature)`

## - `message` is the message to hash

## - `domainSepTag` is the protocol domain separation tag (DST).

type ECP_Jac = EC_ShortW_Jac[Sig.F, Sig.G]

var sig {.noInit.}: ECP_Jac

H.hashToCurve(k, sig, augmentation, message, domainSepTag)

sig.scalarMul(secretKey)

signature.affine(sig)

mratsim · 2024-12-12T19:53:05Z

constantine/ecdsa/ecdsa.nim

+# For easier readibility, define the curve and generator
+# as globals in this file
+const C* = Secp256k1
+const G = Secp256k1.getGenerator("G1")


The signature scheme itself should be generic in constantine/signatures and concrete implementations should appear in constantine/my_protocol.nim

Yup, fair enough. I wrote it by defining the curve like this so that the code itself is already an (almost) valid generic / static proc. Didn't know if we want to make it fully generic immediately or start with secp256k1 only. Will address it.

mratsim · 2024-12-12T19:54:17Z

constantine/ecdsa/ecdsa.nim

+  var h {.noinit.}: sha256
+  h.init()
+  h.update(message)
+  h.finish(result)


This duplicates constantine/hashes.nim

constantine/constantine/hashes.nim

Lines 17 to 54 in 13d5bb7

type

CryptoHash* = concept h, var ctx, type H

## Interface of a cryptographic hash function

##

## - digestSizeInBytes is the hash output size in bytes

## - internalBlockSize, in bits:

## hash functions are supposed to ingest fixed block size

## that are padded if necessary

## - SHA256 block size is 64 bits

## - SHA512 block size is 128 bits

## - SHA3-512 block size is 72 bits

# should we avoid int to avoid exception? But they are compile-time

H.digestSize is static int

H.internalBlockSize is static int

# Context

# -------------------------------------------

ctx.init()

ctx.update(openarray[byte])

ctx.finish(var array[H.digestSize, byte])

ctx.clear()

func hash*[DigestSize: static int](

HashKind: type CryptoHash,

digest: var array[DigestSize, byte],

message: openArray[byte],

clearMem = false) {.genCharAPI.} =

## Produce a digest from a message

static: doAssert DigestSize == HashKind.type.digestSize

var ctx {.noInit.}: HashKind

ctx.init()

ctx.update(message)

ctx.finish(digest)

if clearMem:

ctx.clear()

mratsim · 2024-12-12T19:58:28Z

constantine/ecdsa/ecdsa.nim

+  h.update(message)
+  h.finish(result)
+
+proc toBytes(x: Fr[C] | Fp[C]): array[32, byte] =


Suggested change

proc toBytes(x: Fr[C] | Fp[C]): array[32, byte] =

proc toBytes(x: Fr[C] | Fp[C]): array[32, byte] {.noInit.} =

I assume this is temporary for debugging? Returning big arrays tends to generate bad code:

Out-of-place functions lead to bad codegen #145

Internal API: in-place vs result #21
It's also a recommendation in C / C++ to avoid return values that don't fit in registers.

Ah, that either slipped my mind or I just wasn't aware of it! Good to know. Will change it to an in-place variant.

mratsim · 2024-12-12T20:13:17Z

constantine/ecdsa/ecdsa.nim

+    ## XXX: smarter way for this?
+    let r = Fr[C].fromBig(rx.toBig())


This is the best generic conversion.

In theory for secp256k1, you could take mod r because Fp/Fr can be both use canonical representation and still have a fast reduction (#445) however at the moment only Fp uses canonical repr and fast reduction while Fr uses Montgomery representation and Montgomery reduction.

mratsim · 2024-12-12T20:17:51Z

constantine/ecdsa/ecdsa.nim

+proc getPublicKey*(pk: Fr[C]): EC_ShortW_Aff[Fp[C], G1] =
+  ## Derives the public key from a given private key,
+  ## `privateKey · G` in affine coordinates.
+  result = (pk * G).getAffine()


Can reuse

constantine/constantine/signatures/bls_signatures.nim

Lines 37 to 49 in 13d5bb7

func derivePubkey*[Pubkey, SecKey](pubkey: var Pubkey, seckey: SecKey) =

## Generates the public key associated with the input secret key.

##

## The secret key MUST be in range (0, curve order)

## 0 is INVALID

const Group = Pubkey.G

type Field = Pubkey.F

const EC = Field.Name

var pk {.noInit.}: EC_ShortW_Jac[Field, Group]

pk.setGenerator()

pk.scalarMul(seckey)

pubkey.affine(pk)

mratsim · 2024-12-12T20:24:19Z

constantine/ecdsa/ecdsa.nim

+  # 6. Verify r_computed equals provided r
+  result = bool(r_computed == signature.r)
+
+proc generatePrivateKey*(): Fr[C] =


This is not needed at this level, except maybe for testing.

Blockchain protocols have special ways to generate a private key usually from a mnemonic or via a HD derivation path (hierarchical deterministic) like BIP32 https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki or EIP-2333 (https://github.com/mratsim/constantine/blob/13d5bb7/constantine/ethereum_eip2333_bls12381_key_derivation.nim)

Yeah, I assumed having a way to generate a random private key might be useful. For testing we can just move it to a test file of course.

mratsim · 2024-12-12T20:25:28Z

constantine/ecdsa/ecdsa.nim

+  # Combine all parts
+  let contents = algoId & pubKeyEncoded
+  result.add(byte(contents.len))
+  result.add(contents)


For constantine/serialization and without dynamic alloc

mratsim · 2024-12-12T20:45:24Z

constantine/ecdsa/ecdsa.nim

+    # with `h`: message hash as `Fr[C]` (if we didn't use SHA256 w/ 32 byte output
+    # we'd need to truncate to N bits for N being bits in modulo `n`)
+    k.inv()
+    var s = (k * (message_hash + r * privateKey))


It might be faster to do k*privateKey and then a multiscalarmul([k, k*privkey], [message_hash, r]).

Note that the out-of-place functions are for convenience when debugging/testing/developping but not for production code (#413)

I understand. Still hasn't fully sunk in (intuitively) to avoid out-of-place procs! My bad. 😁

Inspired by Bitcoin's implementation: https://github.com/bitcoin-core/secp256k1/blob/f79f46c70386c693ff4e7aef0b9e7923ba284e56/src/ecdsa_impl.h#L171-L193 Essentially, we just have a 72 byte buffer and only fill it up to `len` bytes.

The size is now a static argument of the type, depending on the curve. `DERSigSize` can be used to calculate the size required for the given curve.

Also cleans up the imports of the ECDSA file and adds the copyright header

Vindaar · 2024-12-24T11:18:16Z

Vindaar added 8 commits December 8, 2024 23:58

[zoo] add generator for secp256k1

a322b87

[ECDSA] add initial ECDSA signing / verifying implementation

83b0c6f

[ecdsa] fix imports

177fa84

[ecdsa] export Secp256k1 as C for convenience

e4144f5

(We might want to take this out before we merge?)

[ecdsa] export toDER proc

f7cbc18

[ecdsa] use isZero instead of old zero comparison

f38aa2e

[ecdsa] rename private key generator & add private -> public key

ab420c5

[tests] add test cases for ECDSA signature verification

aadaa6b

Adds a test vector generator and a test case (and test vectors) for verification of OpenSSL generated signatures.

Vindaar commented Dec 12, 2024

View reviewed changes

mratsim reviewed Dec 12, 2024

View reviewed changes

Vindaar added 15 commits December 21, 2024 13:13

[ecdsa] handle some .noinit. cases

64375f9

[ecdsa] turn toBytes, arrayWith into in-place procedures

fb4b414

[ecdsa] clean up comment about Fp -> Fr conversion

a145ec5

[ecdsa] replace toPemPrivateKey/PublicKey by in-place array variants

de6205d

[ecdsa] replace toDER by non allocating variant

5ea2d19

Inspired by Bitcoin's implementation: https://github.com/bitcoin-core/secp256k1/blob/f79f46c70386c693ff4e7aef0b9e7923ba284e56/src/ecdsa_impl.h#L171-L193 Essentially, we just have a 72 byte buffer and only fill it up to `len` bytes.

[ecdsa] replace out-of-place arithmetic by in-place

93ac7bd

[ecdsa] move ECDSA implementation to ~signatures~ directory

d9fb203

[ecdsa] remove dependence on explicit SHA256 hash function

2dac4c6

[ecdsa] make DERSignature generic under curve by having static size

9269cef

The size is now a static argument of the type, depending on the curve. `DERSigSize` can be used to calculate the size required for the given curve.

[ecdsa] turn more procs generic over curve and hash function

2c6decf

[ecdsa] replace sign/verify API by one matching BLS signatures

e8967ce

[ecdsa] remove global curve & generator constants

b893080

[ecdsa] correctly handle truncation of digests > Fr BigInts

d54908b

create file for common signature ops, derivePubkey for ECDSA & BLS

18425cf

Also cleans up the imports of the ECDSA file and adds the copyright header

create file specifically for ECDSA over secp256k1

6df2c10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ECDSA over secp256k1 signatures and verification #490

Add ECDSA over secp256k1 signatures and verification #490

Vindaar commented Dec 12, 2024 •

edited

Loading

Vindaar Dec 12, 2024

mratsim Dec 12, 2024

Vindaar Dec 13, 2024

Vindaar Dec 12, 2024

mratsim Dec 12, 2024

mratsim left a comment

mratsim Dec 12, 2024

mratsim Dec 12, 2024

mratsim Dec 12, 2024

Vindaar Dec 13, 2024

mratsim Dec 12, 2024

mratsim Dec 12, 2024

Vindaar Dec 13, 2024

mratsim Dec 12, 2024

mratsim Dec 12, 2024

mratsim Dec 12, 2024

Vindaar Dec 13, 2024

mratsim Dec 12, 2024

mratsim Dec 12, 2024 •

edited

Loading

Vindaar Dec 13, 2024

Vindaar commented Dec 24, 2024

		## XXX: raise / what else to do if `sysrand` call fails?
		doAssert b.limbs.sysrand()

		## XXX: smarter way for this?
		let r = Fr[C].fromBig(rx.toBig())

	# Deal with platform mess
	# --------------------------------------------------------------------
	when defined(windows):
	when sizeof(int) == 8:
	const DLLSSLName* = "(libssl-1_1-x64\|ssleay64\|libssl64).dll"
	else:
	const DLLSSLName* = "(libssl-1_1\|ssleay32\|libssl32).dll"
	else:
	when defined(macosx) or defined(macos) or defined(ios):
	const versions = "(.1.1\|.38\|.39\|.41\|.43\|.44\|.45\|.46\|.47\|.48\|.10\|.1.0.2\|.1.0.1\|.1.0.0\|.0.9.9\|.0.9.8\|)"
	else:
	const versions = "(.1.1\|.1.0.2\|.1.0.1\|.1.0.0\|.0.9.9\|.0.9.8\|.48\|.47\|.46\|.45\|.44\|.43\|.41\|.39\|.38\|.10\|)"

	when defined(macosx) or defined(macos) or defined(ios):
	const DLLSSLName* = "libssl" & versions & ".dylib"
	elif defined(genode):
	const DLLSSLName* = "libssl.lib.so"
	else:
	const DLLSSLName* = "libssl.so" & versions

	# OpenSSL wrapper
	# --------------------------------------------------------------------

	# OpenSSL removed direct use of their SHA256 function. https://github.com/openssl/openssl/commit/4d49b68504cc494e552bce8e0b82ec8b501d5abe
	# It isn't accessible anymore in Windows CI on Github Action.
	# But the new API isn't expose on Linux :/

	# TODO: fix Windows
	when not defined(windows):
	proc SHA256[T: byte\|char](
	msg: openarray[T],
	digest: ptr array[32, byte] = nil
	): ptr array[32, byte] {.noconv, dynlib: DLLSSLName, importc.}

	# proc EVP_Q_digest[T: byte\|char](
	# ossl_libctx: pointer,
	# algoName: cstring,
	# propq: cstring,
	# data: openArray[T],
	# digest: var array[32, byte],
	# size: ptr uint): int32 {.noconv, dynlib: DLLSSLName, importc.}

	proc SHA256_OpenSSL[T: byte\|char](
	digest: var array[32, byte],
	s: openArray[T]) =
	discard SHA256(s, digest.addr)
	# discard EVP_Q_digest(nil, "SHA256", nil, s, digest, nil)

	func coreSign*[Sig, SecKey](
	signature: var Sig,
	secretKey: SecKey,
	message: openArray[byte],
	H: type CryptoHash,
	k: static int,
	augmentation: openArray[byte],
	domainSepTag: openArray[byte]) {.genCharAPI.} =
	## Computes a signature for the message from the specified secret key.
	##
	## Output:
	## - `signature` is overwritten with `message` signed with `secretKey`
	##
	## Inputs:
	## - `Hash` a cryptographic hash function.
	## - `Hash` MAY be a Merkle-Damgaard hash function like SHA-2
	## - `Hash` MAY be a sponge-based hash function like SHA-3 or BLAKE2
	## - Otherwise, H MUST be a hash function that has been proved
	## indifferentiable from a random oracle [MRH04] under a reasonable
	## cryptographic assumption.
	## - k the security parameter of the suite in bits (for example 128)
	## - `output`, an elliptic curve point that will be overwritten.
	## - `augmentation`, an optional augmentation to the message. This will be prepended,
	## prior to hashing.
	## This is used for building the "message augmentation" variant of BLS signatures
	## https://www.ietf.org/archive/id/draft-irtf-cfrg-bls-signature-05.html#section-3.2
	## which requires `CoreSign(SK, PK \|\| message)`
	## and `CoreVerify(PK, PK \|\| message, signature)`
	## - `message` is the message to hash
	## - `domainSepTag` is the protocol domain separation tag (DST).

	type ECP_Jac = EC_ShortW_Jac[Sig.F, Sig.G]

	var sig {.noInit.}: ECP_Jac
	H.hashToCurve(k, sig, augmentation, message, domainSepTag)
	sig.scalarMul(secretKey)

	signature.affine(sig)

	type
	CryptoHash* = concept h, var ctx, type H
	## Interface of a cryptographic hash function
	##
	## - digestSizeInBytes is the hash output size in bytes
	## - internalBlockSize, in bits:
	## hash functions are supposed to ingest fixed block size
	## that are padded if necessary
	## - SHA256 block size is 64 bits
	## - SHA512 block size is 128 bits
	## - SHA3-512 block size is 72 bits

	# should we avoid int to avoid exception? But they are compile-time
	H.digestSize is static int
	H.internalBlockSize is static int

	# Context
	# -------------------------------------------
	ctx.init()
	ctx.update(openarray[byte])
	ctx.finish(var array[H.digestSize, byte])
	ctx.clear()

	func hash*[DigestSize: static int](
	HashKind: type CryptoHash,
	digest: var array[DigestSize, byte],
	message: openArray[byte],
	clearMem = false) {.genCharAPI.} =
	## Produce a digest from a message
	static: doAssert DigestSize == HashKind.type.digestSize

	var ctx {.noInit.}: HashKind
	ctx.init()
	ctx.update(message)
	ctx.finish(digest)

	if clearMem:
	ctx.clear()

	proc toBytes(x: Fr[C] \| Fp[C]): array[32, byte] =
	proc toBytes(x: Fr[C] \| Fp[C]): array[32, byte] {.noInit.} =

	func derivePubkey*[Pubkey, SecKey](pubkey: var Pubkey, seckey: SecKey) =
	## Generates the public key associated with the input secret key.
	##
	## The secret key MUST be in range (0, curve order)
	## 0 is INVALID
	const Group = Pubkey.G
	type Field = Pubkey.F
	const EC = Field.Name

	var pk {.noInit.}: EC_ShortW_Jac[Field, Group]
	pk.setGenerator()
	pk.scalarMul(seckey)
	pubkey.affine(pk)

Add ECDSA over secp256k1 signatures and verification #490

Are you sure you want to change the base?

Add ECDSA over secp256k1 signatures and verification #490

Conversation

Vindaar commented Dec 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mratsim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mratsim Dec 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vindaar commented Dec 24, 2024

Vindaar commented Dec 12, 2024 •

edited

Loading

mratsim Dec 12, 2024 •

edited

Loading