Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v0 Oracle Changes #167

Open
wants to merge 23 commits into
base: master
Choose a base branch
from
Open

Conversation

nkohen
Copy link
Contributor

@nkohen nkohen commented Jun 8, 2021

As per our spec meeting discussions, this PR:

  1. Splits the oracle key into separate announcement and attestation keys
  2. Adds Schnorr signatures by the nonce keys to the announcement message
  3. Allows for range timestamps in oracle events

@nkohen nkohen added this to the v0.1 milestone Jun 8, 2021
@nkohen nkohen requested review from Tibo-lg and Christewart June 8, 2021 18:59
Messaging.md Outdated
* [`oracle_event`:`oracle_event`]

where `signature` is a Schnorr signature over a sha256 hash of the serialized `oracle_event`, using the tag `announcement/v0`.
where all `pok_signature`s are Schnorr signatures using the tag `announcement/v0` over a sha256 hash of the serialized `x_point`s concatenated in order and concatenated with `oracle_event`.
That is to say, the signature hash is of the message `oracle_announcement_public_key||oracle_attestation_public_key||oracle_nonces||oracle_event`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thoughts below :)

  1. I think I've expressed this before -- sorry if I'm going over stuff that has already been discussed between you guys -- but I think the announcement should not have the oracle's public keys in it. They must be known beforehand. IIRC they are just being used as identifiers? I still think that is a dangerous choice but even in this case the attestation_pok_signature should not be in the announcement. The attestation_pok_signature should have been verified when the user originally saved the oracle's public key not at the time of any particular announcement.

  2. I think what was there before with announcement_signature made more sense because it's not a pok it's just a signature. Having all the oracle crypto data in oracle_event and having the signature over all of it in oracle_announcement made more sense to me.

  3. Optional extra: IMO it's fine to do MuSig to aggregate all the nonce_pok_signatures into a single thing. Perhaps that is more specification work than you want to do now though. The reason I think it's fine is that the plan is to use MuSig to do cross-input aggregation which implies that it is secure for this purpose as well in a kind of proof of possession model which I think is reasonable here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left the announcement key as an identifier, but removed attestation key and sig from announcement and moved it to a new oracle_keys message.

@nkohen
Copy link
Contributor Author

nkohen commented Jun 11, 2021

I've added a new oracle_keys message but I'm not sure I'm the best person to write in Oracle.md about how it is used, and likewise to write rationales for why we need two public keys and all the proofs of knowledge. @LLFourn would you be willing to write at least these last two things?

@Tibo-lg Tibo-lg added the breaking The proposed change breaks backward compatibility label Sep 16, 2021
Messaging.md Outdated
* [`string`:`oracle_description`]
* [`u32`:`timestamp`]
* [`signature`:`announcement_pok_signature`]
* [`signature`:`attestation_pok_signature`]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think announcement_pok_signature is the right name because it's purpose is not a proof of knowledge. It's purpose is to link the other details to this announcement key. Should just be oracle_metadata_signature or something.

attestation_pok_signature is not guaranteeing any security property -- rogue nonce attacks are easier than the rogue key attack here. We decided not to address this security flaw in this revision so there's not point in having it here. I think we should remove it.

In the future, I think the correct solution is to include a proof of possession via MuSig(attestation_key, nonce_0, nonce_1 .. nonce_n ) in each announcement.

@LLFourn
Copy link
Contributor

LLFourn commented Jan 12, 2022

Meeting note: attestation schemes should be declared separately from the event information. Taking this as an example:

$ curl 'https://h00.ooo/x/BitMEX/BXBT/2022-01-12T01:00:00.price?n=20' | jq .announcement.oracle_event.data -r  |jq

 {
	"id": "/x/BitMEX/BXBT/2022-01-12T01:00:00.price?n=20",
	"expected-outcome-time": "2022-01-12T01:00:00",
	"descriptor": {
		"type": "digit-decomposition",
		"is_signed": false,
		"n_digits": 20,
		"unit": null
	},
	"schemes": {
		"olivia-v1": {
			"nonces": [
				"ea4c2c81a65e4df1c98515bf2af6a0e63ed086f803e4637130447056f2ff15c8",
				"e7a46ba921fab39fe9e953e430f72e7325c43aba4d181517b9078dfc09637054",
				"f9dce3712dfa25c3ab516bef99c5e828b349e53a134321938612f5b5af9dd4bf",
				"74ef83448b8fc533a1dc33f1f58d1c0bd6cfb086f9369d627d917884ccc21da7",
				"63c0e43218a9e31e0ff5b34ab492a0e9fa100eebfc888c7efc51791a55cd8283",
				"9e0c36a0607c16d6d6284c640a8ac00ab65a772c3fa0981be022520a9f44e47a",
				"e288c14d070f463a7d71bc330b4786c574aa8be02f7e7ad00e4fc9a6bdcbb32b",
				"e758b66c5df5ef161f00081fe87685230a0b2846863cf3da00828d112b71220f",
				"1ed08c216fddc19622553925ef9d2753715c7c01201a9a47ae4233a4878cc030",
				"8e5bc962f77346e721f25fe7871a03e95bf38a86e564c5729c86dc2fc5eb6871",
				"68843c83a7c9829f6ef5c4ae77067f9d7f72703ee9b8c6d5498a873180b15574",
				"aee0085e8c98a2afbc7d43d5774e145cfa0168d9e22506dc79fa1b74718e0043",
				"e4c038f95dac570ffe276c16a3518983abcc8216e9d318de39ca42c70d54ea3e",
				"cb4f5cc032ca1e343229256bb0d911b0e04ec2f409d0fa837cedd973ac45b742",
				"1e63f2116013988df505b26ddebec1adaeb41c7a2b55baf375f599431b5fa242",
				"1792e2fed1c346900702dcc208e286ea8a6865975cd961c834a275c505072247",
				"a5672e32a331cc360317a7329fc7f70d7843586ab4a24b961336183cc764bef9",
				"52272eca966fe72012abffcc481276dce404c27e4db783374a5a7c001f9477d2",
				"98649a7620abd662e2ff699659f58b7cf19d9ad4e5257180cc36ba79fff97c68",
				"92dddc74b7491067fab985962c60b8e0ce43e820e35405f6ac2a577e4b6eca6e"
			]
		},
		"ecdsa-v1": {}
	}
} 

so in the oracle_event you have the descriptor and other metadata like times on one level and then you have a list of attestation schemes and their parameters. Note that nonces and public keys are always parameters to a certain attestation scheme (my oracle is still missing the public key which should be in there with the nonces).

Copy link
Contributor

@Christewart Christewart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Tibo-lg @nkohen I believe this needs to be updated with the format in #163. We need to figure out which messages we want to keep as tlvs and what data structures we need to have keep the format in #163

@Tibo-lg
Copy link
Member

Tibo-lg commented May 31, 2022

@Tibo-lg @nkohen I believe this needs to be updated with the format in #163. We need to figure out which messages we want to keep as tlvs and what data structures we need to have keep the format in #163

Unless there is strong arguments, I think it'd be best to move everything to the same format as #163. I can update this PR if we agree on that (or probably make a new one as I'm not sure I can push to @nkohen branch).

@Christewart
Copy link
Contributor

Christewart commented Jun 1, 2022

@Tibo-lg @nkohen I believe this needs to be updated with the format in #163. We need to figure out which messages we want to keep as tlvs and what data structures we need to have keep the format in #163

Unless there is strong arguments, I think it'd be best to move everything to the same format as #163. I can update this PR if we agree on that (or probably make a new one as I'm not sure I can push to @nkohen branch).

I think it comes down to this:

Do we expect announcements/attestations to be sent over the network directly to a peer, or do we always expected them to bet fetched from a 3rd party service like oracle.suredbits.com ? If we think the former, we should keep TLVs, if we think the latter, I think it might be ok to remove them (although I do think this makes parsing harder for the 3rd party service, there is no easy identifier for what the thing is being submitted to the service. A TLV is unique, whereas a subtype is not).

EDIT:

Thinking about this more, my own standard I laid out above is sort of violated already. We host contract_info on oracle.suredbits.com, for instance here is a list of numeric contract infos: https://oracle.suredbits.com/contract/numeric

Since contract_info no longer has a TLV prefix, they cannot be easily identified for parsing. In the case of contract info, we only support building them on the site (and not directly submitting them like we do oracle announcements/attesattions), so this is slightly different. Removing TLVs seems to really break down modularity and ease of parsing of independent pieces of the protocol.

@Tibo-lg
Copy link
Member

Tibo-lg commented Jun 2, 2022

Do we expect announcements/attestations to be sent over the network directly to a peer, or do we always expected them to bet fetched from a 3rd party service like oracle.suredbits.com ? If we think the former, we should keep TLVs, if we think the latter, I think it might be ok to remove them (although I do think this makes parsing harder for the 3rd party service, there is no easy identifier for what the thing is being submitted to the service. A TLV is unique, whereas a subtype is not).

I would think that we should at least plan for enabling peers to exchange announcements/attestations directly. But in that case I think they'd need to be in wire message format, not TLV?

@Christewart
Copy link
Contributor

I'm not sure how you have implemented the wire message format, but in our code base the wire message format uses the nested TLV.

case class LnMessage[+T <: TLV](tlv: T) extends NetworkElement {
  require(tlv.tpe.toLong <= 65535L, s"LN Message format requires UInt16 types")
  val tpe: UInt16 = UInt16(tlv.tpe.toInt)
  val payload: ByteVector = tlv.value
  override lazy val bytes: ByteVector = tpe.bytes ++ payload
  val typeName: String = tlv.typeName
}

The key line is

val tpe: UIn16 = UInt16(tlv.tpe.toInt)

Since the announcement is no longer a TLV, it cannot have a tpe which is only defined for TLVs. How are you proposing we send non TLVs with the wire message format?

@Tibo-lg
Copy link
Member

Tibo-lg commented Jun 5, 2022

I'm not sure what you mean by "nested TLV"? The wire message doesn't have any length prefix and uses a u16 instead of a bigsize as written in the specs.

@Christewart
Copy link
Contributor

the specs.

Meanwhile all typed sub-messages (which follow TLV format) will

^ What I am talking about. I can detail our implementation in Scala if you'd like but I think we have a misunderstanding of what the specification says. You adhere to this with all of the test vectors on #163 (which is why we are compatible), but want to break this convention -- IIUC -- on this PR.

If we are intending to send announcements/attestations over the wire independently they will need to be TLVs

@Tibo-lg
Copy link
Member

Tibo-lg commented Jun 6, 2022

Sorry I should have linked to the spec after #163 : https://github.com/Tibo-lg/dlcspecs-1/blob/serialization-update-proposal/Messaging.md#message-format

You adhere to this with all of the test vectors on #163 (which is why we are compatible), but want to break this convention -- IIUC -- on this PR.

This is because you wanted to keep the oracle message format intact so as not to have to update your oracle infrastructure before this PR.

If we are intending to send announcements/attestations over the wire independently they will need to be TLVs

I don't see why. But maybe it will be easier to discuss it during the spec meeting? It feels indeed like we are not understanding each others.

@Christewart
Copy link
Contributor

Christewart commented Jun 6, 2022

Sorry I should have linked to the spec after #163 : https://github.com/Tibo-lg/dlcspecs-1/blob/serialization-update-proposal/Messaging.md#message-format

You adhere to this with all of the test vectors on #163 (which is why we are compatible), but want to break this convention -- IIUC -- on this PR.

This is because you wanted to keep the oracle message format intact so as not to have to update your oracle infrastructure before this PR.

If we are intending to send announcements/attestations over the wire independently they will need to be TLVs

I don't see why. But maybe it will be easier to discuss it during the spec meeting? It feels indeed like we are not understanding each others.

A TLV stream is defined for each wire message

How are you suggesting to parse an announcement if it isn't a TLV? IIUC, we will have the length of the announcement, but no unique identifier? What is the unique identifier for an announcement so that I can identify it as an announcement before parsing the payload?

Perhaps it would be useful to make this more concrete, can you give me an example announcement you are envisioning in wire message format?

@Tibo-lg
Copy link
Member

Tibo-lg commented Jun 6, 2022

IIUC, we will have the length of the announcement, but no unique identifier? What is the unique identifier for an announcement so that I can identify it as an announcement before parsing the payload?

Wire messages have a u16 type prefix, isn't that enough?

Perhaps it would be useful to make this more concrete, can you give me an example announcement you are envisioning in wire message format?

There might be some things that need to be updated, but roughly the spec should look like it was before the commit that reverted the oracle related messages to the old format: 225e3b0#diff-b8a9c2888fe05c198758fbe393798cd0802b578f7eb07dbf82cc2c90616f97c3R407

Concretely an announcement would be serialized similarly to an offer_dlc message, with first a u16 type prefix, and then the other fields (which wouldn't be TLVs as they are now in #163). When embedding an announcement inside an offer_dlc, I think we agreed that we would just keep the wire message format as is to make things simple, and so you'd have also the type prefix (which you wouldn't actually need in the context of the offer message), and again the other fields of announcement following.

@Christewart
Copy link
Contributor

Christewart commented Jun 6, 2022

with first a u16 type prefix

IMO, you are using a TLV but just not calling it a TLV :-). Or perhaps my mental model of TLVs is incorrect. But functionally speaking, I think we are in agreement.

An announcement needs a type identifier similar to offer_dlc in 163 right?

Screenshot from 2022-06-06 06-37-11

https://github.com/discreetlogcontracts/dlcspecs/pull/163/files#diff-e0f5b925f91a1c09c6daf26f9a7d28816cb9ff9f08863faca719b7ee0a1cc065R67

If that is the case, I don't really care about definitions, we are referring to the same thing with different definitions. I'll do some research later to correct my definitions -- if they are wrong -- so we can use a common language to describe the protocol.

@Tibo-lg
Copy link
Member

Tibo-lg commented Jun 16, 2022

I pushed updated test vectors including these oracle changes.

The questions that I have left:

  • Do we really need a timestamp for the metadata? (v0 Oracle Changes #167 (comment))
  • Do we need the oracle metadata in each announcement? (v0 Oracle Changes #167 (comment))
  • Are we sure we don't want to give proof of knowledge for attestation key and nonces?
  • (more cosmetic) Should we remove all the Oracle prefixes from the field names? It feels redundant to me (and I think we don't add them consistently).

@Tibo-lg
Copy link
Member

Tibo-lg commented Jun 16, 2022

Closes #183

@Tibo-lg
Copy link
Member

Tibo-lg commented Jun 23, 2022

Added proof of knowledge in the Schnorr scheme (@LLFourn when you have time to check)

Messaging.md Outdated
* [`signature:`proof_of_knowledge`]

where `proof_of_knowledge` ensures that the oracle knows the pre-image of the `attestation_public_key` and all `oracle_nonce`.
It is generated following bip-musig2 [signing](https://github.com/jonasnick/bips/blob/musig2/bip-musig2.mediawiki#signing) and [signature aggregation](https://github.com/jonasnick/bips/blob/musig2/bip-musig2.mediawiki#partial-signature-aggregation) algorithms, using the attestation key as the first key and the nonces as the following keys, in the same order as provided in the data structure, and the message `dlcoraclepok`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So does this block implementation on this until all libraries support musig2?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's up for us to decide. The alternatives to musig is to do separate proofs of knowledge, which is fine but we'll end up with rather big announcements for numerical events, with a signature per nonce.

Note it's mainly about binding to the musig2 implementation, once you have the bindings it's quite little work (you can see my implementation here

Copy link
Contributor

@Christewart Christewart Jun 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a musig2 implementation available to you?

Also does @matthewjablack have a musig2 implementation available to him?

We are just implementing it: bitcoin-s/bitcoin-s#4418

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also this brings up stability questions of the current musig2 spec. For instance, I know Schnorr was deployed on the bitcoin cash blockchain years before schnorr got deployed into Bitcoin.

I think things like XOnlyPubKeys weren't included in the schnorr spec, thus leading to incompatibilities between the two implementations.

We could end up in a similar situation if we don't wait for the specification to mature. I guess lnd is deploying musig2 (although not incorporating it in the LN spec, so its easier to change for now!)

https://docs.bitcoincashnode.org/doc/bips/

Screenshot from 2022-06-27 07-37-37

TLDR, how stable is musig2?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taproot hasn't been added to bitcoinjs-lib let alone musig2

If we wanted to get access to musig2 we'd likely need to get it from secp256k1-zkp and update cfd/cfd-core

This is a little annoying since it would mean validation for Oracle Announcements would require cfd-dlc-js and couldn't be done in native JS as it's done currently https://github.com/AtomicFinance/node-dlc/blob/master/packages/messaging/lib/messages/OracleAnnouncementV0.ts#L62

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Christewart

Do you have a musig2 implementation available to you?

Yes, WIP but it's there: BlockstreamResearch/rust-secp256k1-zkp#48

TLDR, how stable is musig2?

I don't know, maybe @LLFourn does?

@matthewjablack

This is a little annoying since it would mean validation for Oracle Announcements would require cfd-dlc-js and couldn't be done in native JS as it's done currently https://github.com/AtomicFinance/node-dlc/blob/master/packages/messaging/lib/messages/OracleAnnouncementV0.ts#L62

Two possible solutions would be use a wasm module generated from either cfd-core or rust-secp256k1-zkp (I think it could work, but honestly haven't tried yet), or if you have access to EC operations in JS implement the key aggregation algorithm which is all you need on client side (not sure it'd be a good idea tbh, and it might take some work).

We can probably discuss at the next meeting whether we want to do that or not. I think it's a bit nicer as it gives more compact and constant size announcements, but if people prefer to have separate signatures we can also go that way.

Copy link
Contributor

@LLFourn LLFourn Jun 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whatever happens with DLCs going forward I expect MuSig2 will be part of it so I think relying on the MuSig key aggregation function is not too much to ask to simplify things a bit. It should be stable soon.

One thing I will bring up is whether the authors of the spec think that MuSig is appropriate for this kind of thing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My personal opinion upon reading this is that perhaps we want to utilize sub types here and have the default (for now) be having all of them but in the future (specifically after musig2 gets a bip number, after which there will be breaking changes in the hash tags to include the bip number ... I think) I think it would be totally reasonable to introduce a new subtype for bip-??? musig aggregated proof of knowledge, which would likely become the default.

@Tibo-lg
Copy link
Member

Tibo-lg commented Jul 13, 2022

Updated specs and test vectors to use regular Schnorr signatures for proof of knowledge as agreed during spec meeting (using a type prefix to enable using something different in the future).

#### Rationale

Separate public keys are required for announcement and attestation because the attestation scheme
used in the current DLC spec reduces the security of non-attestation Schnorr signatures issued by the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any citation/reason why it degrades security? It seems we state this as truth without some sort of reason why.

Messaging.md Outdated

1. data:
* [`bigsize`:`nb_schemes`]
* [`nb_schemes*scheme`:`attestation_schemes`]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is attestations_schemes defined anywhere or is this a typo and meant to be schnorr_scheme?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a attestation_scheme type.

Messaging.md Outdated
* [`oracle_event`:`oracle_event`]

where `signature` is a Schnorr signature over a sha256 hash of the serialized `oracle_event`, using the tag `announcement/v0`.
where both the `announcement_signature` is a Schnorr signature over a sha256 hash of the serialized `oracle_event`, using the tag `announcement/v1`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the public key used to check against the announcement signature?

Copy link
Member

@Tibo-lg Tibo-lg Aug 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarified (it's the oracle_annoucement_public_key

Messaging.md Outdated
* [`x_point`:`oracle_attestation_public_key`]
* [`bigsize`: `nb_outcomes`]
* [`nb_signatures*attested_outcome`:`attested_outcomes`]
1. subtype: `attested_outcome`
Copy link
Contributor

@Christewart Christewart Aug 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I don't understand why this is necessary. Do we use this pattern any where else? Why can't we just serialize the signatures / outcomes with bigsize prefixes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking The proposed change breaks backward compatibility design oracle security
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants