multi: add test vectors for asset/address/proof/PSBT TLV encoding and VM/MS-SMT validation #326

guggero · 2023-05-26T13:05:19Z

Fixes #302.

This change is

dstadulis · 2023-06-13T19:00:51Z

action: go to ready for review

Roasbeef

Nice work with all the hlper functions! Should make it super easy to add new tests in the future.

The main thing I think I'm missing in this PR is: how are the vectors generated in the first place? I see the helper funcs, and also code to verify that we [ass the test vectors, but don't see the test vector generation code.

Roasbeef · 2023-06-15T23:58:42Z

asset/asset.go

@@ -66,11 +66,6 @@ var (
 	NUMSCompressedKey = ToSerialized(NUMSPubKey)
 	NUMSScriptKey     = ScriptKey{
 		PubKey: NUMSPubKey,
-		TweakedScriptKey: &TweakedScriptKey{


Does this inadvertently affect any instances where this script key was previously added to the db? So before we'd actually add the tweak, but now e'd end up leaving it out?

Hmm, good question. Maybe... But I don't think it should matter as we're never going to use the internal key as we always check whether it's the NUMS key before attempting to use it in an lnd RPC.

Roasbeef · 2023-06-16T00:04:22Z

asset/testdata/asset_tlv_encoding.json

+            },
+            "tx_witness": null,
+            "split_commitment": {
+              "proof": "00ffb1e8e8f2dc3b266452988cfe169aa73be25405eeead02ab5dd6b3c6fd0ca8d6700000000000000005a9d70bc8ac90cab6a114615f3d43a6d64c53e8a6279963a496fd3ee910a8c760000000000000000b72fb4b8936ef8080eac83b2e49b661e7c7f9a2db6880a57f72b8a0431cccc2f00000000000000006931a89cad83b819c624da132ec6ec211171ec53baf318524f82bfd72f2113440000000000000000406ab5f9adbe9160c02dbb9223777b8c1d2cc8c89901085291b4df20e2fc87f30000000000000000b4202a895a07ffbe281a3c77093564996bbe95742a6c220b607369c23dfe7f410000000000000000671b49495b6587dfcc71b7450bd7aae30168f79c88381263f461302dff5f605400000000000000001e950033ad9baf6c58162bbba40f8a0d8354b89b30b92b3be64629a8835a3f5700000000000000007a7ca798068c1ff70a0b399342501611334cafaeced087bc4fa020157772db7b0000000000000000784c7fb5305c5e9e8de632ba1a8616db93a9f500bc0783e007f67e057407f1bc0000000000000000e4af20417a0e2779ef199533bc33c3e9a799db5ba68accac248596f0f3bf869a00000000000000002519583687fbef33591fb7206f95f6ccb2dafa88f5220d937dbc530b1996b5090000000000000000bb6074ac0d7112d520ab8cd35c590b11c6b459097406bc84ce2bb6796eac12a40000000000000000e82ae799783c5ed7cdc8bae50c8126163d5dd96405814a597b601d88301b670a0000000000000000671a9efd6d9f9d39c4f2507e96ffb6ab79a6b8d018880ebaeb923f7ac2190a9f0000000000000000ad2adb22f9469609744e32b6fd38df6b71dc6c4fce017f5aef3a4803042b1eab0000000000000000fdbc2ddaeb3ccd9f98343d9e6134de44e8a4f10964ba1929825036ec4a794e1a0000000000000000e9ed92feed0234a638b8c700327571c245ceed153f9ae1aa7577a570ca65d4dd0000000000000000225a3db38630adfd58e7c29b74167d530811824ddc7b31ac9598336dbf902ca8000000000000000085e7870ac2e9c116aa5b64bd0e3e5f95bc31b8abb7d83e4d54bffe08c1ec2c5d00000000000000000586a5f692a01a97cc489cf2343de768fc999b1ce5500904a8c8a656c60710ee0000000000000000009cedb10ac050fa833ea1cc07f3bf8900d9916c24e3afa40f47b800b1115516000000000000000066c511f4b33f01e052cc51050d5741de6859ddff4329f7a77559de71c593435c00000000000000001f6bbd163ae3edb41a0cc40d466e35d1ef3366b64b768a9c64d3025714c337ca0000000000000000dd8fd99d3528e412593755534ef6333965619ad9c78e356513333f1458224b3000000000000000006c7930bbc5825ecd87e20724a03aa503ee2c7ac79dcfc6e719793caf8240d3d300000000000000006f4aa091896384f2d04c33c937174af53946dda0597174ac6c4e791ce6dc6bf20000000000000000b9b01392e35ab25653f6199294a78b7485002a60be44603c30a8e9f68a37e4af0000000000000000f49eef885e45366a0645299be1a9299b275b72757dae4aee7b319a4582600ac20000000000000000fe104ac27da766f50aa4ef116c00b7d96e2f439a7b51f3e8f08a610b9901ab980000000000000000c6edb1b339cc28a8230d0ca91f03674eb93e9396829325fdc1c56033f58cf6580000000000000000c7ebbe81a30fbad9ce087ff9e6792341473fa6f0ee9cf8704ec87c630909daaa0000000000000000ac6068d7e895630d4f5b5a73d9b98ad5a6b72b31a1b9a8854b4f3898d4adc5420000000000000000d38493a94e8fa6e0b98ca4842bb8e335614996800ccddeb29cf444624a42511c00000000000000000c85ed5b5f22acd8eb4abef105d049d0453c6509ace715e097a399237d58b6910000000000000000700f62b2d16425ef1f02f21ecc3393e7a759d4cfb9a5cb9ca52d7619cb80bbc70000000000000000f0b1759555f2cc14dad2eea4c8d542190f44c3666ceba22a0344d1a33b6d81d40000000000000000225dcf27915399aab44dbbc9a509682c1ad446b8eefc3cbdd22ce24c2eaa917800000000000000000ebd80d47384c65cc77f1f26e98ba8674eb8e74018bb08eadc08efb440ad19c80000000000000000ee6ab2fcf31a2e14194cb92b6da20f2949d0502ea688671c43ffb34ad343e08400000000000000005cfb508f9929718da7a6483d6ec886f31cfa19a8e1285de56019f1273299508f0000000000000000ff174846eff1145ffe987fd0b295a25b998449847aceeb14f90aa4a240434e1f0000000000000000cd88985a6eb9669a7e67388968a677590715b8c8eb8fc9ca34747ac8dd8fff02000000000000000085571fe4f735d95ede20e6b204d0c93e444b1d1f70bd0679e0b562eabbe6e2a300000000000000000198e75a02419da99adf8d8b3d6a78f934935354251cc4aed4f98df034213d7500000000000000001ee99a4d65dc4fd34e3424818c65c5844b258b76ff27f96ad155533e3b59e4ba0000000000000000c47c23a922e0edf2d44e1c1e36832b5ac5d98900f2567c5dfc35525fa5c3a6f700000000000000008bf694f65f070da4ec8ec792908813fd811e0203d1b6445b854bbf40b1142ef40000000000000000e9239b3ba64492f8bfc42a53bb310fb1f8b8ff00c47f3eb3cdeb395a797658c200000000000000009eb162aeaf6a358a2a39d339cb97b85deafc64a9f4531fda818950de254c41090000000000000000048881a383fa262d8d24ade8079c3c32b8bbd66a30b4c0ee225ecbf6459e699a00000000000000002b480fd21955dab194a0227eb7dd5078eb2a17079754d782dfe4c73b1eff955f0000000000000000276b80e302c4ac04baaeb5b6b3b82e63e6aa05382cf23a9ee4766d70aa9a2b7a0000000000000000decd54f8d3a3d4247cd7d6b078fe3c6c0dffdba1a3f7903cc9f597acb1e3df3f00000000000000006863d5a3f5c9373863f8fe951f27d4becad824c2e7b8033037c4360715c2fca20000000000000000ebb367273898d9e1c97e323d4b792a59739ebb4e66767673045e1a4288115151000000000000000012e6739b55441ea1cdf2523430a9404d40f6a632f2147a477a0e2dfb6dde3825000000000000000040a72643734680ad0aa4b6b8988ff4cd2b95aff93de927f1976129ac2abfcbd30000000000000000246d82d3b826dad2fb1546df980a19a6afe62a1797a0513f8c7541eeaa79f0c7000000000000000074d7874e9a5acdf0452ce815b98d61493258a7e2e3dc2377fbd9b73ff4cf4d750000000000000000ead24da34ec7878b4b87916008c9723cd3544d048c067863867034b72a0047ed00000000000000000030b09e83d8f9330f121aa76ff8a88a09a2cf1b161cb9b07c9560f01b15ab780000000000000000f85a93dfc9b5a5f3649cb9eec337c507a4c073f7b1b114a6bfcc687cf45740a50000000000000000b398426a304b57a5000fc46d5371eaffd10e44b2e54f75924cf4ee616a7d27160000000000000000fa2b5540043c74c65649c61cc2af795963152fec6c72362c69b7ff466230b6fb0000000000000000b1eda0998f95fed1946dd9813c2367c3d4c4a9f9a096a066fd335d8a28adde9e00000000000000003e7b2afab20c0bfa2024d5418f37188d64ee695d268c7fef6075db2cbb100d790000000000000000360981d0e5d51df11fb86d118430c09826c1ee116fa259c00268b0b50670aa800000000000000000d73e543b6a5a336581600608a55f66d86125e7ad9d5dbd54c509d07d71bd3cc10000000000000000624c236f45995d1a3f0382a953762b454d1342a0d0354af0f7a5478a4dda4e9d00000000000000002953b4e4035f3b4f03df09ab6fa623f03d6b50f656de6d17a550aea26b3d1b4e000000000000000037ca510de62da54011bd24e8b585d0f7df6ecfb019aa83ba9869386f11011b4b00000000000000006ebebcdd5a52d80af6653d627307ec8349f24011df8fee4cdc4b0df1e9356ebc00000000000000001ebf89b756fbaba4d2321e80bf50426f8753489770c5aa60bfd5e86eaa4ce35c000000000000000058366bfb16fc60ff446ee89401a41ac7eb1800d0b8a3a749abc1a84db0b5bdc9000000000000000021265890d0fe6042d92bf2b0644ebd54fb472e5ad6dfe09c648f0230155edce4000000000000000001c722426ea09132bfaecf8c5d50f8523cdd7ab5c67b596adef4a6fdb097744400000000000000000328cd4605f4aa4f0f656770c0e72ccb37b427672ad6362c5d4abb1de7ac237d0000000000000000bb049f04ffdc565c544b1441bc2a5db1c2eaebf09a2ba4920cf6fb0c32f0c522000000000000000049d944f737f8b75acc2308a6f7baa883f6bb0816ca5b65cc88f6068289693cb2000000000000000040ccf59cd2500b796ddeff843136bf23bf4d2b71c084f5f0cf6deae557c4617000000000000000001215cbe3320739c6f662669e4f45acd1b461c889e8edf13ccafb6e590835e3de000000000000000099a23bb3a066851c33a6c4ac3d9e34de2a1423f108fc16a4cce4d0897cf17b8100000000000000004f55bf32ef8ae87b7f895e1804cdecebd98aee992b30d6e53902d895d32ecdac0000000000000000b4e884814be9f3fbf6bc61a33f906484f2b80afe8cb32e722af43f43f40ee42c0000000000000000445ef107ae6bc284343262873e323146932248280d5b29306ef74de8bd6e828d0000000000000000910a34c4d9dd295431e2412d98c67335f25711f2771d7f7e527309492a58ff810000000000000000cacb1e09595fd7be7bb18c75188728e8eaf7d87211fb66e40cb4564f1d7305fc0000000000000000ef2072fc683ba12d2ec2f90d0b3422ccd4239f395f55c79489b0d7f90ef8d1bc0000000000000000d35b4267024e233741bea52bf38f2e241d716989e9d9688fab0f91d487c2aa5400000000000000001c584e1e7d3056380a55cd818923e2624c63cabea891132db922d7bcc88619870000000000000000e0efad633867d94a3cf677fdbd55ad71e7b87697d8800e4cf56f0c3d493924db0000000000000000802e24936e9f30d6542e6d3ffe71d7cea5be09fad48f8568f63b43cd6f60fe630000000000000000c6e0bbdd169985e81d819ee3c5a34a2b5895618ce378483ae630714ad57e88a60000000000000000b0ba3441b98780682df46d9576de635add4d11a8bd1fc7cc619aeb5ca18737630000000000000000010c7cc8b817f69b35a80c69a6680c6abfca28f2ee93c6f8c6f78db51ab24cf50000000000000000e1aeeccde9c78ff2f98c6f1475231865da17a35526fc664b02a932731768b92d0000000000000000509d61745d83c4f9a1a598be8625aac3c20ee4a44056edd2d78de22422cd3c1200000000000000002601be174bea1b1bf1c4e1f3705efba59f695c892e0288495f260991eae3b9510000000000000000da1b29e703753427f319381748129e7c5211a51568684e1c9b59666a08fc8b78000000000000000005b0521648eea74bdc2aa7479ee0141c3bb050498ab38fc683d367774988244b0000000000000000472998072a85f873eb998722e11aa7248789f75f05745623db7c609c66defdf10000000000000000109c4a273dca872b28eb314a3b7800488970a432417a1eb0b14c6ad6a31a8825000000000000000045a8c7d46f1aff70174d3b3bb0ff9fe0dafa042906b42ef26dccd3d9ba5beb6900000000000000004c1989adbe3840b82bd01c0729206180e78fe1c57c6b083ddd100cee15d6507b000000000000000097bc36c342c7f3291a036f3cda6407dd2a917641aa9c68474615a4b9115e103500000000000000007c73beb202ad703eca38a2a49a83900a40e8fb02394a7a6e6cac154dc41ad40500000000000000002c2922fe8373c8058e9cbdecff09fdcd6ddd0d8969ae6529f0d3c3e95a688cc10000000000000000c2bdc1f4daa23c0e9a89311917517340236b0eccd62536d49fed14145ee176240000000000000000d6be99c7c32d3f4fb3337233f304953dc2726edba7284d9888194c24f58ad30000000000000000001796229f952526b0ae47e24815546abaec354a5d941b6452ef13c5f5b4ebba9f00000000000000003dc7b1fededd60b516f9491ef2aefc527a978766641cdff50acb9eb8f8d9f6e300000000000000000a800b0d3539aacd06659ed91d78291d339a5183fc3262882073d55df542379f00000000000000005a08e6b0d00dce54d7a55399dcf5d50ddea3e1ff3ab3f016f3789ba3108a666600000000000000005924f813b3ed42adc7a8b42633db500e19877775ce8b4e099d0c49ee9fdf1fad00000000000000003b796839c1ac3b8f1e4c058af9dcda538e101428188346bd6224aee15d96ba6000000000000000003534bf3e5ea33b0eede684fd6a461511acfc1c017b0d1fa829f6835d4082e59a0000000000000000c5a39aab7b18ced2f522af4d10c443fb2dd73a43db4408f679731fe9cc4dd72b0000000000000000738693f05051cf636f01e3e82c88979b72bac83fab4e874ec253a99cc96b740d00000000000000006e09bcee876a60fb359a3488585b230f8ab056df2c3b850b95f1602735f4c0f50000000000000000217f24852933d72beb0c5eabc8759c6637c9995b753a6ad4e018513a93036bbe0000000000000000ca977817d229f95b468a35e9ab9c8e396dc7b76eaf9f8bae380e1af9435a85d400000000000000009d4a9b7422038a567397ee3ee713969ed1d57bd52b2b7e7c25c5f82fc12439480000000000000000e21dbaf481f6ce7457314ebda93ba044bb7e74f3a3719ef0b16b0b15ac77258e000000000000000013afaf9e901f111e1696a5f192a518ff4ccfbbd0efc1178b3e05ea8ea56f74e500000000000000009842683d5f93af51041b8141967a87253c31db6ee1ef0987e5e725698a83a9ed00000000000000008e6b15fa79dc752f5543ee7d381f5d912fbaa1148697fbb11c2a1e401f6d95790000000000000000e41f644456b1b25513d0cf121abc38ff655bcbf33b5153e656f05efbd5b106cf0000000000000000dd9f6a56cd61c2a906ee04e6397bb72817672b871c70dc17ba7483ae5a687ac60000000000000000214c6a75bfcae3f56a6f93121d2a7cf36d7c7c292a7f9be97346d26648c686f400000000000000006c4d4e7119a03ef8a7c06e0b0ea9e2895e990ace4c0dd369c64e70a94e8457b200000000000000007c4de053b4dcf574c3ade7ebe2bf5d5d171149c42e62145657682511c58aed1500000000000000001fb01fdd40a176533e19a5b858828563d228228858d11bf48bfb402e364b632200000000000000001fdda341735ff99846c8768b4618cee76703af97bd2ae0281ed1e2a6e0756e4c00000000000000009e3db6b22255e2e2f22c74c0821af9bebdedceb6d5022900adc93d0f49008b1300000000000000007271132aa8bfe0e3646a8bede2a8e52c50d27e84488fd49f8f170e4fe053bae00000000000000000fff586123c05b0fe1da74e69f9072f295dcbe076768120a9cf6c7ff77a6ad1dc0000000000000000265c4725fe9148fd3fa5c30ba9d7463a651ca95cc9e647d74a933e51b885dd2d0000000000000000195dcf087848fd07b59967bed6c1daead8dfa975c42e4717ba7a82af6f086aba00000000000000001efe412ccc21106986bd0b1fed6312ff96f32a3030241bc495c0a56a1864a00e00000000000000001fe2afb7422544bcdced59d29f70dcd24cc7a0718139ea91d3b49256795f26a9000000000000000046d435061683ec6d8983682aa492db54591a73e1f426a543b35c52d0a7a96bb800000000000000005134851fe94e37caaf818cbbaab520ca98abc4c306b56a56dd7736482b1d8c5c0000000000000000b1d18d9d5ce49110ac5a3a37ac2a64a87518931408902a8c76abff06183896480000000000000000f029162746c555e57a164b72894bab28e974b88e064ffb6797c9323f6ee7bfbc00000000000000000c8f4ce8d7719607bb3cf60ea34f7d0fce9ece06cba6ec24100e92f546842eaf0000000000000000e2b48549f5e2568a8f0a6d771dcdb3bd7ef8952a29d250750befc7c5b6ae5a070000000000000000a9255a6c00af337deefbb8ce94a5c98f315a817b2fe119a18896efda4bad8ee10000000000000000a29065cc93e1d98d65aad164a26d5d992c3521c4be6152d303714cb282481bc40000000000000000f5b67dea1173b7bcf13b04fcfdaeb8b29cfdc1ac6ec49cc691c7ce1867952e27000000000000000022bfc35d37630538d3946c40eab8c140839d6e5f155e94bf71b5801389098d450000000000000000ca45a9156be0fb3f4e5f8f52025fee1494986c6ebb2469e446b55f19daf16d1b00000000000000009894539a6511ca5356e84584c7bd894220c42d87a8593b2beddc64e16b96c03c000000000000000081e5b4f95f333dc03e4b54b78d2c5ff37ebede6b9ed2e8cdbe6d8a9a8433a6960000000000000000a8d62065d44bb4387dc4f901536c65be7475978b4d7d90f06c7530c43015aed400000000000000002e61715ba0d05ff7f68d596b124c7c877044d32c459d283ee4a922ec95feea130000000000000000c8bb0137d3b6f1146d2160bf756bc114883787d134e093edb0639d4aca26992d00000000000000002c64e06d9f76b876b7ebb0a0f4f0ccc912fc84a397a3d6ded91ffaddee82c8610000000000000000f76a5a266a79f17a13e7d8a3d7133579858d85a75299e7f629ba542ca50196510000000000000000b3e2527d86133cbb95881458ecf92feba792381344952f55af005d55f51705f50000000000000000ea6ac99aff4b4ac89b3d37eb85d2ae2001b912ab732e7aae6cc89c9dac171b4f0000000000000000d835dc798f6f8d6e41f76243d1de9f278feec5f7d1f314c9b131a596372eb8cf0000000000000000c64b06a1e51b2d171e89b958052887039608f684c6dd944fddebc2a094df171d0000000000000000069f795d3776f5efda0c183f5f8fff6b6318e4648775a564a8da5d38813d4e080000000000000000640522708697998b74f12a79b4494f40bdd2ed485fa4d4e9ab7facb067dfc790000000000000000004ea8bb6b6c00faef11c804aa12fce3822bb7da8c4b2c7c45d7a786de95365d900000000000000008a8353e486e5b7e0a9df049e4a8698ec9e00ac9be8334d06b9508db6cf9996df000000000000000030863c863dc3f9fade27b49ec655b97f3f0b8fbb12ec22c895b89a17a992092b00000000000000007a4fb06fbc4c3331613a5e1a3261e5e677fbfe9dd6af6abd02f09e2b2552ad5b00000000000000002b0d5c073f18eaf6986db8fa51633130446a6c61a25a81539f1f478b371c726200000000000000006168f2e82b6e502fadc4ca3bd2450d3511e72dbbeb1fe068bac9a7bbdca42285000000000000000006e92bc21329ae8a6c62642feac64d0ec823c7f0357ae192ededea53814e39bb00000000000000000ee903ed970f70bb28b30795bdab30ad357663d823d86b70984a27dc5207f88200000000000000000a5521edd1f0daeea5b6db94e4d893617498c1a8a5e0bba61bef889360c1533700000000000000004091414f0446039b4b4343264e12ad64b5de0a621cd5a3eb719a467ddcb38c690000000000000000d41254423306dd354767076f911adf037963f2d3ac4118e8f4c05ca827ba486b0000000000000000b23c3617052698c38e08b10edd5c63b6875d57fd2dd0f4fbf840c908cb99cca400000000000000005be5dccc59e0efbd9dba622d18e411b0382ea6dd49db6cfcc446afbc01a78856000000000000000046891b0545b164f18e3bc03c55ceb181bfc1457b4e654dc19ed0de98ac1b334a00000000000000001877f75e1f31934312cb0e223d8f0b2959d0e95404f878cc2a17bec856e1375000000000000000004e9bef2cc815bf0336845eb1ac2ef80858add4e06602319da2e9b1e182da410e00000000000000002af0cd5d9853d611a65dd43c983e3b5545e95bd97c1bac645a21c44f7511e16b0000000000000000c70404f43ad8bc288b7b1f91916e50f977516d09950fe15a6f8919dddff8986e00000000000000003b7f51b2e98673a00d5dcc6345b4bf45f5aac73107f5b6cc8d5bd282f91d52b500000000000000005f87b386f1231159db52c4b2f0ca0acd8d55dd59a2dc738f87993f69378421ac00000000000000009b3f53d9f2218d8c242fa8f53ec020bf7eb2951df4f388c10feb757514c169080000000000000000e41db6a6d142b339f0366ba232f4c4380068c712e3aa98e9ff77390a55bed0930000000000000000160302c9f08b2f09fd640cb4ad87e1d97ad43678847076fbe7f09401c5e5ffaa00000000000000009103711fac48e7787ccdee833d88c4fc7550f78e93e62ab619e02715a9c4d3f400000000000000007e5196db96ce3ad71392020bc1b2fa104381f5ddd0379862156dd91bde4558cf0000000000000000542d29516a86cc042fdb10cf2e7f659113820a3115351cf7ef8429667ebd9be500000000000000000f96dae8381087eafad638190169621191231d3cb08bf1bf769a4106b987474600000000000000007dae745740682775a48400d0b9588f0b00ed9f19d5d5e60453bba51b1e14f4ea00000000000000004d4482f9ad98acad1de9845a667517d0ce406f262ef452bf544b0ea4174652ff0000000000000000ff9f86b4aecd142809f322111759ed0ab40105f6b3ce6a46a5851eef9718cf5b00000000000000008e572ed7c0bbc06f1c79c5630ffaebf96b578c3ae711f3cc3d9a21056eefa2210000000000000000842f952d8dacf5a6a32977d356eb520efff0bf3cc3808ceeb9fac2b9790c33c500000000000000007889e9ecc31b56ec82452de12bafe84ef4a74d91b3598bd54230015582fb31070000000000000000bfa84c5445a5fdd6a4c49847925e19c149374b793f8f7b51b09d899eedb346d20000000000000000ffc8e07bba38e0f37dfc30ebd5f271564541eb384dfe51e57a819e07e74b070c00000000000000008dd5e5b4b5d8a0de9e13187327966b9804ca240addbd84534dcfe2305755d6d60000000000000000eac2ff48eb2eefac63b3747e8f899b7f4538fc3ad9e588b0c832ae85243e0cb50000000000000000966ae608dbf2006bd3391cd0ebfb5399497aa5d66f65e0cbd271bdc54df2a0eb0000000000000000a8d4beaaff4176321b7e013fc346ade2a8b56b012ca10251a0a0b246555634040000000000000000a68426049091bd22a748e7838ff8349bd75f117660083e443c4940b96f7870560000000000000000833497a764eeb2e73d994d780c45005be96d92fcd982a6c8e42f611e4e623d3c0000000000000000d2cf3b3c2f01a7a303e65e6a337196d7028858ef668a794e825d8c12d46206640000000000000000bcb28ce7f4fb028ab06ee3ac2dc3d8c223fdb74daa078e27851785beaaed45ac00000000000000006fdad6ce632d70465803214f024b683cae8eaaecf33a544d748e8a269b6515580000000000000000f753190660bdd2574231fa2051a2a78f3eec199b40be0ec019c1d299ef3605d700000000000000007abb93a7d81af26dd267e86dc7f8a318539e33a03cf10082713d0dcb7f36265d0000000000000000754a6abe5d90e5afdaeeb01ccee0f27d0f700bcb919611343e37826ba6725ddf000000000000000059eb8d883449b591a51a811b5604dffa179aa47d65f6fd17d743cf84a609d50c0000000000000000f4b51e27de77186c31918b2781ea450c6156cf60f1f1e637a7f87f558c62f26c0000000000000000f70017eef9ae9e05a15574b7ad6ea7f8b31a566505794f224f433adf06a68397000000000000000066a0328c81e9d9aaee4a0f1c30cc2e29a384250aae31a23f25da007d815acd05000000000000000067dc49976891cfcbff3ca95753be46666d52c5079b1eb968f5ac27e1d0852d07000000000000000014cee4320445fad2fac70edb932715b846a6a325a9942839542444009a5c3132000000000000000041136d8d973fa17734e1ccd8c40d77b08ffa57b1dc712aabe44e09ea285a0ef50000000000000000b9c608c75bc18fc98f161622c877a422bffeebcef04ed9905767977cd1c7f2cc0000000000000000f0bcb7d86dc0d48fc48fcd0d11da647ecd1c3549d8cbde20e5bac5da18b913630000000000000000fe4202d6777191febf25953a1ea8c1f320dc7e504ab47a4bb294d6f0ed7db31a0000000000000000c1d8a912dc87d709492ebc43cdb455a2d8a97946c8621c634008d102e734d32a0000000000000000d604ca0d3eab232f4e2940163de8a251f0e18c8be906f93a3bc2fd1aba7b7b53000000000000000041cf4ae56ed981cd9bdc9683b9431e77c6158d4761d174e8a59b59063795b36b00000000000000003904b025d0a4e3baa1e3d465e546d3522663e30b8771f1bfe6798a0c8993a3e900000000000000003f60555aaaf52164e662c130fb4cba80c76f1dd6ae874c7ab042384affe5f16800000000000000007a659e4406a0198dbfae13a2b6475df88e617c34e31571defb6088e3aaf493cb0000000000000000834563ff1274aa1dab643b8cf6743dd75d24e9405c658e28eaf6b4bf860c9c3100000000000000007c9ec4048ca793473058dbd8d2779b295b9be7842dc0afa839e60797f815d22f00000000000000006fc0ab9322188733cf05fa7fbdf2e84697aac090695d94a38e4e91a334e6902e0000000000000000b570fc8f3c92725895a743534485d1fb8771735631985f7464f014d2159848e9000000000000000072d00b5cadb331bdd80e9d683891a6b272b4eb9e26b3e2502fa9576005f3080f000000000000000007780bc3ae3733959e06446eeaec1123de1b9ad71d5a14eddfc4652188e4eb1100000000000000006db868361129285aeb542a3f8ed683895e75965dda29b2d0dcff2e7e63f3c9b10000000000000000df532b345eccd565ccd6e7730422d7993a36ca9a3f7036378e96a57c38383d7a0000000000000000c0db25d52879aef53e310593e346fc4f0ccd9e60847fdf55b995e955455b345f000000000000000019ddf72883b507916d9e63f679dbd4b171058c992301a676165609744bb5206d0000000000000000cecc8126135333b32a4fa1eb35c96370b95c55f216abe81bc1b742bbb0292a6c0000000000000000f41887d1f352a2bc614c0db152932619fe892cb9c19eecf12c5195037a365042000000000000000077bb1e77db06fdade7f031b60c4135b77b73db113a7b1557fbdd5dc8812fa9e00000000000000000984935e54049d8e034222d45ca161aac3d6a2c8f555b596afe99c1f386fd345300000000000000008ead572e77bf1522e570e306096945faa70c27cf7fc92e4e412872865ebcd20d000000000000000042af08a53490158553d5138cfe380acfb49d6e1b4d3b223f1c9962bf9e0767260000000000000000a7d596226ff12cc379a1c733f57f8adc54e6f58f572b1da849122381651b02fa0000000000000000cb710c2bc64511ee93f9c6c09c530c9eead588ffe3a3bf81dfcaead04abb9627000000000000000028c5d45761cf38ed4fb2d4820bc57b9e591a04e9623b1f46da49f3ffa9dbf506000000000000000038f542347e7b8ee5d194890efc220bdf3f49d967c8a3e69d7c113aa0a95d290d0000000000000000b024dbaa5aa401f43071f273fdc98d13381628fe6b5cefd482ca8d64275cf29b00000000000000008f945cfd43b0996b000314bc186816cae2e6698c11ca3749b2d7a00718fb8c3f000000000000000017a3d2b53dfb08a1ab386e6c6f846c78d1aee8c4521e558c01dafecfcda757f0000000000000000013fefdf542341f1f92be3d67058cc4e7eab1303c48ee7fb8873c9dd56c30eedc0000000000000000dd8023b710a813409dd35d7ffab8c71a7625eb15b5bdb981862a5d270b1622a30000000000000000a9ed7261d36f1df934db5cfc81e1ea6e7c7117203dc4b88ccf336e6385153ec100000000000000005a61e238f07e3a8114e39670c1e5ff430913d5793028258cf8a49282efee441100000000000000000000000000000000000000000000000001000000000000000000000000000000",


Proof seems kiiinda large here? Would expect smaller with just a simple asset and a compressed inclusion proof with a single leaf.

Ah, good catch! Looks like this was the maximum proof there can be, as it's 0% compressible (we used *mssmt.NewProof(mssmt.EmptyTree[:mssmt.MaxTreeLevels]), to create it).

address/address_test.go

Roasbeef · 2023-06-16T00:07:10Z

address/address_test.go

+
+	testVectors := &TestVectors{}
+	test.ParseTestVectors(
+		t, "testdata/address_tlv_encoding.json", &testVectors,


I think I'm missing how/where the test vectors were generated in the first place. Eg: would expect an optional flag on an existing (?) test to output the vectors in a specified location.

Good point! I went ahead and added a build flag (can be activated using make unit gen-test-vectors=true) to generate the files and also invested some time to make the content deterministic. So it should be as easy as adding new test cases in Golang to generate extended test vectors!

address/address_test.go

Roasbeef · 2023-06-16T00:08:39Z

commitment/mock.go

+}
+
+type TestProof struct {
+	AssetProof        *TestAssetProof        `json:"asset_proof"`


Maybe prefix with Smt? IIUC this is the smt inclusion proof for na asset while the bottom is the entire state transition proof?

I tried to keep the names of the original structs:

taproot-assets/commitment/proof.go

Line 54 in 0a58cdb

AssetProof *AssetProof

Though I agree it's not super clear now. IMO it was way clearer what the proofs meant when they were called AssetProof and TaroProof, as that directly corresponded to the two levels of trees.

Happy to rename either of them, but I would suggest to do it in both the test struct and the "production code" struct?

dstadulis · 2023-06-20T18:56:32Z

There are some randomly generated vectors -- would using a seed from a rand source could help.

TODO / Missing:

[] Considering using a flag.
[] Possibly adding coverage VM
There aren't edge cases address
[] Need more vectors
Default structure is in

We can later add more vectors / edge cases.

guggero · 2023-06-21T16:37:22Z

Taking this out of draft status as we now have deterministically generated test vectors for assets, addresses, proofs and vPSBTs. This does not fully address #302 yet, but is a big step towards that.

guggero · 2023-06-22T13:36:30Z

I added more test vectors in the latest push, now also including VM positive/negative tests and MS-SMT tests, so this PR should now cover everything described in #302.
We should still add more edge case vectors later, but this PR is already large enough, so that should happen in a follow-up PR.

Roasbeef

🎉

Great work giving us a solid base to build on and move towards initial BIP number acquisition.

Reviewed 27 of 32 files at r1, 14 of 14 files at r2, all commit messages.
Reviewable status: all files reviewed, 8 unresolved discussions (waiting on @guggero, @halseth, and @jharveyb)

mssmt/encoding_test.go line 148 at r2 (raw file):

	// Write test vectors to file. This is a no-op if the "gen_test_vectors"
	// build tag is not set.
	test.WriteTestVectors(t, proofsTestVectorName, testVectors)

👍

lightninglabs-deploy · 2023-07-20T00:45:00Z

@halseth: review reminder
@jharveyb: review reminder

jharveyb

LGTM! This is an awesome PR, very excited to see these vectors used by others.

AFAICT only the VM negative test vectors are autogenerated right now, so I'm a bit confused as to why that can't be done for the other packages as well. Non-blocking.

jharveyb · 2023-07-25T12:33:52Z

asset/asset_test.go

+
+	allTestVectorFiles = []string{
+		generatedTestVectorName,
+		"asset_tlv_encoding_error_cases.json",


Are the error_cases tests populated manually, or am I missing where the logic is for generating those?

Yeah, so basically if a test vector file name doesn't have a corresponding package-level variable for it (and it only appears in the allTestVectorFiles slice) then it means it's currently not autogenerated but contains some manual negative tests.

address/address_test.go

vm/vm_test.go

The tweaked script key is intended to be the internal key, which, when using BIP86 (or an actual tweak) is computed to the Taproot output key. So using the same internal key as the outer Taproot key is incorrect, so we remove the anyway unused value.

Some languages (looking at you, JavaScript) have trouble with large (64bit) integers being parsed directly into native types and need to use BigInteger or similar helper types. To avoid MS-SMT leaf values being parsed into native 64bit integer types, we serialize them as strings.

dstadulis mentioned this pull request May 30, 2023

bip: test vector creation #302

Closed

7 tasks

ffranr assigned guggero May 30, 2023

guggero force-pushed the bip-test-vectors branch 3 times, most recently from 1293d61 to 4108f34 Compare June 6, 2023 07:09

guggero force-pushed the bip-test-vectors branch from 4108f34 to 8ab153c Compare June 12, 2023 15:26

Roasbeef self-requested a review June 13, 2023 19:04

Roasbeef reviewed Jun 16, 2023

View reviewed changes

guggero force-pushed the bip-test-vectors branch 3 times, most recently from d175145 to a3ed3c5 Compare June 21, 2023 16:35

guggero changed the title ~~multi: add test vectors for asset TLV encoding~~ multi: add test vectors for asset/address/proof/PSBT TLV encoding Jun 21, 2023

guggero marked this pull request as ready for review June 21, 2023 16:36

guggero requested a review from Roasbeef June 21, 2023 16:37

guggero force-pushed the bip-test-vectors branch from a3ed3c5 to 64cd706 Compare June 22, 2023 13:35

guggero changed the title ~~multi: add test vectors for asset/address/proof/PSBT TLV encoding~~ multi: add test vectors for asset/address/proof/PSBT TLV encoding and VM/MS-SMT validation Jun 22, 2023

guggero force-pushed the bip-test-vectors branch from 64cd706 to 67fb444 Compare June 22, 2023 13:41

dstadulis requested review from jharveyb and halseth June 26, 2023 15:48

dstadulis added the v0.3 label Jul 10, 2023

Roasbeef approved these changes Jul 13, 2023

View reviewed changes

dstadulis linked an issue Jul 24, 2023 that may be closed by this pull request

Add proof file verification BIP test vectors #329

Closed

jharveyb approved these changes Jul 25, 2023

View reviewed changes

guggero added 2 commits July 25, 2023 16:18

make+internal: add build tag for generating test vectors

cf67b08

guggero added 3 commits July 25, 2023 16:18

multi: make random values deterministic and tread safe

e9d41f3

multi: add test vectors for asset TLV encoding

99abb40

address: fix address test loop var issue

a72591c

guggero force-pushed the bip-test-vectors branch from 67fb444 to 0307a71 Compare July 25, 2023 14:23

guggero added 8 commits July 25, 2023 16:46

address+commitment: add test vectors for address TLV encoding

5f1c97c

address: fix race issue introduced with new test

6855fc7

proof+commitment: add test vectors for proof TLV encoding

1919e35

tappsbt: add test vectors for PSBT encoding

2734af4

vm+commitment: add test vectors for VM validation

e149c7d

mssmt: add test vectors for MS-SMT trees

d581396

make+GitHub: assert test vectors are updated

c05cefa

guggero force-pushed the bip-test-vectors branch from 0307a71 to c05cefa Compare July 25, 2023 15:15

guggero added this pull request to the merge queue Jul 25, 2023

Merged via the queue into main with commit 6e7bd85 Jul 25, 2023
13 checks passed

guggero deleted the bip-test-vectors branch July 25, 2023 16:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

multi: add test vectors for asset/address/proof/PSBT TLV encoding and VM/MS-SMT validation #326

multi: add test vectors for asset/address/proof/PSBT TLV encoding and VM/MS-SMT validation #326

guggero commented May 26, 2023 •

edited by Roasbeef

Loading

dstadulis commented Jun 13, 2023

Roasbeef left a comment

Roasbeef Jun 15, 2023

guggero Jun 21, 2023

Roasbeef Jun 16, 2023

guggero Jun 21, 2023

Roasbeef Jun 16, 2023

guggero Jun 21, 2023

Roasbeef Jun 16, 2023

guggero Jun 21, 2023

dstadulis commented Jun 20, 2023

guggero commented Jun 21, 2023

guggero commented Jun 22, 2023

Roasbeef left a comment

lightninglabs-deploy commented Jul 20, 2023

jharveyb left a comment

jharveyb Jul 25, 2023

guggero Jul 25, 2023

multi: add test vectors for asset/address/proof/PSBT TLV encoding and VM/MS-SMT validation #326

multi: add test vectors for asset/address/proof/PSBT TLV encoding and VM/MS-SMT validation #326

Conversation

guggero commented May 26, 2023 • edited by Roasbeef Loading

dstadulis commented Jun 13, 2023

Roasbeef left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dstadulis commented Jun 20, 2023

TODO / Missing:

guggero commented Jun 21, 2023

guggero commented Jun 22, 2023

Roasbeef left a comment

Choose a reason for hiding this comment

lightninglabs-deploy commented Jul 20, 2023

jharveyb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

guggero commented May 26, 2023 •

edited by Roasbeef

Loading