[BUG] - local tx submission is not backward compatible #137
Comments
|
cc @dnadales |
|
cc @amesgen |
|
Generally, there are no guarantees that the tx binary format evolves in a backwards-compatible manner from era to era, as was the case from e.g. Byron to Shelley. So there is no general guarantee that submitting the same bytes will work with era index On the other hand, Ledger does not randomly change the format, so a serialized Alonzo tx might well be also a valid Babbage tx. Can you maybe post the CBOR of the tx you are trying to submit as a Babbage tx? |
|
@amesgen I find your comment contradicting your own comment. Though I explain the issue in the description and is what you describe in your first paragraph, but it should work as per your second paragraph. Below the tx cbor I am submitting with era It should generally work as all previous era tx is a valid Babbage era tx as per the spec. Note that, it does not just return an error, but disconnects the connection, which generally happens when you are communicating with a bad format. But it works well if I use era as |
|
I don't think there is a contradiction, backwards-compatibility is not guaranteed in general (Byron-to-Shelley is the counterexample), but at the same time, it can be satisfied for e.g. Alonzo-to-Babbage. I just tested that deserializing your transaction (after wrapping it appropriately) can be deserialized using the code in this repo both with era index works fine; so the error must be somewhere else. Pretty-printed deserialized transactionHardForkGenTx
{ getHardForkGenTx = S
( S
( S
( S
( S
( Z AlonzoTx
{ body = TxBodyConstr BabbageTxBodyRaw
{ btbrSpendInputs = fromList
[ TxIn
( TxId
{ unTxId = SafeHash "3a79a6a834e7779c67b0b3cbd3b7271883bbbeac15b1a89d78f057edc25e000b" }
)
( TxIx 0 )
, TxIn
( TxId
{ unTxId = SafeHash "3a79a6a834e7779c67b0b3cbd3b7271883bbbeac15b1a89d78f057edc25e000b" }
)
( TxIx 1 )
]
, btbrCollateralInputs = fromList []
, btbrReferenceInputs = fromList []
, btbrOutputs = StrictSeq
{ fromStrict = fromList
[ Sized
{ sizedValue =
( Addr Testnet
( KeyHashObj
( KeyHash "7d5a2560d23c3443b98d84c57b0c491311da4b3098de1945c7bcfc4c" )
)
( StakeRefBase
( KeyHashObj
( KeyHash "63ea8c5404f9ed9ae80d95b5544857b2011e3f26b63ddc3be1abd42d" )
)
)
, MaryValue 2000000
( MultiAsset
( fromList [] )
)
, NoDatum
, SNothing
)
, sizedSize = 65
}
, Sized
{ sizedValue =
( Addr Testnet
( KeyHashObj
( KeyHash "09ecea977429fa7a4993bc045ea618f3697e6b8eac9d5ea68bba7e4b" )
)
( StakeRefBase
( KeyHashObj
( KeyHash "63ea8c5404f9ed9ae80d95b5544857b2011e3f26b63ddc3be1abd42d" )
)
)
, MaryValue 1443799068
( MultiAsset
( fromList
[
( PolicyID
{ policyID = ScriptHash "47be64fcc8a7fe5321b976282ce4e43e4d29015f6613cfabcea28eab" }
, fromList
[
( "54657374"
, 999801002
)
,
( "576f52456d706972654c69636830303739"
, 876868768
)
]
)
,
( PolicyID
{ policyID = ScriptHash "4cd2ea369880853541c5f446725f3e4ecaf141635f0c56c43104923b" }
, fromList
[
( "74464c4143"
, 999999998379995160
)
]
)
,
( PolicyID
{ policyID = ScriptHash "85ef026c7da6a91f7acc1e662c50301bcce79eb401a3217690aa7044" }
, fromList
[
( "74464c4143"
, 9373000000
)
]
)
,
( PolicyID
{ policyID = ScriptHash "92bd3be92d6a6eadd7c01ce9ff485809f3f2eb36845cd7a25c9177bf" }
, fromList
[
( "546f20746865206d6f6f6e"
, 1
)
]
)
]
)
)
, NoDatum
, SNothing
)
, sizedSize = 267
}
]
}
, btbrCollateralReturn = SNothing
, btbrTotalCollateral = SNothing
, btbrCerts = StrictSeq
{ fromStrict = fromList [] }
, btbrWithdrawals = Withdrawals
{ unWithdrawals = fromList [] }
, btbrTxFee = Coin 178613
, btbrValidityInterval = ValidityInterval
{ invalidBefore = SNothing
, invalidHereafter = SJust
( SlotNo 94473975 )
}
, btbrUpdate = SNothing
, btbrReqSignerHashes = fromList []
, btbrMint = MultiAsset
( fromList [] )
, btbrScriptIntegrityHash = SNothing
, btbrAuxDataHash = SNothing
, btbrTxNetworkId = SNothing
}
( blake2b_256: SafeHash "8c3414268c4a22996ed9fa410d74567673e587d3c356fd9ae9f2f7a20584a89c" )
, wits = AlonzoTxWitsRaw
{ atwrAddrTxWits = fromList
[ WitVKeyInternal
{ wvkKey = VKey
( VerKeyEd25519DSIGN "2726733baa5c15d8d856c8d94e7d83bcfc7f5661ec7f952f052f311a2443feb2" )
, wvkSig = SignedDSIGN
( SigEd25519DSIGN "5f9d3d8a703baf700a3015994a3e8702fd7fe2e25d640487944b32ea999f36b314be9674be09b8b8f2c678976ecf994c83086180e854120d81243476c2b89e05" )
, wvkKeyHash = KeyHash "ea00161ec2547143255d715d7e3c5cb5d906e6d52bf3b5b866877b65"
, wvkBytes = "\x82X '&s;ª\\x15ØØVÈÙN}\x83¼ü\x7fVaì\x7f\x95/\x5/1\x1a$Cþ²X@_\x9d=\x8ap;¯p
0\x15\x99J>\x87\x2ý\x7fââ]d\x4\x87\x94K2ê\x99\x9f6³\x14¾\x96t¾\x9¸¸òÆx\x97nÏ\x99L\x83\x8a\x80èT\x12\xd\x81$4v¸\x9e\x5"
}
]
, atwrBootAddrTxWits = fromList []
, atwrScriptTxWits = fromList []
, atwrDatsTxWits = TxDatsConstr TxDatsRaw
( fromList [] )
( blake2b_256: SafeHash "45b0cfc220ceec5b7c1c62c4d4193d38e4eba48e8815729ce75f9c0ab0e4c1c0" )
, atwrRdmrsTxWits = RedeemersConstr RedeemersRaw
( fromList [] )
( blake2b_256: SafeHash "45b0cfc220ceec5b7c1c62c4d4193d38e4eba48e8815729ce75f9c0ab0e4c1c0" )
}
( blake2b_256: SafeHash "0c463af8d9b51d4c504379c56c41743f47a087e842af9963daf82dc24b9db912" )
, isValid = IsValid True
, auxiliaryData = SNothing
}
)
)
)
)
)
}HardForkGenTx
{ getHardForkGenTx = S
( S
( S
( S
( Z AlonzoTx
{ body = TxBodyConstr AlonzoTxBodyRaw
{ atbrInputs = fromList
[ TxIn
( TxId
{ unTxId = SafeHash "3a79a6a834e7779c67b0b3cbd3b7271883bbbeac15b1a89d78f057edc25e000b" }
)
( TxIx 0 )
, TxIn
( TxId
{ unTxId = SafeHash "3a79a6a834e7779c67b0b3cbd3b7271883bbbeac15b1a89d78f057edc25e000b" }
)
( TxIx 1 )
]
, atbrCollateral = fromList []
, atbrOutputs = StrictSeq
{ fromStrict = fromList
[
( Addr Testnet
( KeyHashObj
( KeyHash "7d5a2560d23c3443b98d84c57b0c491311da4b3098de1945c7bcfc4c" )
)
( StakeRefBase
( KeyHashObj
( KeyHash "63ea8c5404f9ed9ae80d95b5544857b2011e3f26b63ddc3be1abd42d" )
)
)
, MaryValue 2000000
( MultiAsset
( fromList [] )
)
, SNothing
)
,
( Addr Testnet
( KeyHashObj
( KeyHash "09ecea977429fa7a4993bc045ea618f3697e6b8eac9d5ea68bba7e4b" )
)
( StakeRefBase
( KeyHashObj
( KeyHash "63ea8c5404f9ed9ae80d95b5544857b2011e3f26b63ddc3be1abd42d" )
)
)
, MaryValue 1443799068
( MultiAsset
( fromList
[
( PolicyID
{ policyID = ScriptHash "47be64fcc8a7fe5321b976282ce4e43e4d29015f6613cfabcea28eab" }
, fromList
[
( "54657374"
, 999801002
)
,
( "576f52456d706972654c69636830303739"
, 876868768
)
]
)
,
( PolicyID
{ policyID = ScriptHash "4cd2ea369880853541c5f446725f3e4ecaf141635f0c56c43104923b" }
, fromList
[
( "74464c4143"
, 999999998379995160
)
]
)
,
( PolicyID
{ policyID = ScriptHash "85ef026c7da6a91f7acc1e662c50301bcce79eb401a3217690aa7044" }
, fromList
[
( "74464c4143"
, 9373000000
)
]
)
,
( PolicyID
{ policyID = ScriptHash "92bd3be92d6a6eadd7c01ce9ff485809f3f2eb36845cd7a25c9177bf" }
, fromList
[
( "546f20746865206d6f6f6e"
, 1
)
]
)
]
)
)
, SNothing
)
]
}
, atbrCerts = StrictSeq
{ fromStrict = fromList [] }
, atbrWithdrawals = Withdrawals
{ unWithdrawals = fromList [] }
, atbrTxFee = Coin 178613
, atbrValidityInterval = ValidityInterval
{ invalidBefore = SNothing
, invalidHereafter = SJust
( SlotNo 94473975 )
}
, atbrUpdate = SNothing
, atbrReqSignerHashes = fromList []
, atbrMint = MultiAsset
( fromList [] )
, atbrScriptIntegrityHash = SNothing
, atbrAuxDataHash = SNothing
, atbrTxNetworkId = SNothing
}
( blake2b_256: SafeHash "8c3414268c4a22996ed9fa410d74567673e587d3c356fd9ae9f2f7a20584a89c" )
, wits = AlonzoTxWitsRaw
{ atwrAddrTxWits = fromList
[ WitVKeyInternal
{ wvkKey = VKey
( VerKeyEd25519DSIGN "2726733baa5c15d8d856c8d94e7d83bcfc7f5661ec7f952f052f311a2443feb2" )
, wvkSig = SignedDSIGN
( SigEd25519DSIGN "5f9d3d8a703baf700a3015994a3e8702fd7fe2e25d640487944b32ea999f36b314be9674be09b8b8f2c678976ecf994c83086180e854120d81243476c2b89e05" )
, wvkKeyHash = KeyHash "ea00161ec2547143255d715d7e3c5cb5d906e6d52bf3b5b866877b65"
, wvkBytes = "\x82X '&s;ª\\x15ØØVÈÙN}\x83¼ü\x7fVaì\x7f\x95/\x5/1\x1a$Cþ²X@_\x9d=\x8ap;¯p
0\x15\x99J>\x87\x2ý\x7fââ]d\x4\x87\x94K2ê\x99\x9f6³\x14¾\x96t¾\x9¸¸òÆx\x97nÏ\x99L\x83\x8a\x80èT\x12\xd\x81$4v¸\x9e\x5"
}
]
, atwrBootAddrTxWits = fromList []
, atwrScriptTxWits = fromList []
, atwrDatsTxWits = TxDatsConstr TxDatsRaw
( fromList [] )
( blake2b_256: SafeHash "45b0cfc220ceec5b7c1c62c4d4193d38e4eba48e8815729ce75f9c0ab0e4c1c0" )
, atwrRdmrsTxWits = RedeemersConstr RedeemersRaw
( fromList [] )
( blake2b_256: SafeHash "45b0cfc220ceec5b7c1c62c4d4193d38e4eba48e8815729ce75f9c0ab0e4c1c0" )
}
( blake2b_256: SafeHash "0c463af8d9b51d4c504379c56c41743f47a087e842af9963daf82dc24b9db912" )
, isValid = IsValid True
, auxiliaryData = SNothing
}
)
)
)
)
} |
|
I can understand that the deserializing works, as it should. Does this mean the problem lies with Ouroboros Network? Because submitting this tx works only with era 4, and not 5 via the mini protocol. |
|
c @coot |
|
There were no protocol level changes since a very long time, so I'd be quite surprised if the problem lives in |
|
if we use cardano-cli, it does not require to define the era hence it calculates the correct era and submits the tx thus works fine. I am using the local tx submit mini protocol directly as detailed in the description, hence I now have to define the tx era myself and that is how this issue is found. If the code is not changed for a long time, possible the issue is present from a long time because so far only IOG tools submit the tx, and some other community tools which submit the tx figures out the era manually and define it before submitting (as described if the correct era is defined it works) If we submit an alonzo era tx in babbage era, with era number as babbage (5), the node just disconnects the connection without an error msg ofc. Note as described in the previous messages, if we submit the same tx with era as 4 while in Babbage era it works fine. |
|
|
|
Oh, I see there's no message back. Is there anything in the |
|
For my own tools, this is the work-around I have to do to ensure I'm submitting the right era for a given transaction. It feels fragile and prone to bugs even though it currently works. If I submit a wrong era, like @ashisherc said, you just get dropped by the node and disconnected with no error message. |
|
We could not reproduce this in a local (Consensus) test. In future releases, we will enable certain tracers by default, which include the tracers that log exceptions that cause local client disconnection. In the meantime could you set the severity of the local tx submission protocol to |
…er (#225) This PR introduces a test to detect errors in the Ledger code, [like this one](#137), early on. Other changes include: - Addition of a new test library to mock a mempool. - Addition of a module to elaborate a `ProtocolInfo` for testing purposes. This protocol info is used when opening a Cardano mempool.
|
update: investigating - please keep this open |
|
@dnadales no logs even after the suggested config change. the connection is just dropped. |
|
@ashisherc @AndrewWestberg Do you still have repros for this Issue, 1.5 years later 😬? The only element that's coming to my mind but is not discussed above is what handshake version was being used for the NTC connection. However (ruling out bugs that would be surprising, eg related to the enable-experimental-eras flags), you wouldn't have been able to submit any Babbage txs if your negotiated version was stale. |
I agree that not having the log message for this is very painful. My guess is that this is the exception that we're all surmising is being thrown https://github.com/IntersectMBO/ouroboros-network/blob/28b731cde005a1de0b4be0ea0bd16852e827c1bc/ouroboros-network-framework/src/Ouroboros/Network/Driver/Simple.hs#L152 --- I don't know why it was not appearing in the node log. |
|
FWIW, I just synced today's node on preprod, and then used a hacked up https://preprod.cexplorer.io/tx/3a79a6a834e7779c67b0b3cbd3b7271883bbbeac15b1a89d78f057edc25e000b So: not only do those bytes decode in Babbage using the methods we think are involved, but the resulting Babbage tx is also valid in the first possible ledger state it could have been valid in just considering its Caveats:
|
|
I don't have any reproduction steps, but I do have a newer work-around method to pick the proper era just from looking at the tx cbor itself. |
There is even no guarantee from one epoch to another. All it takes is a protocol parameter update and a previously valid transaction can become invalid.
More often than not transaction built for previous era will still work for the next era. So I really don't understand why consensus submission protocol requires this era tagging, makes no sense to me. |
THIS exactly. We are using a very similar snippet as shared by @AndrewWestberg to figure a correct era tag to be able to submit tx and it has made tooling very difficult. |
Backwards-compatibility for the tx binary format and for general tx validity (with the latter depending on the protocol parameters) are different things, right?
(I think) the historical reason for this is the fact that transactions between Byron and Shelley were completely different, so it made sense to indicate this as a tag (in combination with the tag that basically every entity (blocks/headers/etc.) has such a tag today). But indeed, it might be nice to change this, as has recently been discussed in #1401, albeit for somewhat different reasons. But the better user experience is definitely a factor! |
Successful deserialization is a prerequisite of a valid transaction. But, yes, they are different things. |
External
Summary
I am using the local transaction submission mini protocol, while submitting a transaction that is built using Alonzo spec, and being submitted with era number 5 (Babbage), the node disconnects the connection and does not return an error message as well.
Steps to reproduce
Build an Alonzo era tx,
Prepare payload with era as
5Submit using the local transaction submission mini protocol
Expected behaviour
Since Babbage era tx spec is Alonzo compatible, a tx built using Alonzo only spec is a valid Babbage era tx. The protocol and node should accept the tx.
The text was updated successfully, but these errors were encountered: