Support async payments in BOLT 12 #1149
Conversation
valentinewallace
force-pushed
the
2024-02-async-payments
branch
from
10f0c5b to
dc6599f
Compare
|
Changed up the feature bits, required single-chain offers, and disallowed setting |
|
Added a fixup requiring the invreq to be included in the payment onion per spec meeting discussion a week ago! |
valentinewallace
force-pushed
the
2024-02-async-payments
branch
from
5a36154 to
5b5fd7f
Compare
Per <lightning/bolts#1149>, when paying a static invoice we need to include our original invoice request in the HTLC onion since the recipient wouldn't have received it previously. We use an experimental TLV type for this new onion payload field, since the spec is still not merged in the BOLTs.
Add a new invoice request parameter to onion_utils::build_onion_payloads. As of this commit it will always be passed in as None, to be updated in future commits. Per <lightning/bolts#1149>, when paying a static invoice we need to include our original invoice request in the HTLC onion since the recipient wouldn't have received it previously.
Add a new invoice request parameter to onion_utils::create_payment_onion. As of this commit it will always be passed in as None, to be updated in future commits. Per <lightning/bolts#1149>, when paying a static invoice we need to include our original invoice request in the HTLC onion since the recipient wouldn't have received it previously.
Add a new invoice request parameter to outbound_payments and channelmanager send-to-route internal utils. As of this commit the invreq will always be passed in as None, to be updated in future commits. Per <lightning/bolts#1149>, when paying a static invoice we need to include our original invoice request in the HTLC onion since the recipient wouldn't have received it previously.
When transitioning outbound payments from AwaitingInvoice to StaticInvoiceReceived, include the invreq in the new state's outbound payment storage for future inclusion in an async payment onion. Per <lightning/bolts#1149>, when paying a static invoice we need to include our original invoice request in the HTLC onion since the recipient wouldn't have received it previously.
|
Rebased after merge of #798 🎉 |
Per <lightning/bolts#1149>, when paying a static invoice we need to include our original invoice request in the HTLC onion since the recipient wouldn't have received it previously. We use an experimental TLV type for this new onion payload field, since the spec is still not merged in the BOLTs.
Add a new invoice request parameter to onion_utils::build_onion_payloads. As of this commit it will always be passed in as None, to be updated in future commits. Per <lightning/bolts#1149>, when paying a static invoice we need to include our original invoice request in the HTLC onion since the recipient wouldn't have received it previously.
Add a new invoice request parameter to onion_utils::create_payment_onion. As of this commit it will always be passed in as None, to be updated in future commits. Per <lightning/bolts#1149>, when paying a static invoice we need to include our original invoice request in the HTLC onion since the recipient wouldn't have received it previously.
Add a new invoice request parameter to outbound_payments and channelmanager send-to-route internal utils. As of this commit the invreq will always be passed in as None, to be updated in future commits. Per <lightning/bolts#1149>, when paying a static invoice we need to include our original invoice request in the HTLC onion since the recipient wouldn't have received it previously.
When transitioning outbound payments from AwaitingInvoice to StaticInvoiceReceived, include the invreq in the new state's outbound payment storage for future inclusion in an async payment onion. Per <lightning/bolts#1149>, when paying a static invoice we need to include our original invoice request in the HTLC onion since the recipient wouldn't have received it previously.
Past commits have set us up to include invoice requests in outbound async payment onions. Here we actually pull the invoice request from where it's stored in outbound_payments and pass it into the correct utility for inclusion in the onion on initial send. Per <lightning/bolts#1149>, when paying a static invoice we need to include our original invoice request in the HTLC onion since the recipient wouldn't have received it previously.
While in the last commit we began including invoice requests in async payment onions on initial send, further work is needed to include them on retry. Here we begin storing invreqs in our retry data, and pass them along for inclusion in the onion on payment retry. Per <lightning/bolts#1149>, when paying a static invoice we need to include our original invoice request in the HTLC onion since the recipient wouldn't have received it previously.
| message when they come online, unblock the HTLC, and expect to receive it | ||
| quickly thereafter. | ||
|
|
||
| Note that if the sender expects to be online when the recipient comes online, |
There was a problem hiding this comment.
Is this an expectation that can be had? I am wondering what this means to the user of a mobile app.
There was a problem hiding this comment.
Discussed offline (as with many of these comments), but this could apply to custodial senders who are always-online.
| @@ -58,6 +58,19 @@ The merchant-pays-user flow (e.g. ATM or refund): | |||
| 3. The merchant confirms the *invoice_node_id* to ensure it's about to pay the correct | |||
| person, and makes a payment to the invoice. | |||
|
|
|||
| The pay-mobile-user flow (e.g. paying a friend back to their mobile node): | |||
| 1. The mobile user supplies some always-online node with a static (i.e. | |||
There was a problem hiding this comment.
How does this relate to the PTLC requirement that was mentioned in the original ML post, is that still a requirement? Maybe not because the payment is keysend now?
There was a problem hiding this comment.
If I'm understanding your question correctly, we still need PTLCs to get proof-of-payment back for async payments. AJ outlined a scheme for this here: https://diyhpl.us/~bryan/irc/bitcoin/bitcoin-dev/linuxfoundation-pipermail/lightning-dev/2023-January/003831.txt
There was a problem hiding this comment.
Yes that was indeed my question. Thanks for the link, understood.
| 3. The payer sends an `invoice_request` to the always-online node, who replies | ||
| with the static invoice previously provided by the mobile user if the mobile user | ||
| is offline. If they are online, the `invoice_request` is forwarded to the mobile | ||
| user as usual. |
There was a problem hiding this comment.
Why would you distinguish between the offline and online case - can't the offline flow be used always for simplicity?
There was a problem hiding this comment.
It's nice to fall back to the regular BOLT 12 flow because then the sender will get a fresh invoice/proof of payment, though as we talked about this won't happen in most cases.
|
|
||
| Setting `static_invoice_pay` indicates that the payer supports receiving a | ||
| `payment_hash`-less invoice in response to their `invoice_request`, and | ||
| subsequently setting `sender_provided_payment_preimage` in their payment onion. |
There was a problem hiding this comment.
Is this 'keysend' in LND terms?
There was a problem hiding this comment.
Yeah! The keysend term doesn't exist in the BOLTs right now so ended up going in this direction with naming...
There was a problem hiding this comment.
Makes sense. Send provided preimage is more descriptive too.
| be the mobile user's channel counterparty, wallet vendor, or another node on the | ||
| network that it has an out-of-band relationship with. | ||
| 2. The mobile user publishes an offer that contains blinded paths that terminate | ||
| at the always-online node. |
There was a problem hiding this comment.
The blinded paths, that isn't a strict requirement, or is it?
Also wondering how pathfinding works for the sender. They only get to do one hold htlc without retries, so it must be first time right?
There was a problem hiding this comment.
The blinded paths, that isn't a strict requirement, or is it?
IIUC it's not a strict requirement, no.
They only get to do one hold htlc without retries, so it must be first time right?
Discussed offline but the plan is to use trampoline such that the sender locks in their HTLC with their LSP using a trampoline onion, allowing said LSP to retry on their behalf.
|
|
||
| #### TLV fields for `release_held_htlc` | ||
|
|
||
| 1. `tlv_stream`: `release_held_htlc` |
There was a problem hiding this comment.
Are there interesting edge cases where the release is signaled, but then the receiver quickly goes offline after all?
There was a problem hiding this comment.
It may justify the sender's LSP resending the held_htlc_available message, otherwise I believe it's just like any other HTLC where the next-hop forward is offline.
There was a problem hiding this comment.
In this case trampoline is also going to help right? Because if there's just a single sender-determined route a retry is not possible because of onion reuse prevention?
| subsequently setting `sender_provided_payment_preimage` in their payment onion. | ||
|
|
||
| Useful if the payee is often offline and the invoice is being returned on | ||
| their behalf by another node, to avoid trusting that other node to not reuse a |
There was a problem hiding this comment.
If you would trust another node to not reuse a hash, does that then still offer much above just letting an LSP receive the money for you and trusting them to hand it over? Maybe it is not necessary to describe the option to not use sender_provided_payment_preimage
There was a problem hiding this comment.
I was thinking it would be good to explain the reasoning behind needing a payment_hash-less invoice, not sure I see what you're getting at with the first sentence though 🤔
There was a problem hiding this comment.
I was just wondering if wording should be stronger. That you can't just trust another node to provide an invoice, and that sender provided preimage is the only way to make it safe?
|
|
||
| 1. `tlv_stream`: `update_add_htlc_tlvs` | ||
| 2. types: | ||
| 1. type: 0 (`hold_htlc`) |
There was a problem hiding this comment.
Just to get a better feel for the design space: In a model where the sender LSP does the routing (I think Breez works like that, and also Phoenix with trampoline?), what extra options would that give?
Holding the htlc at the LSP makes sense of course, so that the sender can go offline. When the LSP knows the final destination though, they could just keep trying to complete the payment until is succeeds, and then claim the incoming htlc with the preimage. This would cut out all (not reliable?) onion message communication for hold and release.
There was a problem hiding this comment.
Discussed offline, but we currently expect async recipient wallets to come online infrequently and briefly. So it wouldn't be ideal for the sender's LSP to send a bunch of HTLCs and lock up liquidity repeatedly over a period of days until the recipient comes online. Onion message reliability is indeed a bit tbd though, although with direct-connect it becomes a lot more reliable today at the cost of privacy.
There was a problem hiding this comment.
Yes that makes sense. Didn't realize that the online periods may be so short that it might be impossible to hit.
| @@ -2047,6 +2079,12 @@ A receiving node: | |||
| - MUST respond with an error as detailed in [Failure Messages](04-onion-routing.md#failure-messages) | |||
| - Otherwise: | |||
| - MUST follow the requirements for the reader of `payload` in [Payload Format](04-onion-routing.md#payload-format) | |||
| - if the `hold_htlc` TLV is present: | |||
| - MUST NOT forward the HTLC until a corresponding `release_held_htlc` onion | |||
There was a problem hiding this comment.
Is it known what value LSPs typically use for max htlcs on their channels? If the LSP is the chan initiator, they may want to keep it low to avoid a high commit tx absolute fee. This would then limit the number of outstanding async payments.
There was a problem hiding this comment.
Good point, although maybe worth noting that the HTLC slot is only taken up on the mobile sender <> LSP channel, which isn't typically being used for any other payment forwards. LSPs may indeed want to limit the number of outstanding async payments though as you say.
There was a problem hiding this comment.
For small tips it may not be a problem because those do not add to the commit tx weight and fee risk for the LSP. Although the risk then is that the LSP might lose the htlc values in a close event.
| @@ -1972,6 +1972,25 @@ is destined, is described in [BOLT #4](04-onion-routing.md). | |||
| * [`sha256`:`payment_hash`] | |||
| * [`u32`:`cltv_expiry`] | |||
| * [`1366*byte`:`onion_routing_packet`] | |||
| * [`update_add_htlc_tlvs`:`tlvs`] | |||
|
|
|||
| 1. `tlv_stream`: `update_add_htlc_tlvs` | |||
There was a problem hiding this comment.
Discussed offline: maybe signal hold_htlc in the onion, so that the option remains open to park htlcs at a more remote node? Potentially as a way to unburden the sender LSP from async payments.
|
Appreciate the feedback and discussion @joostjager! |
This builds on #989 by adding the ability to fetch an invoice from an
always-online node on behalf of an often-offline recipient, e.g. a mobile node.
The idea is that often-offline recipients will supply some always-online node
such as their wallet vendor with a static keysend (i.e.
payment_hash-less)invoice to return on its behalf. The recipient will then publish an offer
containing blinded paths that terminate at this always-online node, who payers
can request the invoice from if the recipient is offline at the time. After
receiving the keysend invoice, payers will commence the protocol outlined in
1 to send the HTLC asynchronously.
Some context on the top commit where we include the invoice request in the payment onion:
This definitely warrants discussion, but the idea is that this field may be useful for often-offline recipients who did not receive the invoice request when it was originally sent. Recipients may want to verify the invreq or be provided some other relevant data about the payment, while keeping the payment stateless until an HTLC is actually received. For example, future extensions have been proposed 2 that require the recipient to know a unique token for a payment, and this field would provide that to them.
Seeking conceptual feedback! I'm also working on the implementation in LDK.
Based on
#798and #989.