Commit Graph

449 Commits

Author SHA1 Message Date
xiaolou86
50e5b0efd1 electrum: fix typos 2023-12-04 14:15:39 +08:00
SomberNight
4cdd199f5b lnworker: add/fix some type hints, add some comments
follow-up recent refactor
2023-11-20 11:34:56 +00:00
ThomasV
1cc92d4890 trampoline forwarding: before failing payment, wait until all htcs
have failed and session is not longer active.
2023-11-18 16:08:11 +01:00
ThomasV
9b1c40e396 Refactor payment forwarding:
- all forwarding types use the same flow
 - forwarding callback returns a htlc_key or None
 - forwarding info is persisted in lnworker:
   - ongoing_forwardings
   - downstream to upstream htlc_key
   - htlc_key -> error_bytes
2023-11-18 16:03:18 +01:00
ThomasV
a338459d45 just-in-time channels:
- a node scid alias is derived from the node ID
 - the channel opening fee is sent in a TLV field of open_channel
 - the server requires htlc settlement before broadcasting
   (server does not trust client)
2023-11-13 10:47:18 +01:00
ThomasV
816e617aaf option_zeroconf
- accept zeroconf channels only from a single node
 - fw_info uses get_scid_or_local_alias
2023-11-13 10:47:18 +01:00
ThomasV
96e143b6c3 lnpeer: abort send_revoke_and_ack and maybe_send_commitment if channel is closed. fixes #8684 2023-11-13 10:43:16 +01:00
SomberNight
6506abf583 lnworker: use PaymentFeeBudget
- introduce PaymentFeeBudget, which contains limits for fee budget and cltv budget
  - when splitting a payment,
    - the fee budget is linearly distributed between the parts
      - this resolves a FIXME in lnrouter ("FIXME in case of MPP")
    - the cltv budget is simply copied
  - we could also add other kinds of budgets later, e.g. for the num in-flight htlcs
- resolves TODO in lnworker ("todo: compare to the fee of the actual route we found")
2023-10-27 16:01:23 +00:00
ThomasV
b776d79f7f lnpeer: move access to payment_secret inside try..except block, as it might raise 2023-10-20 10:12:14 +02:00
SomberNight
22a8348303 renames: use consistent naming of cltv delta vs cltv abs
to avoid confusing relative vs absolute cltvs
(see b0401a6386)
2023-10-19 16:40:05 +00:00
SomberNight
783dc0cdd5 lnpeer.maybe_fulfill_htlc: also check cltv and amt against inner onion
- is_trampoline was True iff we are the final recipient of a trampoline payment
  - in that case, we were only comparing htlc.cltv_expiry against the outer onion cltv
  - we should and do now also compare against the inner onion cltv
- the checks are changed from "!=" to "<", to adapt to bolts PR 1032
  - see b38156b951
  - note that the leniency is needed for the cltv off-by-one
    added in eca10eb04d to work
2023-10-19 15:23:21 +00:00
ThomasV
4c42840c1c lnpeer: obfuscate error pakets of forwarded htlcs, that we
propageate back to the sender.

lnworker: in htlc_fulfilled and htlc_failed, return early if the
htlc was forwarded, so that we do not trigger invoice callbacks
2023-10-19 15:59:26 +02:00
SomberNight
eca10eb04d lnpeer.maybe_forward_trampoline: fix cltv calculation
follow-up b0401a6386
2023-10-18 19:22:52 +00:00
SomberNight
a059a9a256 lnpeer.pay: also log hops_data for trampoline_onion
We were already logging the outer-layer hops_data,
now we also log the inner trampoline-onion hops_data.

Example:
```
  1.12 | I | P/lnpeer.Peer.[MockLNWallet, alice->bob] | lnpeer.pay len(route)=1
  1.12 | I | P/lnpeer.Peer.[MockLNWallet, alice->bob] |   0: edge=9926297x9781928x61754 hop_data=<OnionHopsDataSingle. payload={'amt_to_forward': {'amt_to_forward': 100000000}, 'outgoing_cltv_value': {'outgoing_cltv_value': 601299}, 'payment_data': {'payment_secret': b'\xd2\x9cl\xdfV\xd4\xea_\x06{\xed\xc9\xc7\xa6\xf5\xc0\n\x1a\x95\xad\xad\xd2F\xb8;&\x9f\xa1\xe1\xd1\x07H', 'total_msat': 100000000, 'amount_msat': 100000000}}. hmac=None>
  1.12 | I | P/lnpeer.Peer.[MockLNWallet, alice->bob] | adding trampoline onion to final payload
  1.12 | I | P/lnpeer.Peer.[MockLNWallet, alice->bob] | lnpeer.pay len(t_route)=3
  1.12 | I | P/lnpeer.Peer.[MockLNWallet, alice->bob] |   0: t_node=02389c93b85ef8f7264c6fa3d3b239341c2631c2cab97e815b33453bd8d0254e77 hop_data=<OnionHopsDataSingle. payload={'amt_to_forward': {'amt_to_forward': 100000000}, 'outgoing_cltv_value': {'outgoing_cltv_value': 600723}, 'outgoing_node_id': {'outgoing_node_id': b'\x03\x06\xd9,\x9c\xabRe\x83Mr\x0b\x14(\xf5\x81\xf9\xfb\x9b\xfeV\xc1q\x85&L\xda\xffs\xe5y(\x81'}}. hmac=b'\xe7\x04\xe2>\x9a\xd9\xf0\x92<\xf8Q\xe4\xf4\xd8\x8cr{\x1e\xb1\xee\xb0\xd4R\xba\xe5\xfd\x83\xfc\xd7\xa7\x1dt'>
  1.12 | I | P/lnpeer.Peer.[MockLNWallet, alice->bob] |   1: t_node=0306d92c9cab5265834d720b1428f581f9fb9bfe56c17185264cdaff73e5792881 hop_data=<OnionHopsDataSingle. payload={'amt_to_forward': {'amt_to_forward': 100000000}, 'outgoing_cltv_value': {'outgoing_cltv_value': 600147}, 'outgoing_node_id': {'outgoing_node_id': b'\x03\x85v\xac:\xf8AUW\xcf\x1d\x12e\xcc\xff\xb1\xea\xd6\x01\xd5\x17HX?\x12\x83\x9cD\xbe\xebC\x82o'}}. hmac=b's-\xe1\xdb\xbc\xa5\x88\x90\xc0\xafu\xab\xba\xb6k\x81\xeae)#\x85\x12fm\xe6\xc3\xbd\xf6\x86eR\xd2'>
  1.12 | I | P/lnpeer.Peer.[MockLNWallet, alice->bob] |   2: t_node=038576ac3af8415557cf1d1265ccffb1ead601d51748583f12839c44beeb43826f hop_data=<OnionHopsDataSingle. payload={'amt_to_forward': {'amt_to_forward': 100000000}, 'outgoing_cltv_value': {'outgoing_cltv_value': 600147}, 'payment_data': {'payment_secret': b'B-P\x01\xc3\x1e#\x19\xf9!\xbb\xd8\xd1pu\xc7J\x11A\xa8J\xfe\xb8\x8a\xb8\xc4Oi\x0f\xe8\xac\xab', 'total_msat': 100000000}}. hmac=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'>
  1.12 | I | P/lnpeer.Peer.[MockLNWallet, alice->bob] | starting payment. len(route)=1.
```
2023-10-18 18:07:21 +00:00
SomberNight
b0401a6386 lnpeer.maybe_forward_trampoline: fix abs-cltv vs cltv-delta confusion
lnworker.pay_to_node(min_cltv_expiry=) expects a relative cltv, and
we were passing an absolute one.
It is not so clear what precisely should be passed here, and that is
because pay_to_node's API was not written with forwarding in mind.
The value chosen here is just some guess that typically should work.
2023-10-17 18:22:36 +00:00
ThomasV
cd712f2f60 follow-up 1ea49582ab 2023-10-17 12:52:02 +02:00
ThomasV
1ea49582ab fix type of forwarding_info 2023-10-17 12:36:15 +02:00
ThomasV
026a64de94 channel_announcements:
- construct_channel_announcement: return also whether
   node ids are in reverse order
 - maybe_send_channel_announcement:
   return early if signatures have not been received
2023-10-17 12:15:35 +02:00
ThomasV
98a4d7b60d public channels:
- send node and channel announcements.
 - store channel_flags in constraints
 - store signatures in local and remote configs
2023-10-16 13:54:16 +02:00
ThomasV
aad4fd6d48 Fix sending of 'channel_ready':
- send only once
 - in channel_reestablish, do not send it if we are not funded.
 - lnworker: do not send channel_ready before channel_reestablish
2023-10-15 10:58:37 +02:00
ThomasV
ac177577a6 lnpeer: do not set channel OPEN before channel_ready has been both sent and received.
fixes #8641
2023-10-13 16:56:25 +02:00
ThomasV
71d6cc0b35 lnpeer: split 'pay' method into 'create_onion_for_route'
and 'send_htlc'

This will be useful for just_in_time channels, as we will need
to create an onion before the channel object is available.
2023-09-28 10:35:15 +02:00
ThomasV
1af6972d03 Merge pull request #8493 from spesmilo/jsonpatch_new
partial-writes using jsonpatch
2023-09-28 09:23:14 +02:00
ThomasV
e206d264c8 trampoline forwarding: use routing hints
unit tests:
 - remove 'drop_dave' flag, replace it by depleted channel
 - add test for trampoline forwarding using routing hints
 - lower attempts to 2
2023-09-26 10:07:41 +02:00
ThomasV
7ca89f56ee partial-writes using jsonpatch
- partial writes are append only.

 - StoredDict objects will append partial writes to the wallet
   file when items are added, replaced, removed.

 - Lists in the wallet file that have not been registered
   as StoredObject are converted to StoredList, which
   overloads append() and remove(). Those methods too will
   append partial writes to the wallet file.

 - Unlike the old jsonpatch branch, this branch does not support
   file encryption. Encrypted files always fully rewritten, even
   if the change before encryption is a partial write.
2023-09-24 12:24:09 +02:00
ThomasV
4d68025cca maybe_forward_htlc: detect invalid onion before temporary chan failures 2023-09-19 09:44:06 +02:00
SomberNight
4c63d8729b add sanity checks we don't sign tx including dummy addr
Somewhat a follow-up to 649ce979ab.

This adds some safety belts so we don't accidentally sign a tx that
contains a dummy address.
Specifically we check that tx does not contain output for dummy addr:
- in wallet.sign_transaction
- in network.broadcast_transaction

The second one is perhaps redundant, but I think it does not hurt.
2023-09-16 04:36:08 +00:00
ThomasV
649ce979ab send tx change to lightning 2023-09-09 14:14:43 +02:00
ThomasV
a406f7bba0 lnpeer: return hold invoice callback after checking received amount 2023-09-06 20:59:32 +02:00
ThomasV
04c2129685 follow-up prev commit; do not return tw-info if forwarding is disabled 2023-09-06 10:35:31 +02:00
ThomasV
0e43ef2792 lnpeer: return fw_info for all parts of a MPP in a trampoline onion
Due to an indendation error, fw_info was returned only for one
HTLC of the MPP. Thus, maybe_fulfill_htlc was called again with
the remaining HTLCs, which could possibly be failed due to MPP
timeout, resulting in fund loss for the forwarder.
2023-09-06 09:51:07 +02:00
ThomasV
e80310eb32 lnpeer: trampoline forwarding failures may be indexed by outer or inner onion payment secret 2023-08-31 05:55:28 +02:00
SomberNight
2f2be1a606 lnpeer: follow-up OPTION_SUPPORT_LARGE_CHANNEL
follow-up 40f2087ac3
2023-08-09 15:40:42 +00:00
ThomasV
40f2087ac3 Add option for support_large_channels.
max_funding_sats is a config variable and defaults to the old value.
2023-08-09 16:36:12 +02:00
ThomasV
bf86cd6761 lnpeer and lnworker cleanup:
- rename trampoline_forwardings -> final_onion_forwardings,
   because this dict is used for both trampoline and hold invoices
 - remove timeout from hold_invoice_callbacks (redundant with invoice)
 - add test_failure boolean parameter to TestPeer._test_simple_payment,
   in order to test correct propagation of OnionRoutingFailures.
 - maybe_fulfill_htlc: raise an OnionRoutingFailure if we do not have
   the preimage for a payment that does not have a hold invoice callback.
   Without this, the above unit tests stall when we use test_failure=True
2023-08-09 13:23:26 +02:00
SomberNight
afac158c80 lnworker: clean-up sent_htlcs_q and sent_htlcs_info
- introduce SentHtlcInfo named tuple
  - some previously unnamed tuples are now much shorter:
    create_routes_for_payment no longer returns an 8-tuple!
- sent_htlcs_q (renamed from sent_htlcs), is now keyed on payment_hash+payment_secret
  (needed for proper trampoline forwarding)
2023-08-08 16:37:50 +00:00
SomberNight
44bdd20ccc lnworker: add RecvMPPResolution with "FAILED" state
- add RecvMPPResolution enum for possible states of a pending incoming MPP,
  and use it in check_mpp_status
  - new state: "FAILED", to allow nicely failing back the whole MPP set
- key more things with payment_hash+payment_secret, for consistency
  (just payment_hash is insufficient for trampoline forwarding)
2023-08-08 16:37:46 +00:00
SomberNight
c527ef8967 lnpeer: refuse to forward htlcs that correspond to payreq we created 2023-08-07 18:57:04 +00:00
SomberNight
8dd5865469 rm unused import
follow-up d51f00e2a3
2023-08-04 18:21:50 +00:00
SomberNight
d51f00e2a3 asyncio.wait_for() is too buggy. use util.wait_for2() instead
wasted some time because asyncio.wait_for() was suppressing cancellations. [0][1][2]
deja vu... [3]

Looks like this is finally getting fixed in cpython 3.12 [4]
So far away...
In attempt to avoid encountering this again, let's try using
asyncio.timeout in 3.11, which is how upstream reimplemented wait_for in 3.12 [4], and
aiorpcx.timeout_after in 3.8-3.10.

[0] https://github.com/python/cpython/issues/86296
[1] https://bugs.python.org/issue42130
[2] https://bugs.python.org/issue45098
[3] https://github.com/kyuupichan/aiorpcX/issues/44
[4] https://github.com/python/cpython/pull/98518
2023-08-04 18:18:21 +00:00
ThomasV
8bd1292e9a follow-up e5ac521d38 2023-07-26 19:20:02 +02:00
ThomasV
e5ac521d38 maybe_fulfill_htlc: check trampoline before hold invoice
order is important: if we receive a trampoline onion for a hold
invoice, we need to peel the onion through the recursive call.
2023-07-26 18:40:48 +02:00
ThomasV
49b5bf99ae fw_info: use hex value of payment_key, as this is persisted 2023-07-24 17:22:44 +02:00
ThomasV
141cd524bc lnpeer: do not run maybe_fulfill_htlc more than once, if it
triggered a payment forwarding.

Final onions may trigger a payment forwarding, through the callback
returned by maybe_fulfill_htlc. In that case, we should not fail the
HTLC later; doing so might result in fund loss.

Remove test_simple_payment_with_hold_invoice_timing_out: once we
have accepted to forward a payment HTLC with a hold invoice, we
do not want to time it out, for the same reason.
2023-07-24 12:44:56 +02:00
ThomasV
1296d3361d use payment_secret instead of payment_hash 2023-07-22 09:35:00 +02:00
ThomasV
8630030bd9 Restrict exception type in trampoline_forwarding_failures (follow-up 017186d107) 2023-07-21 19:35:27 +02:00
ThomasV
017186d107 Refactor trampoline forwarding and hold invoices.
- maybe_fulfill_htlc returns a forwarding callback that
   covers both cases.
 - previously, the callback of hold invoices was called as a
   side-effect of lnworker.check_mpp_status.
 - the same data structures (lnworker.trampoline_forwardings,
   lnworker.trampoline_forwarding_errors) are used for both
   trampoline forwardings and hold invoices.
 - maybe_fulfill_htlc still recursively calls itself to perform
   checks on trampoline onion. This is ugly, but ugliness is now
   contained to that method.
2023-07-21 13:40:10 +02:00
ThomasV
827792c14c lnpeer: simplify maybe_fulfill_htlc 2023-07-19 15:38:14 +02:00
ThomasV
e124ff7ee7 Trampoline MPP consolidation:
- fix parameters passed to maybe_forward_trampoline
 - use lnworker.trampoline_forwardings as a semaphore for ongoing
   trampoline payments
 - if a trampoline payment fails, fail all received HTLCs
2023-07-19 10:48:44 +02:00
ThomasV
4b29a46890 lnpeer: fix logging of 'will fullfill htlc' 2023-07-09 10:06:46 +02:00