Commit Graph

722 Commits

Author SHA1 Message Date
ThomasV
a6a2679d8a Merge pull request #8536 from SomberNight/202307_ln_imported_cb
fix sweeping chan after local force-close using cb
2023-07-28 11:16:48 +02:00
ThomasV
1411b75584 swapserver: add test for refund path 2023-07-26 19:20:18 +02:00
ThomasV
83dcc5e4cc payment bundles: fix bundle detection for trampoline
This feels a bit like workaround; it might be better to represent
payment bundles objects using payment secrets rather than payment
hashes.
2023-07-26 18:54:20 +02:00
ThomasV
3bb5ebf137 simplify check_mpp_status
(the distinction that was between is_accepted and is_expired does
not seem to be useful)
2023-07-24 16:43:06 +02:00
ThomasV
017186d107 Refactor trampoline forwarding and hold invoices.
- maybe_fulfill_htlc returns a forwarding callback that
   covers both cases.
 - previously, the callback of hold invoices was called as a
   side-effect of lnworker.check_mpp_status.
 - the same data structures (lnworker.trampoline_forwardings,
   lnworker.trampoline_forwarding_errors) are used for both
   trampoline forwardings and hold invoices.
 - maybe_fulfill_htlc still recursively calls itself to perform
   checks on trampoline onion. This is ugly, but ugliness is now
   contained to that method.
2023-07-21 13:40:10 +02:00
ThomasV
e124ff7ee7 Trampoline MPP consolidation:
- fix parameters passed to maybe_forward_trampoline
 - use lnworker.trampoline_forwardings as a semaphore for ongoing
   trampoline payments
 - if a trampoline payment fails, fail all received HTLCs
2023-07-19 10:48:44 +02:00
ThomasV
aeaf9c71df Add unit test for trampoline MPP consolidation
This tests that a trampoline waits until all incoming HTLCs are
received, and fail or succeed them together
2023-07-19 09:49:05 +02:00
SomberNight
1a46460d11 fix sweeping chan after local force-close using cb
scenario:
- user opens a lightning channel and exports an "imported channel backup"
- user closes channel via local-force-close
  - local ctx is published, to_local output has user's funds and they are CSV-locked for days
- user restores wallet file from seed and imports channel backup
- new wallet file should be able to sweep coins from to_local output (after CSV expires)

This was not working previously, as the local_payment_basepoint was not included in the
imported channel backups, and the code was interpreting the lack of this as the channel not
having option_static_remotekey enabled. This resulted in lnutil.extract_ctn_from_tx
using an incorrect funder_payment_basepoint, and lnsweep not recognising the ctx due to
the garbage ctn value.

The imported channel backup serialisation format is slightly changed to include the
previously missing field, and its version number is bumped (0->1). We allow importing
both version 0 and version 1 backups, however v0 backups cannot handle the above
described scenario (they can only be used to request a remote-force-close).

Note that we were/are setting the missing local_payment_basepoint to the pubkey of
one of the wallet change addresses, which is bruteforceable if necessary, but I
think it is not worth the complexity to add this bruteforce logic. Also note
that the bruteforcing could only be done after the local-force-close was broadcast.

Ideally people with existing channels and already exported v0 backups should re-export
v1 backups... Not sure how to handle this.

closes https://github.com/spesmilo/electrum/issues/8516
2023-07-14 14:29:21 +00:00
SomberNight
fc6486ecdb lnaddr: make payment_secret field mandatory, in both directions
we now require payment_secret both for sending and for receiving
(previously was optional for both)

see
https://github.com/lightning/bolts/pull/898
https://github.com/ACINQ/eclair/pull/1810
https://github.com/ElementsProject/lightning/pull/4646

note: payment_secret depends on var_onion_optin, so that becomes mandatory as well,
however this commit does not yet remove the ability of creating legacy onions
2023-06-29 14:34:02 +00:00
SomberNight
8ef2495096 lnworker: use NamedTuple for received_mpp_htlcs. add/fix type hints
try to avoid long plain tuples
2023-06-29 13:48:02 +00:00
ThomasV
c4c2123b4b fix bundled payments:
- prepayment should be accepted immediately once bundle is here
 - mpp timeout all parts, but accept only current part
2023-06-28 14:51:09 +02:00
ThomasV
c4eb7d8321 lnworker: bundled payments
- htlcs of bundled payments must arrive in the same MPP_TIMEOUT
window, or they will be failed
- add correspoding tests
2023-06-28 13:01:37 +02:00
ThomasV
1acf426fa9 lnworker: add support for hold invoices
(invoices for which we do not have the preimage)

Callbacks and timeouts are registered with lnworker. If the
preimage is not known after the timeout has expired, the payment
is failed with MPP_TIMEOUT.
2023-06-26 11:09:47 +02:00
ThomasV
df5b98792e lnworker: always call check_received_htlc (no only for MPP)
This will be a generic placeholder to decide if we need to wait
before settling a payment (to be used with hold invoices and
bundled payments)
2023-06-26 09:29:40 +02:00
ThomasV
21e06b7065 lnpeer: new payment secret, derived without preimage.
(this is needed for hold invoices)
2023-06-25 19:15:52 +02:00
SomberNight
f7a8e55a6a lnworker: (trivial) clean-up which bolt-09 feature flags we set 2023-06-23 19:53:31 +00:00
SomberNight
ca93af2b8a ln: some clean-up for option_scid_alias
- qt chan details dlg: show both local and remote aliases
- lnchannel: more descriptive names, add clarification in doctstrings,
  and also save the "local_scid_alias" in the wallet file (to remember if
  we sent it)
- lnpeer:
  - resend channel_ready msg after reestablish, to upgrade old existing channels
    to having local_scid_alias
  - forwarding bugfix, to follow BOLT-04:
    > - if it returns a `channel_update`:
    >   - MUST set `short_channel_id` to the `short_channel_id` used by the incoming onion.
2023-06-23 19:51:57 +00:00
ThomasV
4177f8be82 lnworker.create_routes_for_payment: fix MPP detection 2023-06-21 08:55:17 +02:00
SomberNight
033ad0feb9 lnworker: fix rebalance_channels
fixes https://github.com/spesmilo/electrum/issues/8468
2023-06-04 03:07:06 +00:00
SomberNight
24980feab7 config: introduce ConfigVars
A new config API is introduced, and ~all of the codebase is adapted to it.
The old API is kept but mainly only for dynamic usage where its extra flexibility is needed.

Using examples, the old config API looked this:
```
>>> config.get("request_expiry", 86400)
604800
>>> config.set_key("request_expiry", 86400)
>>>
```

The new config API instead:
```
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS
604800
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS = 86400
>>>
```

The old API operated on arbitrary string keys, the new one uses
a static ~enum-like list of variables.

With the new API:
- there is a single centralised list of config variables, as opposed to
  these being scattered all over
- no more duplication of default values (in the getters)
- there is now some (minimal for now) type-validation/conversion for
  the config values

closes https://github.com/spesmilo/electrum/pull/5640
closes https://github.com/spesmilo/electrum/pull/5649

Note: there is yet a third API added here, for certain niche/abstract use-cases,
where we need a reference to the config variable itself.
It should only be used when needed:
```
>>> var = config.cv.WALLET_PAYREQ_EXPIRY_SECONDS
>>> var
<ConfigVarWithConfig key='request_expiry'>
>>> var.get()
604800
>>> var.set(3600)
>>> var.get_default_value()
86400
>>> var.is_set()
True
>>> var.is_modifiable()
True
```
2023-05-25 17:39:48 +00:00
accumulator
ac8a7a0784 channel_db: raise specific exception when channelDB not loaded, allowing lnworker to mark payment as failed. (#8431)
On mobile, it can take a while before channelDB is loaded. If payment is attempted before the DB
is fully loaded, this would result in a payment failure, but also leaves the payment attempt in IN_PROGRESS
state. This patch adds a more specific ChannelDBNotLoaded exception class, so we can handle this case more
gracefully, since we know the payment didn't succeed.
2023-05-16 13:08:26 +00:00
SomberNight
38ec72527f wallet.get_bolt11_invoice: handle request not having LN part
fixes https://github.com/spesmilo/electrum/issues/8402

To reproduce,
- create wallet from a zpub
- LN is disabled there by default
- create a receive request, which won't have a lightning part
- enable config var "bip21_lightning"
- enable LN
- existing request is now ~breaking receive tab
2023-05-07 00:39:06 +00:00
SomberNight
312f2641e7 don't use bare except
use "except Exception", or if really needed explicitly "except BaseException"
2023-04-24 12:58:01 +00:00
ThomasV
0fcf423fbe Revert "lnworker: schedule_force_closing is not async"
This reverts commit fcf836bc94.
2023-04-22 17:20:47 +02:00
ThomasV
fcf836bc94 lnworker: schedule_force_closing is not async 2023-04-22 15:03:39 +02:00
SomberNight
6ac3f84095 wallet/lnworker/etc: add sanity checks to start_network methods
related: https://github.com/spesmilo/electrum/issues/8301
2023-04-17 16:56:57 +00:00
SomberNight
db4943ff86 wallet.get_full_history: more consistent sort order
before:
```
>>> [print(wallet.get_tx_status(txid, wallet.adb.get_tx_height(txid))) for txid in list(wallet.get_full_history())[-10:]]
(7, '2023-04-04 16:13')
(7, '2023-04-04 16:13')
(7, '2023-04-04 16:13')
(7, '2023-04-04 16:13')
(0, 'Unconfirmed [20. sat/b, 0.00 MB]')
(2, 'in 2 blocks')
(3, 'Local [180.4 sat/b]')
(3, 'Local [180.2 sat/b]')
(2, 'in 2016 blocks')
(0, 'Unconfirmed [180. sat/b, 0.00 MB]')
```

after:
```
>>> [print(wallet.get_tx_status(txid, wallet.adb.get_tx_height(txid))) for txid in list(wallet.get_full_history())[-10:]]
(7, '2023-04-04 16:13')
(7, '2023-04-04 16:13')
(7, '2023-04-04 16:13')
(7, '2023-04-04 16:13')
(0, 'Unconfirmed [20. sat/b, 0.00 MB]')
(0, 'Unconfirmed [180. sat/b, 0.00 MB]')
(2, 'in 2016 blocks')
(2, 'in 2 blocks')
(3, 'Local [180.4 sat/b]')
(3, 'Local [180.2 sat/b]')
```
2023-04-04 18:32:53 +00:00
SomberNight
a30cda4ebd lnutil: test ImportedChannelBackupStorage.from_bytes
regression test - we should not inadvertently break deserialising existing backups
2023-03-19 19:15:44 +00:00
SomberNight
f2dc651c9b Qt history list: Ctrl+F filter to work for "Short ID" (scid) 2023-03-10 14:03:57 +00:00
SomberNight
ec889b8c3f wallet: fix import_requests, and mention quirk re preimages 2023-03-03 16:35:34 +00:00
SomberNight
a1a1fae4cc invoices.py: small clean-up 2023-03-03 16:02:19 +00:00
SomberNight
d11237d6a1 lnworker: start watching already redeemed chans if txs are missing
This fixes a bug where if one runs `wallet.clear_history()` they would see exceptions later:
```
Traceback (most recent call last):
  File "/home/user/wspace/electrum/electrum/gui/qt/main_window.py", line 866, in timer_actions
	self.update_wallet()
  File "/home/user/wspace/electrum/electrum/gui/qt/main_window.py", line 1021, in update_wallet
	self.update_tabs()
  File "/home/user/wspace/electrum/electrum/gui/qt/main_window.py", line 1033, in update_tabs
	self.utxo_list.update()
  File "/home/user/wspace/electrum/electrum/gui/qt/utxo_list.py", line 103, in update
	self.refresh_row(name, idx)
  File "/home/user/wspace/electrum/electrum/gui/qt/utxo_list.py", line 124, in refresh_row
	parents = self.wallet.get_tx_parents(txid)
  File "/home/user/wspace/electrum/electrum/wallet.py", line 885, in get_tx_parents
	result.update(self.get_tx_parents(_txid))
  File "/home/user/wspace/electrum/electrum/wallet.py", line 881, in get_tx_parents
	for i, txin in enumerate(tx.inputs()):
AttributeError: 'NoneType' object has no attribute 'inputs'
```
This is related to the privacy analysis, which assumes that for each tx item in the history list
we should have the raw tx in the db. This is no longer true after wallet.clear_history(), if
the wallet has certain LN channels. E.g. an already redeemed channel that was local-force-closed,
as that closing tx is not related to the wallet directly.

In commit 3541ecb576, we decided not to watch already redeemed channels.
This is potentially good for e.g. privacy, as the server would otherwise see us subscribe to that chan.
However it means that after running wallet.clear_history() txs related to the channel but not to the
wallet won't be re-downloaded.

Instead, now if there are missing txs for a redeemed channel, we start watching it, hence the
synchronizer will re-downloaded the txs.
2023-03-01 16:20:42 +00:00
ThomasV
719b468eee Refresh bolt11 routing hints when channel liquidity changes:
- wallet_db update: separate Invoices and Requests.
 - do not store bolt11 invoice in Request
2023-02-28 15:33:17 +01:00
ThomasV
72fb43f950 lnworker: do not assume MPP in num_sats_can_receive 2023-02-25 12:23:34 +01:00
SomberNight
373db76ac9 util: kill bh2u
no longer useful, and the name is so confusing...
2023-02-17 11:43:11 +00:00
ThomasV
768eb35c86 follow-up 5d9678a269 2023-01-24 14:45:55 +01:00
ThomasV
062051b530 lnworker: store onchain default labels in a cache 2023-01-23 10:48:25 +01:00
ThomasV
5d9678a269 lnworker: extend swap label only if we are still watching the address
Without this, old swap transactions for which we have deleted the
channel are incorrectly labeled.
2023-01-21 12:39:19 +01:00
ThomasV
b9393b0603 Support scid alias:
- save remote alias for use in invoices
 - derive local alias from wallet xpub
 - send channel_type without the option_scid_alias bit
   (apparently LND does not like it)
2023-01-13 15:47:30 +01:00
ThomasV
c109d5e722 lnwire: update csv files with recent BOLTs
Note: there are no more optional fields in msgdata, per f068dd0d8d
2023-01-13 12:50:48 +01:00
ThomasV
d95f3e5622 Use different trampoline bits than Eclair. Fixes #8141 2023-01-13 12:48:13 +01:00
ThomasV
d70a574780 Increase trampoline fee if we receive TEMPORARY_CHANNEL_FAILURE
Sometimes the trampoline node fails to send a payment and returns
TEMPORARY_CHANNEL_FAILURE, while it succeeds with a higher trampoline
fee. My guess is that the initial fee was not sufficient to try
all routes, and that a higher fee allows to use extra routes. I
suppose it would make more sense if the trampoline node returned
TRAMPOLINE_FEE_INSUFFICIENT in that case.
2023-01-13 11:02:19 +01:00
accumulator
6e536d2d91 add invoice status to invoice_status callback (#8020)
* add invoice status to invoice_status callback

* debug statement fails tests

* removed commented lines, added progress/attempt counter comment in lnworker.pay_to_node,
and update the invoice_status event handler in qeinvoicelistmodel.py
2022-10-21 16:45:12 +00:00
ThomasV
8867387847 lnworker: fix stuck payment loop (fixes #7995)
If the short_channel_id of a channel update received
with update_fail_htlc does not match the channel in
our route, blacklist the channel in our route.
2022-10-04 10:51:18 +02:00
ThomasV
20bcb15480 Factorize code in create_routes_for_payment.
Calling suggest_splits with exclude_single_part_payments=False
2022-09-20 13:55:22 +02:00
ThomasV
ead886781a create_routes_for_payment: fix 'we atomically loop'
Before this commit, partial results were yielded before we had
examined the whole split configuration.

I think the correct behaviour is to create all the routes for the
current configuration, then to yield them. If there is an exception,
we should try the next one.
2022-09-20 12:52:36 +02:00
ThomasV
5e77886b98 trampoline MPP: handle direct payment to trampoline node 2022-09-20 11:44:23 +02:00
ThomasV
478046289a follow-up c6f940d092 2022-09-20 10:55:19 +02:00
ThomasV
2af59e32b2 lnworker: define use_trampoline() for code clarity 2022-09-19 17:43:13 +02:00
ThomasV
68bf714ae6 Trampoline: Remember failed routes (fixes #7967).
Upon receiving UNKNOWN_NEXT_PEER, TEMPORARY_NODE_FAILURE
or TEMPORARY_CHANNEL_FAILURE, remember the trampoline route
that was used, and try other routes before raising the
trampoline fee.

Before this commit, we used to raise the trampoline fee
upon receiving any error message, in order to ensure
termination of the payment loop.

Note that we will still retry failing routes after we have
raised the trampoline fee. This choice is questionable, it
is unclear if doing so significantly increases the probability
of success.

Tests: add a test for trampoline handling of UNKNOWN_NEXT_PEER
2022-09-19 16:42:46 +02:00