Commit Graph

177 Commits

Author SHA1 Message Date
SomberNight
d4338fb503 tests: clean-up use of asyncio 2023-02-20 16:53:44 +00:00
SomberNight
c5bdd5007c tests: rework testnet
Inheritance was overkill here, and now we can use inheritance for new functionality X
without having to create classes for all combinations of {X, is_testnet}.
2023-02-18 11:29:54 +00:00
SomberNight
373db76ac9 util: kill bh2u
no longer useful, and the name is so confusing...
2023-02-17 11:43:11 +00:00
ThomasV
b9393b0603 Support scid alias:
- save remote alias for use in invoices
 - derive local alias from wallet xpub
 - send channel_type without the option_scid_alias bit
   (apparently LND does not like it)
2023-01-13 15:47:30 +01:00
ThomasV
d95f3e5622 Use different trampoline bits than Eclair. Fixes #8141 2023-01-13 12:48:13 +01:00
SomberNight
762a9a9e7a follow-up prev: fix tests 2023-01-13 10:55:52 +00:00
ThomasV
6e429ebf9a test_lnpeer: instead of sleeping for 0.2s, wait until peers are initialized
this should fix https://cirrus-ci.com/task/6585628493086720
2022-09-23 12:42:15 +02:00
ThomasV
fe6a83d6c1 fix test_fail_pending_htlcs_on_shutdown: poll number of htlcs 2022-09-22 18:26:02 +02:00
ThomasV
79bcedd064 test_lnpeer: reset electrum.trampoline._TRAMPOLINE_NODES_UNITTESTS in tearDown 2022-09-20 19:17:32 +02:00
ThomasV
20bcb15480 Factorize code in create_routes_for_payment.
Calling suggest_splits with exclude_single_part_payments=False
2022-09-20 13:55:22 +02:00
ThomasV
9251f482b5 test_lnpeer: add unit test for direct payment to trampoline node 2022-09-20 11:28:51 +02:00
ThomasV
2af59e32b2 lnworker: define use_trampoline() for code clarity 2022-09-19 17:43:13 +02:00
ThomasV
68bf714ae6 Trampoline: Remember failed routes (fixes #7967).
Upon receiving UNKNOWN_NEXT_PEER, TEMPORARY_NODE_FAILURE
or TEMPORARY_CHANNEL_FAILURE, remember the trampoline route
that was used, and try other routes before raising the
trampoline fee.

Before this commit, we used to raise the trampoline fee
upon receiving any error message, in order to ensure
termination of the payment loop.

Note that we will still retry failing routes after we have
raised the trampoline fee. This choice is questionable, it
is unclear if doing so significantly increases the probability
of success.

Tests: add a test for trampoline handling of UNKNOWN_NEXT_PEER
2022-09-19 16:42:46 +02:00
ThomasV
26245c1ac9 unit tests: split test_payment_trampoline into two tests 2022-09-03 11:16:37 +02:00
SomberNight
05863a611f tests: test_lnpeer.py: clean-up temp dirs created in /tmp, in tearDown 2022-08-10 18:16:17 +02:00
ThomasV
dbf055de9a EventListener class to handle callbacks
and QtEventListener for Qt
2022-06-22 02:07:46 +02:00
ThomasV
9fe93524b7 Index lightning requests with rhash instead of onchain address.
get_unused_addresses() has been broken since #7730, because
addresses are considered as permanently used if they are in
the list of keys of receive_requests. This is true even if
an address is used as fallback for a lightning payment. This
means that the number of lightning payments we can receive
is constrained by the gap limit.

If a payment succeeds off-chain, we want to be able to reuse
its fallback address in other requests (this does not reduce
privacy, because invoices already share the same public key).

This implies that we should not use the onchain address as key
for lightning-enabled requests in wallet.receive_requests. If
we did, paid invoices would be overwritten when the address is
reused. That is the reason for the wallet_db upgrade.

Related: a3faf85e3c
2022-06-15 18:44:52 +02:00
ThomasV
a3faf85e3c wallet:
- add new index: requests_rhash_to_key (fixes #7845)
 - when creating a request, do not save its description in labels.
   Instead, return it as default value in wallet.get_label_by_rhash
lnworker:
  - rename 'payments' to 'payment_info'
  - add note to delete_payment_info
commands: rename 'rmrequest' to 'delete_request'
2022-06-14 13:39:18 +02:00
ThomasV
121d8732f1 Persist LNWatcher transactions in wallet file:
- separate AddressSynchronizer from Wallet and LNWatcher
 - the AddressSynchronizer class is referred to as 'adb' (address database)
 - Use callbacks to replace overloaded methods
2022-06-10 13:07:53 +02:00
SomberNight
ef4477a930 lnpeer.reestablish_chan: enforce order of replaying commitsig/revack
When replaying messages during channel-reestablishment,
previously we first resent all update messages, along with potential commitment_signed messages,
and then we potentially resent a single revoke_and_ack.

This can result in incorrect behaviour in case both a commitment_signed and a revoke_and_ack needs to be resent.
When replaying messages, the relative order of commitment_signed and revoke_and_messages needs to be preserved.
(the order of updates (htlc/fee) in relation to the revack messages does not matter)

implements https://github.com/lightning/bolts/pull/810

The logic here is somewhat based on what c-lightning does:
01e5f1886e/channeld/channeld.c (L3059)
2022-05-25 19:44:44 +02:00
ThomasV
53151244e2 LNWorker: Add suggest_rebalance methods for sending and receiving.
These methods return a list of channels that can be rebalanced,
in order to receive or send a given amount.

Also add 'channels' parameter to submarine swaps.
Previously, swaps were not considering which channel to use.

When we do not have liquidity to pay an invoice:
 - add 'rebalance' option in order to pay an invoice
 - use the suggested channel in the 'swap' option

When we do not have the liquidity to receive an invoice:
 - add 'Rebalance' and 'Swap' buttons to the receive tab
2022-05-21 20:25:44 +02:00
SomberNight
dd5cb2a5c1 lnworker: rework num_sats_can_receive and routing_hints_for_invoice
follow-up https://github.com/spesmilo/electrum/pull/7818

- note it matters whether a sender pays us end-to-end-trampoline or just via legacy
  - consider: Alice has 0.1 BTC recv cap in chan1 and 1 BTC recv cap in chan2, both with border-node T1
    - if sender is paying e2e trampoline, it can realistically pay even ~1.1 BTC, as T1 can resplit the HTLCs
    - if sender is paying legacy, it will have a hard time trying to pay more than 1 BTC, in practice
      - although note if T1 has implemented non-strict-forwarding (see BOLT-04), achieving 1 BTC is easy,
        as T1 can redirect HTLCs (but cannot split them, in this case)
  - to make num_sats_can_receive realistic, it assumes the legacy case
- To calc num_sats_can_receive, we sort our channels in decreasing order of receive-capacities, iterate over them
  and calculate a running sum - we stop adding channels when the next chan's recv cap is small compared to
  the running total.
- When putting routing hints in an invoice, we do the same, with the added condition that we keep adding channels
  if their recv cap is larger than the invoice amount.
  - consider: Alice has 0.1 BTC recv cap in chan1 with Bob, and 1 BTC recv cap in chan2 with Carol
    - if Alice wants to recv 100 sats, it is useful to add hints for both channels into the invoice, for redundancy
    - if Alice wants to recv 0.9 BTC, it is questionable whether adding the smaller chan is useful - the code here won't add it
2022-05-19 18:28:04 +02:00
ThomasV
f90a08bbe2 Filter nodes for receiving:
- increase MPP_RECEIVE_CUTOFF from 5 to 20 percent
 - filter channels by node_id, not channel_id
 - make num_sats_can_receive consistent with routing hints
2022-05-18 18:11:40 +02:00
SomberNight
872ce82418 tests: clean up event-loop creation 2022-05-04 01:53:21 +02:00
SomberNight
2c57c78ebe asyncio: stop using get_event_loop(). introduce ~singleton loop.
asyncio.get_event_loop() became deprecated in python3.10. (see https://github.com/python/cpython/issues/83710)
```
.../electrum/electrum/daemon.py:470: DeprecationWarning: There is no current event loop
  self.asyncio_loop = asyncio.get_event_loop()
.../electrum/electrum/network.py:276: DeprecationWarning: There is no current event loop
  self.asyncio_loop = asyncio.get_event_loop()
```
Also, according to that thread, "set_event_loop() [... is] not deprecated by oversight".
So, we stop using get_event_loop() and set_event_loop() in our own code.
Note that libraries we use (such as the stdlib for python <3.10), might call get_event_loop,
which then relies on us having called set_event_loop e.g. for the GUI thread. To work around
this, a custom event loop policy providing a get_event_loop implementation is used.

Previously, we have been using a single asyncio event loop, created with
util.create_and_start_event_loop, and code in many places got a reference to this loop
using asyncio.get_event_loop().
Now, we still use a single asyncio event loop, but it is now stored as a global in
util._asyncio_event_loop (access with util.get_asyncio_loop()).

I believe these changes also fix https://github.com/spesmilo/electrum/issues/5376
2022-04-29 18:49:07 +02:00
ThomasV
f4e902e907 LNWorker: give up payment after timeout, not number of attempts.
Limiting attempts may interrupt a MPP before we receive a MPP_timeout
The attempts parameter is still used in unit tests.
2022-04-29 12:17:38 +02:00
ThomasV
149cccbc4a fix tests (follow-up 2117118047) 2022-04-23 19:40:21 +02:00
ThomasV
60865f3902 Show options if we do not have the liquidity to pay a lightning invoice:
pay onchain, open channel, rebalance.

If we do a swap or open a channel, the payment will be scheduled.
2022-04-20 12:48:22 +02:00
ThomasV
cb39bbbd94 lnworker: make calc_routing_hints_for_invoice and create_invoice non-async 2022-03-29 17:42:04 +02:00
bitromortac
3915045067 lnpeer: warnings for shutdown and open_channel 2022-03-09 13:40:44 +01:00
bitromortac
9e800172ec lnpeer: send/handle error and warning messages
* adds methods for sending protocol errors/warnings
* handling of warning messages
2022-03-09 13:40:44 +01:00
ThomasV
6667a79f10 modern shutdown:
- clarify TODOs
 - add tests for shutdown with modern negotiation
2022-03-08 11:57:19 +01:00
ThomasV
0b203f0b94 lnpeer: refactor fee negotiation in _shutdown
- the fee negotiation is split into smaller functions, reducing the scope of variables.
  - the while loop logic is condensed in a few lines, so it is easier to understand termination conditions.
  - removed code that was never executed
2022-03-08 11:55:40 +01:00
SomberNight
556b98736e lnworker.try_force_closing: changed to not be async (and renamed)
This is to ensure that the channel is "immediately" set to FORCE_CLOSING.
(previously it took at least one event loop iteration)
2022-02-21 18:09:45 +01:00
ThomasV
b268877d53 Merge pull request #7636 from bitromortac/2201-channel-type
lightning: implement channel types
2022-02-21 12:08:54 +01:00
ThomasV
4ebe41b3a7 Trampoline MPP: save fee level in sent_htlcs_info.
If multiple HTLCs fail at the same fee level with
TRAMPOLINE_INSUFFICIENT_FEE, bump trampoline_fee_level only once.
2022-02-19 15:20:54 +01:00
ThomasV
9fd18ae7f4 Merge pull request #7623 from bitromortac/2201-multi-trampoline-mpp
Multi-trampoline multipart payments
2022-02-19 14:44:21 +01:00
ThomasV
b2f84187bc Split code in reestablish_channel:
Messages are sent in reestablish_channel (async)
  Message checks and force_close are performed in on_channel_reestablish (not async).
  That task should not be cancelled if the connection is closed.
  Revert 57583c05cf
2022-02-19 10:37:50 +01:00
SomberNight
2f549e84db fix tests: follow-up 96fcf68d84 2022-02-18 19:32:27 +01:00
bitromortac
a4f5cfc91a trampoline: refactor routes, enable e2e mpp
* Refactor `create_trampoline_route`.
* Enables end-to-end multi-trampoline multipart payments.
  Trampoline-to-legacy payments are still not enabled, as this is
  currently not supported by Eclair.
* Reverts to a global trampoline fee level, as trampoline failures
  are currently not handled properly, see (#7648), which doubles
  fee rates.
2022-02-18 10:14:51 +01:00
SomberNight
40c1597c0a lntransport: change name used in logs to make collisions unlikely
In particular, in the regtests, with incoming peers, we can have multiple transports open with the same node simultaneously
(see e.g. lnworker._request_force_close_from_backup).
We now use the first few bytes of peer_pubkey, as that is potentially familiar to users,
and the first few bytes of sha256(id(self)) to mitigate collisions in case the peer_pubkeys collide.

log excerpt:
```
I/P | lnpeer.Peer.[LNWallet, 030f0bf260-e0b33756] | handshake done for 030f0bf260acdbd3edcad84d7588ec7c5df4711e87e6a23016f989b8d3a4147230@163.172.94.64:9735
D/P | lnpeer.Peer.[LNWallet, 030f0bf260-e0b33756] | Sending INIT
I/P | lnpeer.Peer.[LNWallet, 03933884aa-5e5dce45] | handshake done for 03933884aaf1d6b108397e5efe5c86bcf2d8ca8d2f700eda99db9214fc2712b134@34.250.234.192:9735
D/P | lnpeer.Peer.[LNWallet, 03933884aa-5e5dce45] | Sending INIT
D/P | lnpeer.Peer.[LNWallet, 030f0bf260-e0b33756] | Received INIT
I/P | lnpeer.Peer.[LNWallet, 02651acf4a-79696c42] | handshake done for 02651acf4a7096091bf42baad19b3643ea318d6979f6dcc16ebaec43d5b0f4baf2@82.119.233.36:19735
D/P | lnpeer.Peer.[LNWallet, 02651acf4a-79696c42] | Sending INIT
D/P | lnpeer.Peer.[LNWallet, 03933884aa-5e5dce45] | Received INIT
I/P | lnpeer.Peer.[LNWallet, 030f0bf260-e0b33756] | saved remote_update
D/P | lnpeer.Peer.[LNWallet, 030f0bf260-e0b33756] | Received CHANNEL_REESTABLISH
```
2022-02-16 18:53:24 +01:00
SomberNight
c9c094cfab requirements: bump min aiorpcx to 0.22.0
aiorpcx 0.20 changed the behaviour/API of TaskGroups.
When used as a context manager, TaskGroups no longer propagate
exceptions raised by their tasks. Instead, the calling code has
to explicitly check the results of tasks and decide whether to re-raise
any exceptions.
This is a significant change, and so this commit introduces "OldTaskGroup",
which should behave as the TaskGroup class of old aiorpcx. All existing
usages of TaskGroup are replaced with OldTaskGroup.

closes https://github.com/spesmilo/electrum/issues/7446
2022-02-15 18:22:44 +01:00
bitromortac
6915e3cb10 lnpeer+wallet: use channel type for channel open
* channel_type is put into storage, serialized as int and
  deserialized as ChannelType
* check for static_remotekey is done via channel type
2022-01-20 16:47:48 +01:00
bitromortac
2d2b889312 lnpeer tests: add spp trampoline payment 2021-12-20 16:45:14 +01:00
bitromortac
e8c94cf5d7 tests: implement graph with flexible definitions 2021-12-20 16:45:07 +01:00
SomberNight
7354feeffe follow-up fix tests: logic typo
follow-up https://github.com/spesmilo/electrum/pull/7202

defaultdict[int] is a type!

```
>>> from collections import defaultdict
>>> d = defaultdict[int]
>>> d[2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: There are no type variables left in collections.defaultdict[int]
```

Also, prior to py3.9, it is a TypeError.
2021-12-17 15:21:21 +01:00
ghost43
ce44a03c24 Merge pull request #7202 from bitromortac/2104-mpp-channel-splitting
MPP splitting algorithm: redesign and split within channels
2021-12-17 13:58:54 +00:00
SomberNight
56b03e2e8d lnpeer: more forwarding is now event-driven
This should make unit tests less reliant on sleeps.
2021-11-04 19:16:02 +01:00
SomberNight
12f3525df0 lnpeer: disable msg processing rate-limiting in tests 2021-11-04 18:04:16 +01:00
SomberNight
cb55a2d654 tests: try to reduce flakyness of test_fail_pending_htlcs_on_shutdown
Alice sends and HTLC: Alice->Carol->Dave
we need a lot of messages back and forth to happen:
- Alice adds HTLC to chan_AC, sends sig, Carol revacks, sends sig, Alice revacks;
- only then Carol adds HTLC to chan_CD, sends sig, Dave revacks, sends sig, Carol revacks
on CI, 0.5 seconds is often not enough for this it seems.
2021-10-27 16:46:15 +02:00