Commit Graph

634 Commits

Author SHA1 Message Date
ThomasV
dd3481c9e1 Merge pull request #7821 from SomberNight/202205_lnworker_num_sats_can_receive
lnworker: rework num_sats_can_receive and routing_hints_for_invoice
2022-05-20 10:01:59 +02:00
SomberNight
2ec9e869b3 invoice.get_amount_sat: handle None in more places
I believe lightning requests created before https://github.com/spesmilo/electrum/pull/7730
can have an amount of None - ones created after have amount 0 instead.
We could do a wallet db upgrade potentially.
Regardless, the type hint is `get_amount_sat(self) -> Union[int, str, None]`,
so None should be handled. (well, arguably "!" should be handled too...)

```
E | gui.qt.exception_window.Exception_Hook | exception caught by crash reporter
Traceback (most recent call last):
  File "...\electrum\electrum\gui\qt\request_list.py", line 101, in item_changed
    self.parent.show_receive_request(req)
  File "...\electrum\electrum\gui\qt\main_window.py", line 1279, in show_receive_request
    URI = req.get_bip21_URI(lightning=bip21_lightning)
  File "...\electrum\electrum\invoices.py", line 164, in get_bip21_URI
    amount = int(self.get_amount_sat())
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
```

```
E | gui.qt.exception_window.Exception_Hook | exception caught by crash reporter
Traceback (most recent call last):
  File "...\electrum\electrum\gui\qt\request_list.py", line 101, in item_changed
    self.parent.show_receive_request(req)
  File "...\electrum\electrum\gui\qt\main_window.py", line 1281, in show_receive_request
    can_receive_lightning = self.wallet.lnworker and req.get_amount_sat() <= self.wallet.lnworker.num_sats_can_receive()
TypeError: '<=' not supported between instances of 'NoneType' and 'decimal.Decimal'
```
2022-05-19 19:43:07 +02:00
SomberNight
dd5cb2a5c1 lnworker: rework num_sats_can_receive and routing_hints_for_invoice
follow-up https://github.com/spesmilo/electrum/pull/7818

- note it matters whether a sender pays us end-to-end-trampoline or just via legacy
  - consider: Alice has 0.1 BTC recv cap in chan1 and 1 BTC recv cap in chan2, both with border-node T1
    - if sender is paying e2e trampoline, it can realistically pay even ~1.1 BTC, as T1 can resplit the HTLCs
    - if sender is paying legacy, it will have a hard time trying to pay more than 1 BTC, in practice
      - although note if T1 has implemented non-strict-forwarding (see BOLT-04), achieving 1 BTC is easy,
        as T1 can redirect HTLCs (but cannot split them, in this case)
  - to make num_sats_can_receive realistic, it assumes the legacy case
- To calc num_sats_can_receive, we sort our channels in decreasing order of receive-capacities, iterate over them
  and calculate a running sum - we stop adding channels when the next chan's recv cap is small compared to
  the running total.
- When putting routing hints in an invoice, we do the same, with the added condition that we keep adding channels
  if their recv cap is larger than the invoice amount.
  - consider: Alice has 0.1 BTC recv cap in chan1 with Bob, and 1 BTC recv cap in chan2 with Carol
    - if Alice wants to recv 100 sats, it is useful to add hints for both channels into the invoice, for redundancy
    - if Alice wants to recv 0.9 BTC, it is questionable whether adding the smaller chan is useful - the code here won't add it
2022-05-19 18:28:04 +02:00
ThomasV
f90a08bbe2 Filter nodes for receiving:
- increase MPP_RECEIVE_CUTOFF from 5 to 20 percent
 - filter channels by node_id, not channel_id
 - make num_sats_can_receive consistent with routing hints
2022-05-18 18:11:40 +02:00
ThomasV
0a6495c490 lnworker: if trampoline is active, make num_sats_can_send
return the max value allowed for legacy payments.
(that is, do not assume we can do MPP involving two trampolines)
This reverts fb6047ec46

Reason: num_sats_can_send may be used to decide the amount
we put in a new channel.
2022-05-18 09:48:22 +02:00
ThomasV
1274ec7655 Qt balance piechart: show lightning funds that are frozen 2022-05-10 20:07:30 +02:00
SomberNight
2c57c78ebe asyncio: stop using get_event_loop(). introduce ~singleton loop.
asyncio.get_event_loop() became deprecated in python3.10. (see https://github.com/python/cpython/issues/83710)
```
.../electrum/electrum/daemon.py:470: DeprecationWarning: There is no current event loop
  self.asyncio_loop = asyncio.get_event_loop()
.../electrum/electrum/network.py:276: DeprecationWarning: There is no current event loop
  self.asyncio_loop = asyncio.get_event_loop()
```
Also, according to that thread, "set_event_loop() [... is] not deprecated by oversight".
So, we stop using get_event_loop() and set_event_loop() in our own code.
Note that libraries we use (such as the stdlib for python <3.10), might call get_event_loop,
which then relies on us having called set_event_loop e.g. for the GUI thread. To work around
this, a custom event loop policy providing a get_event_loop implementation is used.

Previously, we have been using a single asyncio event loop, created with
util.create_and_start_event_loop, and code in many places got a reference to this loop
using asyncio.get_event_loop().
Now, we still use a single asyncio event loop, but it is now stored as a global in
util._asyncio_event_loop (access with util.get_asyncio_loop()).

I believe these changes also fix https://github.com/spesmilo/electrum/issues/5376
2022-04-29 18:49:07 +02:00
ThomasV
f4e902e907 LNWorker: give up payment after timeout, not number of attempts.
Limiting attempts may interrupt a MPP before we receive a MPP_timeout
The attempts parameter is still used in unit tests.
2022-04-29 12:17:38 +02:00
SomberNight
f10752b523 lnworker: LNWallet.start_network to call super().start_network
The call to super was removed in 4efcb53d24 ,
so that LNWallet's taskgroup would not run _maintain_connectivity, AFAICT.
The way it was done meant that "main_loop" itself would not run for LNWallet, only for LNGossip - very confusing.
It is only due to a quirk in the behaviour of TaskGroups that the group "started" at all.
2022-04-26 20:24:08 +02:00
ThomasV
2117118047 set_request_status: pass the right key to request_status callback. 2022-04-23 19:24:31 +02:00
ThomasV
8354dd006b fix receiving lightning requests without amount 2022-04-23 12:06:24 +02:00
SomberNight
5f7388a475 lnworker: fix get_onchain_history if running with --offline
```
Traceback (most recent call last):
  File "...\electrum\electrum\gui\qt\main_window.py", line 898, in timer_actions
    self.update_wallet()
  File "...\electrum\electrum\gui\qt\main_window.py", line 1040, in update_wallet
    self.update_tabs()
  File "...\electrum\electrum\gui\qt\main_window.py", line 1047, in update_tabs
    self.history_model.refresh('update_tabs')
  File "...\electrum\electrum\util.py", line 439, in <lambda>
    return lambda *args, **kw_args: do_profile(args, kw_args)
  File "...\electrum\electrum\util.py", line 435, in do_profile
    o = func(*args, **kw_args)
  File "...\electrum\electrum\gui\qt\history_list.py", line 275, in refresh
    transactions = wallet.get_full_history(
  File "...\electrum\electrum\util.py", line 439, in <lambda>
    return lambda *args, **kw_args: do_profile(args, kw_args)
  File "...\electrum\electrum\util.py", line 435, in do_profile
    o = func(*args, **kw_args)
  File "...\electrum\electrum\wallet.py", line 947, in get_full_history
    lnworker_history = self.lnworker.get_onchain_history() if self.lnworker and include_lightning else {}
  File "...\electrum\electrum\lnworker.py", line 911, in get_onchain_history
    tx_height = self.lnwatcher.get_tx_height(swap.funding_txid)
AttributeError: 'NoneType' object has no attribute 'get_tx_height'
```
2022-04-22 19:18:07 +02:00
ThomasV
fb6047ec46 lnworker: fix can_pay_invoice for trampoline MPP
Call it from the GUI
2022-04-22 15:13:12 +02:00
ThomasV
60865f3902 Show options if we do not have the liquidity to pay a lightning invoice:
pay onchain, open channel, rebalance.

If we do a swap or open a channel, the payment will be scheduled.
2022-04-20 12:48:22 +02:00
ThomasV
2c6e36e89d Fallback addresses
- add fallback address to BOLT-11 LN invoices
   - Qt: if LN payment fails, propose onchain fallback
2022-04-20 12:48:22 +02:00
ThomasV
7102fb732e follow-up prev:
- detect payment of requests both onchain or LN
 - create single type of requests in GUI
2022-04-20 12:48:22 +02:00
ThomasV
e392197ab9 wallet_db upgrade:
- unify lightning and onchain invoices, with optional fields for bip70 and lightning
 - add receive_address fields to submarine swaps
2022-04-20 12:48:22 +02:00
ThomasV
877a9f553b Merge pull request #7748 from spesmilo/confirm_reverse_swaps
Confirm reverse swaps
2022-04-09 10:41:02 +02:00
ThomasV
3e6595ae98 Merge pull request #7756 from SomberNight/202204_qt_taskthread_cancel
qt taskthread cleaner shutdown
2022-04-09 10:38:06 +02:00
SomberNight
96c063028a qt TaskThread: implement cancellation of tasks, for cleaner shutdown
fixes https://github.com/spesmilo/electrum/issues/7750

Each task we schedule on `TaskThread` can provide an optional `cancel` method.
When stopping `TaskThread`, we call this `cancel` method on all tasks in the queue.
If the currently running task does not implement `cancel`, `TaskThread.stop` will block
until that task finishes.

Note that there is a significant change in behaviour here:
`ElectrumWindow.run_coroutine_from_thread` and `ElectrumWindow.pay_lightning_invoice`
previously serialised the execution of their coroutines via wallet.thread.
This is no longer the case: they will now schedule coroutines immediately.
So for example, the GUI now allows trying to pay multiple LN invoices "concurrently".
2022-04-07 19:51:58 +02:00
ThomasV
ad41f4aed0 trampoline legacy mpp: link to eclair issue 2022-04-07 19:50:30 +02:00
ThomasV
f01197b6b5 Reverse swaps: Wait until funding tx is confirmed
- override with allow_instant_swaps option (Qt)
2022-04-01 14:05:01 +02:00
ThomasV
cb39bbbd94 lnworker: make calc_routing_hints_for_invoice and create_invoice non-async 2022-03-29 17:42:04 +02:00
ThomasV
a15dac2b8c channel_establishment_flow: do not save wallet file backup in the background.
Instead, display a popup everytime, if the channel is not recoverable.
2022-03-24 14:55:45 +01:00
SomberNight
556b98736e lnworker.try_force_closing: changed to not be async (and renamed)
This is to ensure that the channel is "immediately" set to FORCE_CLOSING.
(previously it took at least one event loop iteration)
2022-02-21 18:09:45 +01:00
ThomasV
b268877d53 Merge pull request #7636 from bitromortac/2201-channel-type
lightning: implement channel types
2022-02-21 12:08:54 +01:00
ThomasV
4ebe41b3a7 Trampoline MPP: save fee level in sent_htlcs_info.
If multiple HTLCs fail at the same fee level with
TRAMPOLINE_INSUFFICIENT_FEE, bump trampoline_fee_level only once.
2022-02-19 15:20:54 +01:00
ThomasV
9fd18ae7f4 Merge pull request #7623 from bitromortac/2201-multi-trampoline-mpp
Multi-trampoline multipart payments
2022-02-19 14:44:21 +01:00
ThomasV
b2f84187bc Split code in reestablish_channel:
Messages are sent in reestablish_channel (async)
  Message checks and force_close are performed in on_channel_reestablish (not async).
  That task should not be cancelled if the connection is closed.
  Revert 57583c05cf
2022-02-19 10:37:50 +01:00
SomberNight
96fcf68d84 lnworker.force_close_channel: set chan state before broadcast
It is not safe to keep using the channel after we attempted to broadcast a force-close,
even if the broadcast errored: the server cannot be trusted wrt to errors.

Note that if there is a network-error, due to the state-transition, the GUI won't offer
the force-close option to the user again. However, LNWallet will periodically rebroadcast
the tx automatically (in on_channel_update); and we also save the tx as local into the
wallet.
2022-02-18 17:20:30 +01:00
bitromortac
a4f5cfc91a trampoline: refactor routes, enable e2e mpp
* Refactor `create_trampoline_route`.
* Enables end-to-end multi-trampoline multipart payments.
  Trampoline-to-legacy payments are still not enabled, as this is
  currently not supported by Eclair.
* Reverts to a global trampoline fee level, as trampoline failures
  are currently not handled properly, see (#7648), which doubles
  fee rates.
2022-02-18 10:14:51 +01:00
ThomasV
57583c05cf request_force_close: add 1s delay before closing the tranport,
so that the remote task does not get cancelled.
2022-02-16 18:54:42 +01:00
SomberNight
c9c094cfab requirements: bump min aiorpcx to 0.22.0
aiorpcx 0.20 changed the behaviour/API of TaskGroups.
When used as a context manager, TaskGroups no longer propagate
exceptions raised by their tasks. Instead, the calling code has
to explicitly check the results of tasks and decide whether to re-raise
any exceptions.
This is a significant change, and so this commit introduces "OldTaskGroup",
which should behave as the TaskGroup class of old aiorpcx. All existing
usages of TaskGroup are replaced with OldTaskGroup.

closes https://github.com/spesmilo/electrum/issues/7446
2022-02-15 18:22:44 +01:00
SomberNight
c131831373 util: rm SilentTaskGroup. this does not seem to be needed anymore
I think this was originally needed due to incorrect management of group lifecycles,
which our current code is doing better.

also note that if we needed this, in newer aiorpcx, the name of
the field was ~changed from `_closed` to `joined`:
239002689a
2022-02-15 18:22:40 +01:00
SomberNight
3f3212e94d some clean-ups now that we require python 3.8
In particular, asyncio.CancelledError no longer inherits Exception (it inherits BaseException directly)
2022-02-15 18:22:36 +01:00
bitromortac
6915e3cb10 lnpeer+wallet: use channel type for channel open
* channel_type is put into storage, serialized as int and
  deserialized as ChannelType
* check for static_remotekey is done via channel type
2022-01-20 16:47:48 +01:00
ghost43
ce44a03c24 Merge pull request #7202 from bitromortac/2104-mpp-channel-splitting
MPP splitting algorithm: redesign and split within channels
2021-12-17 13:58:54 +00:00
SomberNight
0df05dd914 qt preferences: always show cb for LN/"Create recoverable channels"
This makes it explicit that the option cannot be enabled if so (instead of hiding the checkbox).
2021-11-17 17:54:52 +01:00
SomberNight
56b03e2e8d lnpeer: more forwarding is now event-driven
This should make unit tests less reliant on sleeps.
2021-11-04 19:16:02 +01:00
SomberNight
4f907e3889 lnworker: change api of 'htlc_{fulfilled,failed}' events 2021-11-04 16:41:23 +01:00
bitromortac
f2f8c4533b implement option_shutdown_anysegwit
https://github.com/lightningnetwork/lightning-rfc/pull/672

We check the received shutdown script against higher segwit versions and
accept closing to that script if option_shutdown_anysegwit has been
negotiated.
2021-10-26 14:51:09 +02:00
SomberNight
1ff9f9910f ln update_fee: enforce that feerate is over default min relay fee
(this was always already the case when we are the funder, but we were
not checking it when remote is responsible for update_fee)
2021-09-28 19:48:31 +02:00
bitromortac
4ee5fa75f6 lnrouter+lnworker: rename my_channels 2021-07-29 16:26:23 +02:00
bitromortac
68bc9c2474 lnworker: improve route creation
- Separates the trampoline and local routing multi-part payment cases.
- Ask only for splits that don't send over a single channel (those have
  been tried already in the single-part case).
- Makes sure that create_routes_for_payment only yields partial routes
  that belong to a single split configuration.
- Tracks trampoline fee levels on a per node basis, previously, in the
  case of having two channels with a trampoline forwarder, the global
  fee level would have increased by two levels upon first try.
2021-07-16 11:27:36 +02:00
bitromortac
3c521f4ce3 mpp_split: split algorithm with channel splits
- The splitting algorithm is redesigned to use random distribution of
  subsplittings over channels.
- Splittings can include multiple subamounts within a channel.
- The single-channel splittings are implicitly activated once the
  liquidity hints don't support payments of large size.
2021-07-16 11:22:38 +02:00
SomberNight
8481afb286 lnchannel: introduce HTLCWithStatus NamedTuple 2021-07-15 01:35:24 +02:00
SomberNight
6a049d9901 wallet: make sure payment requests are persisted
Fixes: after adding a payment request, if the process was killed,
the payreq might get lost. In case of using the GUI, neither the
callee nor the caller called wallet.save_db().

Unclear where wallet.save_db() should be called...
Now each method tries to persist their changes by default,
but as an optimisation, the caller can pass write_to_disk=False
e.g. when calling multiple such methods and then call wallet.save_db() itself.

If we had partial writes, which would either rm the need for wallet.save_db()
or at least make it cheaper, this code might get simpler...

related: https://github.com/spesmilo/electrum/pull/6435
related: https://github.com/spesmilo/electrum/issues/4823
2021-07-05 18:39:10 +02:00
SomberNight
a339338958 LN private route hints: don't include low receive capacity channels
see code comment.

While the balance in the channels might shift before the sender tries to
pay the invoice, as we are not a forwarding node, that seems unlikely to matter.
2021-07-02 19:52:36 +02:00
SomberNight
3a7f5373ac trampoline: improve payment success
- better error handling: previously we stopped all attempts on any of
  TRAMPOLINE_EXPIRY_TOO_SOON, UNKNOWN_NEXT_PEER, TEMPORARY_NODE_FAILURE.
  Instead we should retry (but see code comments).
- previously payments failed if ALL of the following criteria applied:
  - sender is paying via trampoline, but not via the ACINQ node (which is
    special cased)
  - receiver only has private channels and has put r_tags into invoice, along
    with setting the trampoline feature bit in the invoice, however the receiver
    is not connected to any trampoline forwarders directly
  The sender would then assume that the private routing hints in the invoice
  correspond to trampoline forwarders.
- also, previously if both the sender and the recipient used trampoline and
  they shared a trampoline forwarder (that they were both connected to), the
  private channels the recipient had (with nodes other than the shared TF)
  would never be attempted.
2021-07-02 18:44:39 +02:00
SomberNight
57e52da77f lnaddr: clean-up SEGWIT_HRP vs BOLT11_HRP confusion
With signet, SEGWIT_HRP != BOLT11_HRP, so the previous "currency" string
became a flawed concept. Instead we pass around net objects now.
2021-06-22 16:16:21 +02:00