Commit Graph

583 Commits

Author SHA1 Message Date
ThomasV
f90a08bbe2 Filter nodes for receiving:
- increase MPP_RECEIVE_CUTOFF from 5 to 20 percent
 - filter channels by node_id, not channel_id
 - make num_sats_can_receive consistent with routing hints
2022-05-18 18:11:40 +02:00
SomberNight
872ce82418 tests: clean up event-loop creation 2022-05-04 01:53:21 +02:00
SomberNight
2c57c78ebe asyncio: stop using get_event_loop(). introduce ~singleton loop.
asyncio.get_event_loop() became deprecated in python3.10. (see https://github.com/python/cpython/issues/83710)
```
.../electrum/electrum/daemon.py:470: DeprecationWarning: There is no current event loop
  self.asyncio_loop = asyncio.get_event_loop()
.../electrum/electrum/network.py:276: DeprecationWarning: There is no current event loop
  self.asyncio_loop = asyncio.get_event_loop()
```
Also, according to that thread, "set_event_loop() [... is] not deprecated by oversight".
So, we stop using get_event_loop() and set_event_loop() in our own code.
Note that libraries we use (such as the stdlib for python <3.10), might call get_event_loop,
which then relies on us having called set_event_loop e.g. for the GUI thread. To work around
this, a custom event loop policy providing a get_event_loop implementation is used.

Previously, we have been using a single asyncio event loop, created with
util.create_and_start_event_loop, and code in many places got a reference to this loop
using asyncio.get_event_loop().
Now, we still use a single asyncio event loop, but it is now stored as a global in
util._asyncio_event_loop (access with util.get_asyncio_loop()).

I believe these changes also fix https://github.com/spesmilo/electrum/issues/5376
2022-04-29 18:49:07 +02:00
ThomasV
f4e902e907 LNWorker: give up payment after timeout, not number of attempts.
Limiting attempts may interrupt a MPP before we receive a MPP_timeout
The attempts parameter is still used in unit tests.
2022-04-29 12:17:38 +02:00
ThomasV
149cccbc4a fix tests (follow-up 2117118047) 2022-04-23 19:40:21 +02:00
ThomasV
60865f3902 Show options if we do not have the liquidity to pay a lightning invoice:
pay onchain, open channel, rebalance.

If we do a swap or open a channel, the payment will be scheduled.
2022-04-20 12:48:22 +02:00
ThomasV
bf4455ef30 follow-up db invoice unification:
- fix command line
 - deprecate add_lightning_request
2022-04-20 12:48:22 +02:00
ThomasV
cb39bbbd94 lnworker: make calc_routing_hints_for_invoice and create_invoice non-async 2022-03-29 17:42:04 +02:00
SomberNight
5149ee02a5 wallet: fix delete_address removing too many transactions
See testcase:
- imported wallet with addr1 and addr2
- three txs: tx1 funds addr1, tx2 funds addr2, tx3 spends all
- if we rm addr1 from the wallet,
  - previously both tx1 and tx3 was removed (as tx3 is a child of tx1)
  - now only tx1 is removed (tx3 still relates to the wallet via addr2)

fixes https://github.com/spesmilo/electrum/issues/7587
2022-03-21 19:30:08 +01:00
bitromortac
3915045067 lnpeer: warnings for shutdown and open_channel 2022-03-09 13:40:44 +01:00
bitromortac
9e800172ec lnpeer: send/handle error and warning messages
* adds methods for sending protocol errors/warnings
* handling of warning messages
2022-03-09 13:40:44 +01:00
ThomasV
6667a79f10 modern shutdown:
- clarify TODOs
 - add tests for shutdown with modern negotiation
2022-03-08 11:57:19 +01:00
ThomasV
0b203f0b94 lnpeer: refactor fee negotiation in _shutdown
- the fee negotiation is split into smaller functions, reducing the scope of variables.
  - the while loop logic is condensed in a few lines, so it is easier to understand termination conditions.
  - removed code that was never executed
2022-03-08 11:55:40 +01:00
bitromortac
ec740d45f1 lnpeer: modern fee negotiation
Updates the closing fee negotiation to comply with most recent spec
changes, see https://github.com/lightning/bolts/pull/847
The closing negotiation is backwards compatible with the old
negotiation.
2022-03-07 10:36:20 +01:00
Marko Bencun
1d652a2a04 tests: unit test address_to_payload 2022-03-02 14:54:11 +01:00
ghost43
6603359c20 Merge pull request #7668 from SomberNight/202202_signmessage_segwit_trezor
signmessage: also accept Trezor-type sigs for p2wpkh and p2wpkh-p2sh addrs
2022-02-22 17:10:08 +00:00
SomberNight
556b98736e lnworker.try_force_closing: changed to not be async (and renamed)
This is to ensure that the channel is "immediately" set to FORCE_CLOSING.
(previously it took at least one event loop iteration)
2022-02-21 18:09:45 +01:00
ThomasV
b268877d53 Merge pull request #7636 from bitromortac/2201-channel-type
lightning: implement channel types
2022-02-21 12:08:54 +01:00
ThomasV
4ebe41b3a7 Trampoline MPP: save fee level in sent_htlcs_info.
If multiple HTLCs fail at the same fee level with
TRAMPOLINE_INSUFFICIENT_FEE, bump trampoline_fee_level only once.
2022-02-19 15:20:54 +01:00
ThomasV
9fd18ae7f4 Merge pull request #7623 from bitromortac/2201-multi-trampoline-mpp
Multi-trampoline multipart payments
2022-02-19 14:44:21 +01:00
ThomasV
b2f84187bc Split code in reestablish_channel:
Messages are sent in reestablish_channel (async)
  Message checks and force_close are performed in on_channel_reestablish (not async).
  That task should not be cancelled if the connection is closed.
  Revert 57583c05cf
2022-02-19 10:37:50 +01:00
SomberNight
2f549e84db fix tests: follow-up 96fcf68d84 2022-02-18 19:32:27 +01:00
bitromortac
a4f5cfc91a trampoline: refactor routes, enable e2e mpp
* Refactor `create_trampoline_route`.
* Enables end-to-end multi-trampoline multipart payments.
  Trampoline-to-legacy payments are still not enabled, as this is
  currently not supported by Eclair.
* Reverts to a global trampoline fee level, as trampoline failures
  are currently not handled properly, see (#7648), which doubles
  fee rates.
2022-02-18 10:14:51 +01:00
SomberNight
5cdb4471ec signmessage: also accept Trezor-type sigs for p2wpkh and p2wpkh-p2sh addrs
The signatures we create are unchanged but we now also accept signatures created by Trezor and others.

see https://github.com/spesmilo/electrum/issues/3861
2022-02-16 19:25:46 +01:00
SomberNight
9bbf92b80b tests: add test for signmessage with segwit address 2022-02-16 19:24:43 +01:00
SomberNight
4f9e4c520f ecc: API changes: verify_message_hash to return bool instead of raising
verify_message_hash and verify_message_for_address now return bool
instead of raising Exceptions on bad signatures.
2022-02-16 19:24:38 +01:00
SomberNight
40c1597c0a lntransport: change name used in logs to make collisions unlikely
In particular, in the regtests, with incoming peers, we can have multiple transports open with the same node simultaneously
(see e.g. lnworker._request_force_close_from_backup).
We now use the first few bytes of peer_pubkey, as that is potentially familiar to users,
and the first few bytes of sha256(id(self)) to mitigate collisions in case the peer_pubkeys collide.

log excerpt:
```
I/P | lnpeer.Peer.[LNWallet, 030f0bf260-e0b33756] | handshake done for 030f0bf260acdbd3edcad84d7588ec7c5df4711e87e6a23016f989b8d3a4147230@163.172.94.64:9735
D/P | lnpeer.Peer.[LNWallet, 030f0bf260-e0b33756] | Sending INIT
I/P | lnpeer.Peer.[LNWallet, 03933884aa-5e5dce45] | handshake done for 03933884aaf1d6b108397e5efe5c86bcf2d8ca8d2f700eda99db9214fc2712b134@34.250.234.192:9735
D/P | lnpeer.Peer.[LNWallet, 03933884aa-5e5dce45] | Sending INIT
D/P | lnpeer.Peer.[LNWallet, 030f0bf260-e0b33756] | Received INIT
I/P | lnpeer.Peer.[LNWallet, 02651acf4a-79696c42] | handshake done for 02651acf4a7096091bf42baad19b3643ea318d6979f6dcc16ebaec43d5b0f4baf2@82.119.233.36:19735
D/P | lnpeer.Peer.[LNWallet, 02651acf4a-79696c42] | Sending INIT
D/P | lnpeer.Peer.[LNWallet, 03933884aa-5e5dce45] | Received INIT
I/P | lnpeer.Peer.[LNWallet, 030f0bf260-e0b33756] | saved remote_update
D/P | lnpeer.Peer.[LNWallet, 030f0bf260-e0b33756] | Received CHANNEL_REESTABLISH
```
2022-02-16 18:53:24 +01:00
SomberNight
c9c094cfab requirements: bump min aiorpcx to 0.22.0
aiorpcx 0.20 changed the behaviour/API of TaskGroups.
When used as a context manager, TaskGroups no longer propagate
exceptions raised by their tasks. Instead, the calling code has
to explicitly check the results of tasks and decide whether to re-raise
any exceptions.
This is a significant change, and so this commit introduces "OldTaskGroup",
which should behave as the TaskGroup class of old aiorpcx. All existing
usages of TaskGroup are replaced with OldTaskGroup.

closes https://github.com/spesmilo/electrum/issues/7446
2022-02-15 18:22:44 +01:00
bitromortac
6915e3cb10 lnpeer+wallet: use channel type for channel open
* channel_type is put into storage, serialized as int and
  deserialized as ChannelType
* check for static_remotekey is done via channel type
2022-01-20 16:47:48 +01:00
bitromortac
401a429080 lnutil+lnwire: implement ChannelType 2022-01-20 16:47:48 +01:00
bitromortac
2d2b889312 lnpeer tests: add spp trampoline payment 2021-12-20 16:45:14 +01:00
bitromortac
e8c94cf5d7 tests: implement graph with flexible definitions 2021-12-20 16:45:07 +01:00
SomberNight
7354feeffe follow-up fix tests: logic typo
follow-up https://github.com/spesmilo/electrum/pull/7202

defaultdict[int] is a type!

```
>>> from collections import defaultdict
>>> d = defaultdict[int]
>>> d[2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: There are no type variables left in collections.defaultdict[int]
```

Also, prior to py3.9, it is a TypeError.
2021-12-17 15:21:21 +01:00
ghost43
c8f47ed4fe Merge pull request #7562 from SomberNight/202111_blockchain_target
blockchain.py: bugfixes re `bits_to_target` and `target_to_bits`
2021-12-17 14:07:55 +00:00
ghost43
ce44a03c24 Merge pull request #7202 from bitromortac/2104-mpp-channel-splitting
MPP splitting algorithm: redesign and split within channels
2021-12-17 13:58:54 +00:00
SomberNight
7e2d9c48d1 blockchain: fix bugs in bits_to_target and target_to_bits
This fixes three bugs:
- too large targets: the fixme in target_to_bits, which meant that we could
  not handle targets where the first bit was non-zero. This however cannot
  happen due to these being over MAX_TARGET. (difficulty 1)
- too small targets: in bits_to_target, very small targets were not handled well:
  ```
  >>> Blockchain.bits_to_target(0x03008000)
  32768
  ```
  We could not process headers with targets smaller than the above value.
  (note that these small targets would only occur at astronomically high mining difficulty)
- non-canonically encoded targets:
  we would not accept headers that had targets encoded in compact form (nBits) in a non-canonical way.
  I think this bug could actually be triggered by mining such a header.
  E.g. consider the target "1167130406913723784571467005534932607254396928"
  ```
  Blockchain.target_to_bits(1167130406913723784571467005534932607254396928).to_bytes(4, "big").hex()
  '13345600'
  ```
  Bitcoin Core when used to e.g. mine a block would encode this target as 0x13345600 in compact form.
  However, AFAICT, when validating Bitcoin Core would equally accept 0x14003456 which decodes to the
  same target. We were however rejecting such non-canonical compact nBits.
  ```
  >>> from electrum.blockchain import Blockchain
  >>> Blockchain.bits_to_target(0x14003456)
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/home/user/wspace/electrum/electrum/blockchain.py", line 548, in bits_to_target
      raise Exception("Second part of bits should be in [0x8000, 0x7fffff]")
  Exception: Second part of bits should be in [0x8000, 0x7fffff]
  >>> Blockchain.bits_to_target(0x13345600)
  1167130406913723784571467005534932607254396928
  ```
2021-11-13 04:31:08 +01:00
ghost43
a5c4b9e719 Merge pull request #7545 from yanmaani/unix_sockets
Add support for Unix domain sockets
2021-11-04 19:32:33 +01:00
SomberNight
56b03e2e8d lnpeer: more forwarding is now event-driven
This should make unit tests less reliant on sleeps.
2021-11-04 19:16:02 +01:00
SomberNight
12f3525df0 lnpeer: disable msg processing rate-limiting in tests 2021-11-04 18:04:16 +01:00
yanmaani
82b9cd12eb tests: Add rudimentary integration test for Unix domain socket functionality 2021-11-03 12:00:00 +00:00
ghost43
5787f3ab74 Merge pull request #7542 from bitromortac/2109-dust-limit
Implement recent spec changes regarding collab channel close outputs
2021-10-27 16:51:07 +02:00
SomberNight
cb55a2d654 tests: try to reduce flakyness of test_fail_pending_htlcs_on_shutdown
Alice sends and HTLC: Alice->Carol->Dave
we need a lot of messages back and forth to happen:
- Alice adds HTLC to chan_AC, sends sig, Carol revacks, sends sig, Alice revacks;
- only then Carol adds HTLC to chan_CD, sends sig, Dave revacks, sends sig, Carol revacks
on CI, 0.5 seconds is often not enough for this it seems.
2021-10-27 16:46:15 +02:00
bitromortac
f2f8c4533b implement option_shutdown_anysegwit
https://github.com/lightningnetwork/lightning-rfc/pull/672

We check the received shutdown script against higher segwit versions and
accept closing to that script if option_shutdown_anysegwit has been
negotiated.
2021-10-26 14:51:09 +02:00
SomberNight
1ff9f9910f ln update_fee: enforce that feerate is over default min relay fee
(this was always already the case when we are the funder, but we were
not checking it when remote is responsible for update_fee)
2021-09-28 19:48:31 +02:00
ghost43
437a9e4358 Merge pull request #7505 from bitromortac/2109-activate-watchtower
watchtower: continuously check for added channels
2021-09-28 16:46:19 +02:00
bitromortac
ff61020dd2 watchtower: watch new channels 2021-09-27 10:31:44 +02:00
ThomasV
baff4fa625 save_backup: do not remove deterministic LN key (see #7513) 2021-09-26 12:18:25 +02:00
ThomasV
7a0904c0f4 wallet update: move fields that have string keys out of channel log 2021-09-20 14:47:20 +02:00
SomberNight
85780eea1a multiple max spends: add unit test
re https://github.com/spesmilo/electrum/pull/7492
2021-09-15 16:42:32 +02:00
JeremyRand
b1c4bb8914 Add Cirrus CI (#7431)
* Cirrus: Add Tox task

* Cirrus: Add Locale task

* Cirrus: Add Flake8 Task

* Cirrus: Add Regtest task

* Regtest: Flush stdout

Allows viewing output sooner.

* Regtest: Read process.stdout in text mode

Improves ability to quickly see output.

* Cirrus: Add Windows task

* Cirrus: Add Android task

* Cirrus: Add macOS task

* Cirrus: Add AppImage task

* Cirrus: Add tarball task

* Cirrus: Add Submodules task

* Android: remove superfluous cp/rm

* Add .dockerignore

Symlink to .gitignore.
2021-09-13 16:20:54 +00:00