Commit Graph

724 Commits

Author SHA1 Message Date
SomberNight
4c63d8729b add sanity checks we don't sign tx including dummy addr
Somewhat a follow-up to 649ce979ab.

This adds some safety belts so we don't accidentally sign a tx that
contains a dummy address.
Specifically we check that tx does not contain output for dummy addr:
- in wallet.sign_transaction
- in network.broadcast_transaction

The second one is perhaps redundant, but I think it does not hurt.
2023-09-16 04:36:08 +00:00
SomberNight
139eb632d7 (trivial) follow-up paysession.use_two_trampolines arg change
follow-up b2053c68f1

- use_two_trampolines is already passed as arg to init
- strings in gui/messages.py should not end with a newline
2023-09-12 14:05:12 +00:00
SomberNight
6819f685d7 tests: add tests for "recv mpp confusion" bug
see https://github.com/spesmilo/electrum/security/advisories/GHSA-8r85-vp7r-hjxf
2023-09-12 13:58:57 +00:00
ThomasV
f245b347f1 Merge pull request #8609 from SomberNight/202309_getconfig_default_value
getconfig/setconfig to use configvars
2023-09-08 19:06:25 +02:00
SomberNight
8c9fec4ab8 commands: getconfig to use default values, add existence checks
- getconfig and setconfig now both check configvars for existence
- getconfig returns default values when applicable
- setconfig does not side-step type-checks for values

fixes https://github.com/spesmilo/electrum/issues/8607
closes https://github.com/spesmilo/electrum/pull/8608
2023-09-08 15:11:55 +00:00
ThomasV
136978e9d0 submarine swaps: fail received HTLCs of normal swap htlcs if
the swap is still unfunded and the refund delay has expired.
2023-09-08 16:38:08 +02:00
SomberNight
6a2806c2c5 simple_config: implement complex default values
Besides a literal value, the default can now also be a callable,
which gets called with the config and evaluated as needed, lazily.

This potentially allows e.g. the default value of one configvar to
depend on the current value of another configvar.
2023-09-07 15:30:46 +00:00
SomberNight
f29cd810e3 commands: "bumpfee" to accept either a raw tx or a txid as its 1st arg
closes https://github.com/spesmilo/electrum/issues/8603
2023-09-07 13:18:23 +00:00
SomberNight
c3562c00e6 tests: fix warnings in test_network.py
```
electrum/tests/test_network.py::TestNetwork::test_can_connect_during_backward
electrum/tests/test_network.py::TestNetwork::test_chain_false_during_binary
electrum/tests/test_network.py::TestNetwork::test_fork_conflict
electrum/tests/test_network.py::TestNetwork::test_fork_noconflict
  /tmp/cirrus-ci-build/electrum/interface.py:410: RuntimeWarning: coroutine 'Interface.run' was never awaited
    task = await self.network.taskgroup.spawn(self.run())
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
```

closes https://github.com/spesmilo/electrum/pull/7817
2023-09-06 15:53:40 +00:00
SomberNight
dd3966070d tests: rm some redundant "electrum_path" temp dirs 2023-09-06 14:12:30 +00:00
SomberNight
4f90632b1d tests: fix warnings in test_storage_upgrade.py
Manage global event loop so that it is accessible to threads
other than the asyncio thread. In particular, the Plugins
thread needs to reference it in its on_stop().
2023-09-06 14:12:26 +00:00
SomberNight
00ae370a45 tests: split out wallets from test_storage_upgrade.py
so that Thomas' emacs does not crash when editing the file :D
2023-08-30 18:40:41 +00:00
ghost43
e8c0767ca7 Merge pull request #8584 from SomberNight/202308_fix8582
invoices: also run amount-validator on setter
2023-08-30 16:27:00 +00:00
ThomasV
b2053c68f1 make use_to_trampolines a user visible option, disable it by default 2023-08-27 11:43:17 +02:00
SomberNight
392f6d8e30 fix tests: another follow-up to daemon managing Plugins object
In python 3.8 and 3.9, asyncio.Event/Lock/etc cannot be created before the
event loop itself is created. Hence, to have Plugins.__init__ create an
Event, we need to postpone creating Plugins() from setUpClass to setUp.

follow-up 90f39bce88
2023-08-24 18:21:23 +00:00
SomberNight
ffa3acc013 invoices: don't modify .amount_msat directly 2023-08-22 18:12:15 +00:00
SomberNight
4e6e6f76ca invoices: also run amount-validator on setter
- @amount_msat.validator prevents the creation of invoices with e.g. too large amounts
- however the qml gui is mutating invoices by directly setting the `amount_msat` field,
  and it looks like attrs validators only run during init.
  We can use `on_setattr` (introduced in attrs==20.1.0).
- a wallet db upgrade is added to rm existing insane invoices
- btw the qml gui was already doing its own input validation on the textedit
  (see qeconfig.btcAmountRegex). however that only limits the input to not have more
  chars than what is needed to represent 21M BTC (e.g. you can still enter 99M BTC,
  which the invoice logic does not tolerate later on - but is normally caught).

fixes https://github.com/spesmilo/electrum/issues/8582
2023-08-22 18:10:21 +00:00
ThomasV
fc183e5143 Fix regtests (follow-up 0ebb6469ff)
Note: We should probably require the password in load_wallet,
and store it in memory as long as the wallet is loaded. In that
case, commands that currently require a password would no longer
require it if the wallet is loaded.
2023-08-18 08:52:02 +02:00
ThomasV
b96cc82333 Make storage a field of db
This comes from the jsonpatch_new branch.
I rather have in master now, because it touches a lot of filese.
2023-08-18 08:08:31 +02:00
SomberNight
98bda60c01 lnworker: move sent_buckets into PaySession 2023-08-11 22:01:19 +00:00
SomberNight
00e88c4e50 lnworker: introduce PaySession cls, refactor pay_to_node 2023-08-11 22:01:16 +00:00
SomberNight
98bea49a3c lnworker.pay_to_node: make trampoline fee_level and failed_routes local
multiple instances of pay_to_node might run concurrently, esp for trampoline forwarding
2023-08-11 22:01:11 +00:00
SomberNight
35c9ac8f31 lnworker: MPP send: more aggressively split large htlcs
related: https://github.com/spesmilo/electrum/issues/7987#issuecomment-1670002482
2023-08-11 22:01:08 +00:00
ThomasV
9f5f802cd1 config: save ports instead of net addresses (follow-up 012ce1c1bb) 2023-08-11 08:12:54 +02:00
ThomasV
bf86cd6761 lnpeer and lnworker cleanup:
- rename trampoline_forwardings -> final_onion_forwardings,
   because this dict is used for both trampoline and hold invoices
 - remove timeout from hold_invoice_callbacks (redundant with invoice)
 - add test_failure boolean parameter to TestPeer._test_simple_payment,
   in order to test correct propagation of OnionRoutingFailures.
 - maybe_fulfill_htlc: raise an OnionRoutingFailure if we do not have
   the preimage for a payment that does not have a hold invoice callback.
   Without this, the above unit tests stall when we use test_failure=True
2023-08-09 13:23:26 +02:00
SomberNight
afac158c80 lnworker: clean-up sent_htlcs_q and sent_htlcs_info
- introduce SentHtlcInfo named tuple
  - some previously unnamed tuples are now much shorter:
    create_routes_for_payment no longer returns an 8-tuple!
- sent_htlcs_q (renamed from sent_htlcs), is now keyed on payment_hash+payment_secret
  (needed for proper trampoline forwarding)
2023-08-08 16:37:50 +00:00
SomberNight
44bdd20ccc lnworker: add RecvMPPResolution with "FAILED" state
- add RecvMPPResolution enum for possible states of a pending incoming MPP,
  and use it in check_mpp_status
  - new state: "FAILED", to allow nicely failing back the whole MPP set
- key more things with payment_hash+payment_secret, for consistency
  (just payment_hash is insufficient for trampoline forwarding)
2023-08-08 16:37:46 +00:00
SomberNight
c527ef8967 lnpeer: refuse to forward htlcs that correspond to payreq we created 2023-08-07 18:57:04 +00:00
SomberNight
d51f00e2a3 asyncio.wait_for() is too buggy. use util.wait_for2() instead
wasted some time because asyncio.wait_for() was suppressing cancellations. [0][1][2]
deja vu... [3]

Looks like this is finally getting fixed in cpython 3.12 [4]
So far away...
In attempt to avoid encountering this again, let's try using
asyncio.timeout in 3.11, which is how upstream reimplemented wait_for in 3.12 [4], and
aiorpcx.timeout_after in 3.8-3.10.

[0] https://github.com/python/cpython/issues/86296
[1] https://bugs.python.org/issue42130
[2] https://bugs.python.org/issue45098
[3] https://github.com/kyuupichan/aiorpcX/issues/44
[4] https://github.com/python/cpython/pull/98518
2023-08-04 18:18:21 +00:00
SomberNight
d9d281338c tests: fix logic bug in some regtests
follow-up https://github.com/spesmilo/electrum/pull/8489
2023-08-03 11:40:01 +00:00
ThomasV
a6a2679d8a Merge pull request #8536 from SomberNight/202307_ln_imported_cb
fix sweeping chan after local force-close using cb
2023-07-28 11:16:48 +02:00
ThomasV
1b14692f30 swapserver: cleanup, add description 2023-07-26 19:20:18 +02:00
ThomasV
1411b75584 swapserver: add test for refund path 2023-07-26 19:20:18 +02:00
ThomasV
098c65d732 submarine swap server plugin:
- hold invoices
 - uses the same web API as the Boltz backend
2023-07-26 19:20:18 +02:00
ThomasV
141cd524bc lnpeer: do not run maybe_fulfill_htlc more than once, if it
triggered a payment forwarding.

Final onions may trigger a payment forwarding, through the callback
returned by maybe_fulfill_htlc. In that case, we should not fail the
HTLC later; doing so might result in fund loss.

Remove test_simple_payment_with_hold_invoice_timing_out: once we
have accepted to forward a payment HTLC with a hold invoice, we
do not want to time it out, for the same reason.
2023-07-24 12:44:56 +02:00
ThomasV
017186d107 Refactor trampoline forwarding and hold invoices.
- maybe_fulfill_htlc returns a forwarding callback that
   covers both cases.
 - previously, the callback of hold invoices was called as a
   side-effect of lnworker.check_mpp_status.
 - the same data structures (lnworker.trampoline_forwardings,
   lnworker.trampoline_forwarding_errors) are used for both
   trampoline forwardings and hold invoices.
 - maybe_fulfill_htlc still recursively calls itself to perform
   checks on trampoline onion. This is ugly, but ugliness is now
   contained to that method.
2023-07-21 13:40:10 +02:00
ThomasV
e124ff7ee7 Trampoline MPP consolidation:
- fix parameters passed to maybe_forward_trampoline
 - use lnworker.trampoline_forwardings as a semaphore for ongoing
   trampoline payments
 - if a trampoline payment fails, fail all received HTLCs
2023-07-19 10:48:44 +02:00
ThomasV
aeaf9c71df Add unit test for trampoline MPP consolidation
This tests that a trampoline waits until all incoming HTLCs are
received, and fail or succeed them together
2023-07-19 09:49:05 +02:00
ThomasV
1e3b3b82d5 test_lnpeer: deepcopy graph definitions in test setup 2023-07-19 09:43:18 +02:00
SomberNight
1767d26de9 tests: make regtest tests somewhat faster by faster polling in e-x
using 4e66804dc0

on my machine, before-after:
Ran 9 tests in 495.865s
Ran 9 tests in 376.183s
2023-07-14 15:18:36 +00:00
SomberNight
1a46460d11 fix sweeping chan after local force-close using cb
scenario:
- user opens a lightning channel and exports an "imported channel backup"
- user closes channel via local-force-close
  - local ctx is published, to_local output has user's funds and they are CSV-locked for days
- user restores wallet file from seed and imports channel backup
- new wallet file should be able to sweep coins from to_local output (after CSV expires)

This was not working previously, as the local_payment_basepoint was not included in the
imported channel backups, and the code was interpreting the lack of this as the channel not
having option_static_remotekey enabled. This resulted in lnutil.extract_ctn_from_tx
using an incorrect funder_payment_basepoint, and lnsweep not recognising the ctx due to
the garbage ctn value.

The imported channel backup serialisation format is slightly changed to include the
previously missing field, and its version number is bumped (0->1). We allow importing
both version 0 and version 1 backups, however v0 backups cannot handle the above
described scenario (they can only be used to request a remote-force-close).

Note that we were/are setting the missing local_payment_basepoint to the pubkey of
one of the wallet change addresses, which is bruteforceable if necessary, but I
think it is not worth the complexity to add this bruteforce logic. Also note
that the bruteforcing could only be done after the local-force-close was broadcast.

Ideally people with existing channels and already exported v0 backups should re-export
v1 backups... Not sure how to handle this.

closes https://github.com/spesmilo/electrum/issues/8516
2023-07-14 14:29:21 +00:00
SomberNight
b04ade5d7d tests: add failing test for sweeping chan after local fclose using cb 2023-07-14 13:57:02 +00:00
ThomasV
5cf320b2c7 Merge pull request #8462 from spesmilo/payment_identifiers2
Payment identifiers
2023-07-09 10:02:14 +02:00
Sander van Grieken
f980bd97b5 payment_identifier: factor out bip21 functions to bip21.py to break cyclic dependencies,
parse bolt11 only once, store invoice internally instead of bolt11 string
add is_onchain method to indicate if payment identifier can be paid onchain
2023-07-08 12:18:37 +02:00
SomberNight
6b43eac6fd lnonion: rm support for legacy (pre-TLV) onions
see https://github.com/lightning/bolts/pull/962
2023-06-29 14:34:06 +00:00
SomberNight
fc6486ecdb lnaddr: make payment_secret field mandatory, in both directions
we now require payment_secret both for sending and for receiving
(previously was optional for both)

see
https://github.com/lightning/bolts/pull/898
https://github.com/ACINQ/eclair/pull/1810
https://github.com/ElementsProject/lightning/pull/4646

note: payment_secret depends on var_onion_optin, so that becomes mandatory as well,
however this commit does not yet remove the ability of creating legacy onions
2023-06-29 14:34:02 +00:00
SomberNight
a66b0c6a12 lnaddr: rm some tests where feature bits do not make sense
Not all feature flags are supposed to go into the invoice.
2023-06-29 13:55:15 +00:00
Sander van Grieken
fc141c0182 payment_identfier: refactor qml and tests 2023-06-28 16:49:28 +02:00
ThomasV
7caa6ccf57 test_lnpeer: fix variable names after rename 2023-06-28 13:22:42 +02:00
ThomasV
c4eb7d8321 lnworker: bundled payments
- htlcs of bundled payments must arrive in the same MPP_TIMEOUT
window, or they will be failed
- add correspoding tests
2023-06-28 13:01:37 +02:00