Commit Graph

122 Commits

Author SHA1 Message Date
f321x 246f03fe20 allow all plugins to be either zip or directory based 2025-03-17 16:27:33 +01:00
ThomasV 81d4e90f66 Merge branch 'master' into 140325-force-close-exception 2025-03-14 19:38:13 +01:00
SomberNight 977d8b1dd6 wallet: kill create_transaction 2025-03-14 17:19:41 +00:00
f321x 5edbf923cd fix sweeping anchor outputs with multiple change addresses option enabled, don't consider tx inputs sufficient value if there are no outputs so change outpu gets added 2025-03-14 18:15:04 +01:00
SomberNight 3c3778db9c wallet: towards killing create_transaction: rm "sign" arg 2025-03-14 16:44:46 +00:00
ThomasV fbebe7de1a Make lnwatcher not async
This fixes offline history not having the proper labels
2025-03-14 11:09:11 +01:00
SomberNight 8ccd31fe49 wallet: set_frozen_state_of_coins to handle freeze=None
Internally whether a coin is frozen is tri-state:
- forced-True, set by the user
- forced-False, set by the user
- unset/default: is_frozen_coin() can decide whether the coin should be frozen

This patch lets set_frozen_state_of_coins() undo a previous explicit setting of True/False,
by calling it with a value of None.
Note: there is still no way in the GUI to undo an explicit setting of True/False.
2025-03-13 18:32:58 +00:00
ThomasV bdb7a82220 batch payment manager:
The class TxBatcher handles the creation, broadcast and replacement
of replaceable transactions. Callers (LNWatcher, SwapManager) use
methods add_payment_output and add_sweep_info. Transactions
created by TxBatcher may combine sweeps and outgoing payments.

Transactions created by TxBatcher will have their fee bumped
automatically (this was only the case for sweeps before).

TxBatcher manages several TxBatches. TxBatches are created
dynamically when needed.

The GUI does not touch txbatcher transactions:
  - wallet.get_candidates_for_batching excludes txbatcher
    transactions
  - RBF dialogs do not work with txbatcher transactions

wallet:
  - instead of reading config variables, make_unsigned_transaction
    takes new parameters: base_tx, send_change_to_lighting

tests:
  - unit tests in test_txbatcher.py (replaces test_sswaps.py)
  - force all regtests to use MPP, so that we sweep transactions
    with several HTLCs. This forces the payment manager to aggregate
    first-stage HTLC tx inputs. second-stage are not batched for now.
2025-03-13 10:17:10 +01:00
ThomasV 798df671ea If we have proposed htlcs in a channel that was force-closed,
call lnworker.htlc_failed once the htlc_tx is deeply mined.

In the case of a forwarding, this will fail incoming htlcs.
(fixes #8547)
2025-03-12 20:11:11 +01:00
SomberNight 0d120145d5 tests/regtest.py: trivial clean-up
from https://github.com/spesmilo/electrum/commit/dde7fd09ac6ae82d8fe746e1a4dc29fad2b34111
2025-03-10 14:54:50 +00:00
ThomasV 707d87a0d4 Revert "regtests: in py ctx, refer to configvars directly, not as str literals"
This reverts commit dde7fd09ac.
2025-03-10 12:01:41 +01:00
Sander van Grieken 6331ac0f85 lnutil: imports, whitespace, remove unused code 2025-03-09 12:30:02 +01:00
SomberNight dde7fd09ac regtests: in py ctx, refer to configvars directly, not as str literals
though in the bash script, we cannot do this
2025-03-07 17:26:12 +00:00
SomberNight a7c634b1ab regtests: extend "extract_preimage" to cover both types of extracts 2025-03-07 17:06:55 +00:00
SomberNight c146321924 regtests: e-x: update COIN name
ref https://github.com/spesmilo/electrumx/commit/ca59151d7365aabd2394fb39aac1faa25949b49d
2025-03-07 15:40:12 +00:00
SomberNight 6ea6c80f21 regtests: split up long line on e-x invocation 2025-03-07 15:40:05 +00:00
ThomasV 8011ae00ab Merge pull request #9586 from f321x/jit-block-preimage
Add mechanism to block htlcs from settling back
2025-03-07 12:04:39 +01:00
f321x 109ad2b3da add mechanism to block htlcs from settling back 2025-03-07 12:03:59 +01:00
ghost43 32b0e62cdc Merge pull request #9608 from SomberNight/202503_asyncio_gc
asyncio: hold our own strong refs for tasks and futures
2025-03-06 16:07:55 +00:00
ThomasV 074ede8f12 make_unsigned_transaction: pass merge_duplicate_outputs as
a parameter, because it must not be used in some contexts
(e.g. tx batcher).

This makes config.MERGE_DUPLICATE_OUTPUTS a GUI option
2025-03-06 09:22:12 +01:00
SomberNight 0b3a283586 asyncio: hold our own strong refs for tasks and futures
see https://docs.python.org/3.13/library/asyncio-task.html#asyncio.create_task :

> Important
>
> Save a reference to the result of this function, to avoid a task
> disappearing mid-execution. The event loop only keeps weak references
> to tasks. A task that isn’t referenced elsewhere may get garbage
> collected at any time, even before it’s done. For reliable
> “fire-and-forget” background tasks, gather them in a collection

ref https://github.com/python/cpython/issues/91887
ref https://github.com/beeware/toga/pull/2814
2025-03-05 17:06:04 +00:00
SomberNight 2aa427b4bf Merge branch 'pr/9507': qt: refactor NetworkChoiceLayout to ProxyWidget+ServerWidget
ref https://github.com/spesmilo/electrum/pull/9507
2025-03-05 15:01:05 +00:00
ThomasV ab14c3e138 tx batching in GUI:
- discard config.WALLET_BATCH_RBF
 - allow the user to choose base_tx from a list of batching
   candidates in ConfirmTxDialog
2025-03-05 14:07:02 +01:00
ThomasV 840243e029 separate fee policy from config
- Wallet.make_unsigned_transaction takes a FeePolicy parameter
 - fee sliders act on a FeePolicy instead of config
 - different fee policies may be used for different purposes
 - do not detect dust outputs in lnsweep, delegate that to lnwatcher
2025-03-05 10:29:26 +01:00
SomberNight ffc84ab7d3 tests: fix warning in test_lntransport.py::TestLNTransport::test_loop
$ python3 -X dev -m pytest tests/test_lntransport.py

```
=============================== warnings summary ===============================
tests/test_lntransport.py::TestLNTransport::test_loop
  /usr/local/lib/python3.11/asyncio/streams.py:414: ResourceWarning: unclosed <StreamWriter transport=<_SelectorSocketTransport closing fd=15 read=idle write=<idle, bufsize=0>> reader=<StreamReader transport=<_SelectorSocketTransport closing fd=15 read=idle write=<idle, bufsize=0>>>>
    warnings.warn(f"unclosed {self!r}", ResourceWarning)
```
2025-03-04 15:49:22 +00:00
Sander van Grieken fea598cfbe network: create ProxySettings class replacing dict and encapsulating proxy related funcs,
allow enable/disable proxy without nuking proxy mode, host and port (explicit enable_proxy config setting),
move tor probe from frontend to backend code, add probe buttons for Qt and QML
2025-03-04 14:23:33 +01:00
f321x d348da811a introduce gossip query handling and forwarding 2025-03-04 13:10:37 +01:00
SomberNight f445a476f9 config: raise on setting mistyped or non-existent ConfigVar 2025-03-03 17:41:23 +00:00
ThomasV 3d2531cb93 reintroduce separate request types for lightning and onchain
cmdline: add_request has a --lightning option
2025-02-25 11:27:32 +01:00
ThomasV ea4adbb4d6 Merge pull request #9573 from f321x/simplify_subswap_fee
Simplify submarine swap onchain fee model to single base fee 'mining_fee'
2025-02-20 18:42:56 +01:00
ThomasV 7eec7e6a20 increase sleep time in TestOnionMessageManager.test_forward
Co-authored-by: ghost43 <somber.night@protonmail.com>
2025-02-20 18:05:21 +01:00
f321x 6f97b7b5f9 simplify submarine swap onchain fee model to single base fee 2025-02-20 16:51:48 +01:00
Sander van Grieken ad6eb73dd3 onion_messages: guard onion message forwarding behind config option
EXPERIMENTAL_LN_FORWARD_PAYMENTS (default False)
2025-02-20 00:31:36 +01:00
Sander van Grieken c3c5aaab3d onion_messages: add tests for forwards, receive unsolicited. 2025-02-19 17:43:36 +01:00
Sander van Grieken 0b86e39121 onion_messages: additional checks and comments;
- check on initial_node_id as the type can in theory contain a sciddir.
- log allowed_features if present in recipient_data, with an additional comment describing
the handling of allowed_features in the future.
- document the SHOULD constraint on onion_message payload size
2025-02-19 14:29:50 +01:00
Sander van Grieken 560b244b4d onion_messages: replace more hardcoded test vector values with symbols 2025-02-19 14:29:50 +01:00
Sander van Grieken 6e35ffe4b5 lnmsg: support both primitive and complex types (subtypes) in LNSerializer.
This renames lnmsg._{read,write}_field to lnmsg._{read,write}_primitive_field, renames
LNSerializer._{read,write}_complex_type to LNSerializer.{read,write}_field and allows
LNSerializer.{read,write}_field to handle both primitive and complex types.

Also makes these funcs public, as these encodings are used outside of lnmsg as well
(e.g. encoding blinded paths in BOLT12 invoice_request)
2025-02-19 14:29:50 +01:00
ThomasV d814796484 Deobfuscate names
The name 'requestreply' do not mean anything. If something can either
be a request or a reply, perhaps we can call it 'message', instead of
introducing a new name?

In general, coming up with new names comes at a cost, because you
are forcing other developers to learn and use your terminology.
Please minimize that.
2025-02-19 14:29:50 +01:00
ThomasV 3314ee1de1 test_onion_messages: cleanup time constants, make speedup homogeneous.
1. Do not pass request_reply_timeout and request_reply_retry_delay
to OnionMessageManager.

Arguments passed to a function are useful only if their value might
change during the execution of a program. If that is not the case,
it is preferable to define them as constants. That makes the code
easier to read, because the reader can focus on arguments that are
actually relevant.

2 . Multiply all time constants by the same factor, instead
of doing so incompletely and on a per parameter basis. (note
that before this commit, the speedup factor was not consistent)

3. Do not use util.now(), because it is rounded as integer.

With these changes, the tests can effectively run with a 100x speedup
2025-02-19 14:29:34 +01:00
ThomasV 4efd4a0ff7 test_onion_message: enable logging so that we can see what is going on 2025-02-19 14:29:34 +01:00
Sander van Grieken 7109c22317 unasync, no add_peer in create_onion_message_route_to, add manager tests 2025-02-19 14:29:34 +01:00
Sander van Grieken 7b4180202a add onion message support 2025-02-19 14:29:02 +01:00
f321x 9394b18b4e add method list_enabled_ln_feature_bits 2025-02-18 15:21:24 +01:00
ThomasV 18baf003d4 submarine swaps: backport changes from batch_payment_manager
- add 'is_funded' method to SwapData
 - sign claim transactions using the 'make_witness' method
2025-02-15 10:29:19 +01:00
SomberNight 3de90952d1 tests: add a test for util.ShortID class 2025-02-10 16:36:19 +00:00
ThomasV d25aca7a53 Merge pull request #9447 from f321x/node_ann_dns
Add gossip address field serialization, parsing and tests
2025-01-17 10:52:14 +01:00
ThomasV 64e07732f8 Merge pull request #9430 from SomberNight/202501_funding_pubkey_deriv
lightning: change derivation of funding_pubkey
2025-01-17 10:49:16 +01:00
f321x 1f626f3ad8 add gossip address field serialization, parsing and tests
add space

add gossip address field serialization, parsing and tests

fix linter

consolidate tests, fix intendation

refactor test in loops

add gossip address field serialization, parsing and tests
2025-01-17 10:33:56 +01:00
SomberNight 6c2a4b61a1 tests: lnpeer: add test payment_trampoline_e2e A->T1->C>T2->E
Adds a new test case for e2e trampoline payment, where there are multiple
Trampoline Forwarders which themselves are not directly connected.

This adds a regression test for https://github.com/spesmilo/electrum/pull/9431
https://github.com/spesmilo/electrum/commit/dffdebf778327baae0502b3ec0b57092d48a24d6
2025-01-15 15:12:05 +00:00
SomberNight cba073dfd1 lightning: change derivation of funding_pubkey
Ideally, given an on-chain backup, after the remote force-closes, we should be able to spend our anchor output,
to CPFP the remote commitment tx (assuming the channel used OPTION_ANCHORS).
To spend the anchor output, we need to be able to sign with the local funding_privkey.

Previously we derived the funding_key from the channel_seed (which comes from os.urandom).
Prior to anchors, there was no use case for signing with the funding_key given a channel backup.
Now with anchors, we should make its derivation deterministic somehow, in a way so that it can
be derived given just an on-chain backup.
- one way would be to put some more data into the existing OP_RETURN
  - uses block space
  - the OP_RETURNs can be disabled via "use_recoverable_channels"
  - only the initiator can use OP_RETURNs (so what if channel is in incoming dir?)
- instead, new scheme for our funding_key:
  - we derive the funding_privkey from the lnworker root secret (derived from our bip32 seed)
  - for outgoing channels:
    - lnworker_root_secret + remote_node_id + funding_tx_nlocktime
  - for incoming channels:
    - lnworker_root_secret + remote_node_id + remote_funding_pubkey
  - a check is added to avoid reusing the same key between channels:
      not letting to user open more than one channel with the same peer in a single block
  - only the first 16 bytes of the remote_node_id are used, as the onchain backup OP_RETURNs only contain that
- as the funding_privkey cannot be derived from the channel_seed anymore, it is included in the
imported channel backups, which in turn need a new version defined
  - a wallet db upgrade is used to update already stored imported cbs
  - alternatively we could keep the imported cbs as-is, so no new version, no new funding_privkey field, as it is clearly somewhat redundant given on-chain backups can reconstruct it
    - however adding the field seems easier
      - otherwise the existing code would try to derive the funding_privkey from the channel_seed
      - also note: atm there is no field in the imported backups to distinguish anchor channels vs static-remotekey channels
2025-01-14 17:56:48 +00:00