3925 Commits

Author SHA1 Message Date
Peter Neuroth
3606106c4f lsp_plugin: add dev-eneabled flag for client
While this is still experimental, we only want to enable the client when
explicitly defined!

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-11-13 10:58:49 +10:30
Rusty Russell
b05dbeff4e lightningd: don't allow invoices with 640 byte descriptions.
They are invalid!  This is because our BOLT11_FIELD_BYTE_LIMIT is not the limit,
it's one greater than the limit.

Reported-by: https://github.com/noblepayne
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `invoice` no longer accepts 640-byte descriptions (it would produce malformed invoices).
2025-11-13 10:57:49 +10:30
Wes Payne
89270024e8 keysend: enforce BOLT11 description length limit
The keysend plugin previously used `> 1023` as the cutoff for
description length when inserting an invoice. This was
inconsistent with invoice.c, which enforces the BOLT11 description
field limit defined in `common/bolt11.h`.

This patch switches to using `BOLT11_FIELD_BYTE_LIMIT` directly.
As a result, keysend no longer fails on descriptions between
641–1023 bytes, which previously caused unexpected failures.

A new regression test (`test_keysend_description_size_limit`)
exercises boundary cases just below, at, and above the limit.

Changelog-Fixed: Protocol: `keysend` with descriptions of length 640-1023 bytes fixed.
Signed-off-by: Wes Payne <noblepayne@noblepayne.com>
2025-11-13 10:57:49 +10:30
Rusty Russell
ec05e5da91 autoclean: clean network events (30 days by default).
We also document this in the listnetworkevents command itself.

The test_autoclean_once was getting repetitive, so I cleaned that
up too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `autoclean` will remove networkevents after 30 days by default.
2025-11-12 13:58:43 +10:30
Rusty Russell
9f54f01470 lightningd: delnetworkevent support
Changelog-Added: JSON-RPC: `delnetworkevent` to delete from listnetworkevents.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-12 13:58:43 +10:30
Rusty Russell
08de61cd38 sql: add support for listnetworkevents
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `sql` now supports the `networkevents` table.
2025-11-12 13:58:43 +10:30
Rusty Russell
3b332948dd lightningd: implement listnetworkevents.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-12 13:58:43 +10:30
Rusty Russell
565f7deec0 connectd: at disconnected, tell lightningd how long we were connected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-12 13:58:43 +10:30
Matt Whitlock
36a44ea09d tests: work around socket path name too long on Linux
When running the integration test suite in a deeply nested directory
tree, the path name of the Unix domain socket might be longer than can
fit in a struct sockaddr_un. On Linux, we can use the /proc/self/cwd
trick to shorten the path name.

Changelog-Fixed: Integration tests no longer fail when run in a deeply nested directory on Linux.
2025-11-11 08:13:34 +10:30
Chandra Pratap
fa28e07355 fuzz-tests: Add coverage-increasing inputs to seed corpora
Change in the fuzzing scheme of fuzz-hsm_encryption led to the
discovery of test inputs that result in greater in code coverage.
Add these inputs to the test's seed corpus.

[ Changed from fuzz-hsm_encryption to fuzz-hsm_secret --RR ]
2025-11-10 16:59:01 +10:30
Chandra Pratap
9be3eeea34 fuzz-tests: get rid of magic numbers in fuzz-hsm_encryption.c
Changelog-None: `fuzz-hsm_encryption.c` hard codes the lengths
sizeof(struct secret) as 32 and crypto_pwhash_argon2id_PASSWD_MAX
as 4294967295. Replace the latter with the former to improve
readability and maintainability.

While at it, replace the `tal_free()` call on our secret key with
`discard_key()`. This has the benefit of testing `discard_key()`.

[ Changed from fuzz-hsm_encryption to fuzz-hsm_secret --RR ]
2025-11-10 16:59:01 +10:30
Rusty Russell
b33f86bbb5 fuzz/fuzz-handle_onion_message.c: fix header order.
We check this now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-10 16:59:01 +10:30
Chandra Pratap
7bbaecd30e fuzz-tests: Add a seed corpus for the new test
Add a minimal input set as a seed corpus for the newly introduced
test. This leads to discovery of interesting code paths faster.
2025-11-10 15:36:36 +10:30
Chandra Pratap
76a7781cc3 fuzz-tests: Add a test for full_channel operations
Changelog-None: Functions defined in `channeld/full_channel.h`
contain channel operations like `ADD_HTLC`, `FULFILL_HTLC`,
`UPDATE_FEERATE`, etc.

Since they are a critical part of the HTLC state machine and may
be influenced by external agents, add a stateful test for them.
2025-11-10 15:36:36 +10:30
Chandra Pratap
df3043c4ef fuzz-tests: Add a seed corpus for the new test
Add a minimal input set as a seed corpus for the newly introduced
test. This leads to discovery of interesting code paths faster.
2025-11-10 15:14:31 +10:30
Chandra Pratap
c684e350f0 fuzz-tests: Add a test for the onion message handler
Changelog-None: `handle_onion_message()` in `connectd/onion_message.c`
is responsible for handling incoming onion messages from a peer.
Since it deals with external input, add a test for it.
2025-11-10 15:14:31 +10:30
Peter Neuroth
21fb7600f3 lightningd: add invoice_amt to the htlc_accepted hook
This commit introduces a new field `invoice_msat` to the htlc_accepted
hook. If this field is specified it will replace the amount of the
invoice that belongs to the payment_hash of the HTLC on internal checks.

This is useful in scenarios where we actually expect a smaller amount
than initially specified in an invoice.

Changelog-Changed: Plugins: `htlc_accepted` hook can now override the
expected total amount of the invoice that belongs to the HTLC.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-11-10 15:05:27 +10:30
Rusty Russell
1e7ffeb89d tests: don't assume newaddr returns bech32 by default.
Either use p2tr (if not elements) or explicitly ask for a bech32 address.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-07 10:51:04 +10:30
Rusty Russell
ac60568398 lightningd: fix case where injectpaymentonion failure results in listsendpays "pending".
If we failed after we register (e.g. channeld not available), we don't
mark it failed.  We shouldn't register until we've definitely created
the htlc.

Changelog-Fixed: `xpay` would sometimes leave payment parts status `pending` in failure cases (as seen in listpays or listsendpays).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/8629
2025-11-04 11:06:50 +10:30
Rusty Russell
350578d767 pytest: test that we correctly mark a payment part failed if we cannot queue it to the channeld for the peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-04 11:06:50 +10:30
Rusty Russell
43d7db7581 plugins/bcli: use -stdin to feed arguments, in case we have a giant tx.
```
lightningd-1 2025-10-27T11:26:04.285Z **BROKEN** plugin-bcli: bitcoin-cli exec failed: Argument list too long
```

Use -stdin to bitcoin-cli: we can then handle arguments of arbitrary length.

Fixes: https://github.com/ElementsProject/lightning/issues/8634
Changelog-Fixed: plugins: `bcli` would fail with "Argument list too long" when sending a giant tx.
2025-11-03 14:49:33 +10:30
Rusty Russell
5d64eb9ac4 pytest: test for bcli crash with huge PSBTs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-03 14:49:33 +10:30
Rusty Russell
bd1798323a bookkeeper: fix assert() which happens with parallel queries.
```
bookkeeper: plugins/bkpr/bookkeeper.c:1226: parse_and_log_chain_move: Assertion `e->db_id > bkpr->chainmoves_index' failed.
bookkeeper: FATAL SIGNAL 6 (version v25.09-245-g901714b-modded)
0x5d7d8718b40f send_backtrace
        common/daemon.c:36
0x5d7d8718b4ab crashdump
        common/daemon.c:81
0x7a6086c4532f ???
        ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x7a6086c9eb2c __pthread_kill_implementation
        ./nptl/pthread_kill.c:44
0x7a6086c9eb2c __pthread_kill_internal
        ./nptl/pthread_kill.c:78
0x7a6086c9eb2c __GI___pthread_kill
        ./nptl/pthread_kill.c:89
0x7a6086c4527d __GI_raise
        ../sysdeps/posix/raise.c:26
0x7a6086c288fe __GI_abort
        ./stdlib/abort.c:79
0x7a6086c2881a __assert_fail_base
        ./assert/assert.c:96
0x7a6086c3b516 __assert_fail
        ./assert/assert.c:105
0x5d7d8717505d parse_and_log_chain_move
        plugins/bkpr/bookkeeper.c:1226
0x5d7d871754f4 listchainmoves_done
        plugins/bkpr/bookkeeper.c:169
0x5d7d87182a4b handle_rpc_reply
        plugins/libplugin.c:1072
0x5d7d87182b5c rpc_conn_read_response
        plugins/libplugin.c:1361
0x5d7d871ba660 next_plan
        ccan/ccan/io/io.c:60
0x5d7d871bab31 do_plan
        ccan/ccan/io/io.c:422
0x5d7d871babee io_ready
        ccan/ccan/io/io.c:439
```

Reported-by: @michael1011
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: plugins: assertion crash in bookkeeper when fresh records arrive while multiple queries in progress.
2025-11-03 14:03:25 +10:30
Rusty Russell
c38699559d pytest: test for parallel bookkeeper queries.
If both refresh new events, we will get an assertion:

```
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-03 14:03:25 +10:30
Rusty Russell
07c57b6015 askrene: implement 10-second deadline.
We have another report of looping.  This maxparts code is being completely
rewritten, but it's good to have a catchall for any other cases which might
emerge.

I had to make it customizable since our tests under valgrind are SLOW!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-03 12:54:37 +10:30
Rusty Russell
03d38b612d askrene: fix infinite loop if refine_flows() cuts down our last flow with 1 remaining before maxparts.
1. We would find a flow.
2. refine_flow would reduce it so it doesn't deliver enough.
3. So we need to find another, but we are at the limit.
4. So we remove the flow we found.
5. Goto 1.

This can be fixed by disabling a channel which we caused us to reduce the flow,
so we should always make forward progress.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: `askrene` could enter an infinite loop when maxparts is restricted.
2025-11-03 12:54:37 +10:30
Rusty Russell
6b480c74f8 pytest: test for askrene infinite loop with maxparts set.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-03 12:54:37 +10:30
Rusty Russell
5106440c32 bookkeeper: fix restoration of derived wallet blockheights on restart.
We complain:
```
lightningd-1 2025-10-31T00:55:00.377Z **BROKEN** plugin-bookkeeper: Unparsable blockheight datastore entry: {"key":["bookkeeper","blockheights","756999f870a7a7c97f5c143f12b9096a50d1b1acd74aeb9ab2dc251a5c361494"],"generation":0,"hex":"00000067"}
```

And we don't have the blockheight:

```
                   {
                       'account': 'external',
         -             'blockheight': 103,
         ?                            - -
         +             'blockheight': 0,
                       'credit_msat': 555555000,
                       'currency': 'bcrt',
                       'debit_msat': 0,
                       'origin': 'wallet',
                       'outpoint': '756999f870a7a7c97f5c143f12b9096a50d1b1acd74aeb9ab2dc251a5c361494:0',
                       'tag': 'deposit',
                       'timestamp': 1761872097,
                       'type': 'chain',
                   },
```

Reported-by: @michael1011
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: `bookkeeper` now correctly restores chain event blockheights it has derived.
2025-11-03 12:36:38 +10:30
Rusty Russell
5f2e3248af pytest: test for blockheight entries in bookkeeper being saved across restart.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-03 12:36:38 +10:30
Rusty Russell
6a84fc2d47 pytest: don't ask for p2tr addresses on liquid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-26 12:37:58 +10:30
Rusty Russell
42f9361375 ccan: update to get improved grab_file API, and adapt code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-26 12:37:58 +10:30
Sangbida Chaudhuri
310405761a schema: update schema to remove bip86 and add mnemonic to expose secret
This schema change updates newaddr to remove bip86 which was previously added, since don't want to make unnecessary schema changes this is being removed.

The generated files for the exposesecret schema change are also being added
2025-10-26 12:37:58 +10:30
Sangbida Chaudhuri
b4984fd94e hsm_encryption: delete hsm_encryption
Changelog-Removed: Remove hsm_encryption files as they have now been replaced by hsm_secret
2025-10-26 12:37:58 +10:30
Rusty Russell
631a8ccdc1 fuzz: translate fuzz-hsm_encryption to fuzz-hsm_secret.
Copy corpora, since it's the same code paths.

Thanks Claude!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-26 12:37:58 +10:30
Sangbida Chaudhuri
22b17daab9 tests: Update test assertions and comments for unified derivation approach 2025-10-26 12:37:58 +10:30
Sangbida Chaudhuri
9754603dbe options: remove --use-bip86-derivation
We're removing --use-bip86-derivation. Since a mnemonic will now be the standard hsm_secret BIP86 base wallet addresses will also be the standard.
2025-10-26 12:37:58 +10:30
Sangbida Chaudhuri
c7f369514c test: Add a psbt test and an index boundary test 2025-10-26 12:37:58 +10:30
Sangbida Chaudhuri
7f3a57cc41 tests: add BIP86 support 2025-10-26 12:37:58 +10:30
Sangbida Chaudhuri
de9c6305a1 exposesecret: Add support for mnemonic-based HSM secrets
Update the exposesecret plugin to work with the new unified HSM secret
format that supports BIP39 mnemonics.

Changelog-Added - exposesecret now has a mnemonic field
2025-10-26 12:37:58 +10:30
Sangbida Chaudhuri
218dc2fe20 hsmd: use the new mnemonic-compatible hsm_secret routines.
Changelog-Changed: hsmd: New nodes will now be created with a BIP-39 12-word phrase as their root secret.
Changelog-Deprecated: config: `encrypted-hsm` to require a passphrase (use `hsm-passphrase`).
Changelog-Added: config: `hsm-passphrase` indicates we should use a manual passphrase with the hsm secret.
2025-10-26 12:37:58 +10:30
Sangbida Chaudhuri
3da0f16f14 tools: update hsmtool to use the new hsm_secret API.
Changelog-Added: `hsmtool` now supports hsm_secret files using a 12-word mnemonic.
Changelog-Removed: hsmtool support for mnemonics in non-english languages removed.
2025-10-26 12:37:58 +10:30
Rusty Russell
71f164a454 autogenerate-rpc-examples.py: add examples for listchainmoves and listchannelmoves.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-24 11:30:17 +10:30
Rusty Russell
9987b6f93a lightningd: try harder to ensure uniqueness in --dev-save-plugin-io names.
Incorporate a time: this covers the restart case as well.  And make it time_mono(),
which doesn't get overridden when we override normal wall time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-24 11:30:17 +10:30
Rusty Russell
478a0d5792 wallet: make sure to watch all txids in transactions table.
We watch if they are to do with a channel, or have outputs going to us, but otherwise
we didn't, so we never updated the blockheight in the db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `listtransactions` now correctly updates `blockheight` for txs created by `sendpsbt` which have no change outputs.
2025-10-24 11:30:17 +10:30
Rusty Russell
2591ac486d pytest: add test that we notice height change of sendpsbt with no change.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-24 11:30:17 +10:30
Rusty Russell
89eaf8341b lightningd: cancel watching original funding when we switch to the new one via splice.
This happens if the channel is *not* announcable yet.  Then we hit the assertion
in funding_depth_cb that the txid is the same as the current funding.txid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: fixed crash when we splice a channel which hasn't been announced yet.
2025-10-24 11:30:17 +10:30
Rusty Russell
d4a541c58a pytest: test for splicing while channel is not announced yet.
```
DEBUG   lightningd: Got depth change 2->3 for e9e31956f77c3844ee2e6e4607dbfebdee95a9aa549668a7a429b8246a6a29de
**BROKEN** lightningd: FATAL SIGNAL 6 (version v25.09-20-g003ba4a)
**BROKEN** lightningd: backtrace: common/daemon.c:41 (send_backtrace) 0x619bef20e274
**BROKEN** lightningd: backtrace: common/daemon.c:78 (crashdump) 0x619bef20e408
**BROKEN** lightningd: backtrace: ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0 ((null)) 0x7a1ccf24532f
**BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:44 (__pthread_kill_implementation) 0x7a1ccf29eb2c
**BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:78 (__pthread_kill_internal) 0x7a1ccf29eb2c
**BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:89 (__GI___pthread_kill) 0x7a1ccf29eb2c
**BROKEN** lightningd: backtrace: ../sysdeps/posix/raise.c:26 (__GI_raise) 0x7a1ccf24527d
**BROKEN** lightningd: backtrace: ./stdlib/abort.c:79 (__GI_abort) 0x7a1ccf2288fe
**BROKEN** lightningd: backtrace: ./assert/assert.c:96 (__assert_fail_base) 0x7a1ccf22881a
**BROKEN** lightningd: backtrace: ./assert/assert.c:105 (__assert_fail) 0x7a1ccf23b516
**BROKEN** lightningd: backtrace: lightningd/peer_control.c:2202 (funding_depth_cb) 0x619bef1ac497
**BROKEN** lightningd: backtrace: lightningd/watch.c:223 (txw_fire) 0x619bef1cfcbf
**BROKEN** lightningd: backtrace: lightningd/watch.c:292 (watch_topology_changed) 0x619bef1cffa4
**BROKEN** lightningd: backtrace: lightningd/chaintopology.c:829 (updates_complete) 0x619bef144a8c
**BROKEN** lightningd: backtrace: lightningd/chaintopology.c:1047 (get_new_block) 0x619bef14561e
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-24 11:30:17 +10:30
Rusty Russell
eb0d042693 common: add brace hack for jsonrpc_async_parse.
This is a trick from bcli: we ask bitcoind for the block, and it hands
us a 2MB hex blob (which we read in multiple parts).  Our parser wades
through it all, but a quick search for '}' makes it much faster.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-24 11:28:57 +10:30
Rusty Russell
c88c8eacd0 lightningd: use jsonrpc_io for reading JSON commands.
This is more efficient if we have lots of incoming commands, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-24 11:28:57 +10:30
Rusty Russell
365add06f7 fuzz/fuzz-hsm_encryption: don't run as unit test under valgrind.
Thanks to Argon hashing, this is intolerably slow under valgrind, and
times out under CI.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-23 06:44:04 +10:30