Improvements in the fuzz-testing scheme of
`fuzz-bolt12-offer-decode` led to the discovery of test inputs
that result in greater in code coverage.
Add these inputs to the test's seed corpus.
Changelog-None: Currently, the `BOLT #12` offer parsing test only
tests the offer decode function. Add a test for the encoding
function as well by making the test roundtrip.
Changelog-Added: askrene: add a new layer auto.include_fees thhat makes fees be deducted from the payment amount making in effect the receiver pay for routing fees.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
We used non-persistent layers before, but what if we save to the datastore and restore?
This takes it from 29 to 45 seconds.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This changes various tests in minor ways:
1. The "l2" secret key in tests/plugins/channeld_fakenet.c is updated.
2. The decompressed gossip data node id needs changing.
3. The coinmoves order changes in bookkeeper for anchors.
4. Various harcoded gossip constants change.
5. Some hardcoded makesecret results change.
6. zeroconf tests which hardcoded node ids change.
7. Arbitrary rune strings change.
8. A log message which uses node ids changes.
These are explicitly written to check that the values don't accidentally change,
which applies to both old and new styles.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
bookkeeper's fee tracking depends on deterministic channel structure, which depends on node ID ordering. I'm marking this with the old_hsmsecret option for now.
Since we have changed our hsm_secret seed the node ids have all change hence the asseertions on the scids break. This changes it so that the direction is determined when the test executes.
We keep a history of logs internally, so we can drop them to disk on a
crash. This "black box recorder" was some of the first code I wrote
for CLN, but I can't remember the last time we use a crash log to
diagnose a problem.
We attempt to prune it to keep it under 10MB, but the complexity
and cost is rarely worth it: simplify it to use a ringbuffer.
Changelog-Changed: lightningd: logging is now more efficient internally (no more pruning, simple ringbuffer).
```
139993 DEBUG lightningd: fixup_scan: block 786151 with 1203 txs
===> 55388 DEBUG plugin-bcli: Log pruned 1001 entries (mem 10508118 -> 10298662)
33000 DEBUG gossipd: Unreasonable timestamp in 0102000a38ec41f9137a5a560dac6effbde059c12cb727344821cbdd4ef46964a4791a0f67cd997499a6062fc8b4284bf1b47a91541fd0e65129505f02e4d08542b16fe28c0ab6f1b372c1a6a246ae63f74f931e8365e15a089c68d61900000000000d9d56000ba40001690fe262010100900000000000000001000003e8000001f30000000000989680
23515 DEBUG hsmd: Client: Received message 14 from client
22269 DEBUG 024b9a1fa8e006f1e3937f65f66c408e6da8e1ca728ea43222a7381df1cc449605-hsmd: Got WIRE_HSMD_ECDH_REQ
14409 DEBUG gossipd: Enqueueing update for announce 0102002f7e4b4deb19947c67292e70cb22f7fac837fa9ee6269393f3c513d0431d52672e7387625856c19299cfd584e1a3f39e0f98df13c99090df9f4d5cca8446776fe28c0ab6f1b372c1a6a246ae63f74f931e8365e15a089c68d61900000000000e216b0008050001692e1c390101009000000000000003e800000000000013880000004526945a00
12534 DEBUG gossipd: Previously-rejected announce for 514127x248x1
10761 DEBUG 02e01367e1d7818a7e9a0e8a52badd5c32615e07568dbe0497b6a47f9bef89d6af-channeld-chan#70770: Got it!
10761 DEBUG 02e01367e1d7818a7e9a0e8a52badd5c32615e07568dbe0497b6a47f9bef89d6af-channeld-chan#70770: ... , awaiting 1120
10761 DEBUG 02e01367e1d7818a7e9a0e8a52badd5c32615e07568dbe0497b6a47f9bef89d6af-channeld-chan#70770: Sending master 1020
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Everyone should be using the new name.
Changelog-Removed: JSON-RPC: `listpeers` `features` array string "option_anchors_zero_fee_htlc_tx": use "option_anchors" (spec renamed it). Deprecated in 24.08.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means:
1. downgrade changes (we no longer fail due to node biases).
2. various deprecations no longer are
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If l4 sends a WIRE_QUERY_SHORT_CHANNEL_IDS at the wrong time, we will
get that and be upset the response is wrong:
```
2026-01-13T14:45:55.9059786Z E AssertionError: assert ['010806226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00000000000f42400100110000006800000100000000690000010000', '010506226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f001100000068000001000000006900000100000103000402'] in (['010806226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00000000000f42400100110000006800000100000000690000010000'], ['010806226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00000000000f42400100110000006900000100000000680000010000'])
2026-01-13T14:45:55.9063357Z
2026-01-13T14:45:55.9063527Z tests/test_gossip.py:762: AssertionError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Because l1 and l3 allow localhost as a broadcastable address, they can
try to reconnect. Disable reconnections, so we don't race:
```
> l2.rpc.connect(l3.info['id'], 'localhost', l3.port)
tests/test_plugin.py:4146:
...
elif "error" in resp:
> raise RpcError(method, payload, resp['error'])
E pyln.client.lightning.RpcError: RPC call failed: method: connect, payload: {'id': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d', 'host': 'localhost', 'port': 45035}, error: {'code': 402, 'message': 'disconnected during connection'}
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Seems like sleep(1) isn't always enough. Give in and put a log
message there, and use that:
```
waitfut = executor.submit(l2.rpc.wait, subsystem='forwards', indexname='deleted', nextvalue=1)
time.sleep(1)
l2.rpc.delforward(scid12, 1, 'failed')
waitres = waitfut.result(TIMEOUT)
> assert waitres == {'subsystem': 'forwards',
'deleted': 1,
'forwards': {'in_channel': scid12,
'in_htlc_id': 1,
'status': 'failed'}}
E AssertionError: assert {'subsystem': 'forwards', 'deleted': 1} == {'subsystem': 'forwards', 'deleted': 1, 'forwards': {'in_channel': '103x2x0', 'in_htlc_id': 1, 'status': 'failed'}}
E
E Common items:
E {'deleted': 1, 'subsystem': 'forwards'}
E Right contains 1 more item:
E {'forwards': {'in_channel': '103x2x0', 'in_htlc_id': 1, 'status': 'failed'}}
E
E Full diff:
E {
E 'deleted': 1,
E - 'forwards': {
E - 'in_channel': '103x2x0',
E - 'in_htlc_id': 1,
E - 'status': 'failed',
E - },
E 'subsystem': 'forwards',
E }
tests/test_misc.py:3599: AssertionError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There's one more complaint we can see when plugins get upset:
```
lightningd-1 2026-01-12T06:10:49.317Z **BROKEN** plugin-cln-xpay: askrene-create-layer failed with {"code":-4, "message":"Plugin terminated before replying to RPC call."}
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In fact, you *must* use mnemonic to successfully recover a modern node!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `recover` takes a 12-word mnemonic for nodes created by v25.12 or later.
We cannot use the codex32 or raw hex for recovery of 25.12 nodes,
since they will then use the incorrect derivation for all paths, and
be unable to spend (or even find!) their funds.
So implement `getsecret` to replace `getcodexsecret`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: `lightning-hsmtool`: `getsecret` replaces `getcodexsecret` for modern nodes (gives mnemonic).
Changelog-Deprecated: `lightning-hsmtool`: `getcodexsecret`. Use `getsecret`.
Changelog-Fixed: askrene: fix a plugin crash triggered during single path payments when a channel fees doesn't fit u32.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
It's done as HTLCs finalize, but we can close the incoming HTLC as soon as we get the
preimage, so that entire thing could finish before the outgoing HTLC.
```
> check_channel_moves(l1, expected_channel1)
tests/test_coinmoves.py:307:
...
E Full diff:
E [
E {
E 'account_id': '58d371ab100e0ea847a11c9550add273ef8531bc12bb51b0e30c8f833506a772',
E 'created_index': 1,
E 'credit_msat': 0,
E 'debit_msat': 1000000,
E 'fees_msat': 0,
E 'group_id': 1318196858430961660,
E 'part_id': 1,
E 'payment_hash': '8da829ab29715106a4e767facc0b58776ae5bfc11c4e9dcda3063013e1ef8ef0',
E 'primary_tag': 'invoice',
E },
E {
E 'account_id': '0b872506f67b363803cd85cf9ff6807ebc1dc8a4521aa191386b4c5366d490d7',
E 'created_index': 2,
E 'credit_msat': 100000,
E 'debit_msat': 0,
E 'fees_msat': 0,
E 'primary_tag': 'pushed',
E },
E {
E + 'account_id': '0b872506f67b363803cd85cf9ff6807ebc1dc8a4521aa191386b4c5366d490d7',
E + 'created_index': 3,
E + 'credit_msat': 10000100001,
E + 'debit_msat': 0,
E + 'fees_msat': 100001,
E + 'payment_hash': '0ebfa5387de5fd12c15089833b0193fb6007e9f494ec24d479e327a96ac8e8c0',
E + 'primary_tag': 'routed',
E + },
E + {
E 'account_id': '58d371ab100e0ea847a11c9550add273ef8531bc12bb51b0e30c8f833506a772',
E - 'created_index': 3,
E ? ^
E + 'created_index': 4,
E ? ^
E 'credit_msat': 0,
E 'debit_msat': 10000000000,
E 'fees_msat': 100001,
E 'payment_hash': '0ebfa5387de5fd12c15089833b0193fb6007e9f494ec24d479e327a96ac8e8c0',
E 'primary_tag': 'routed',
E },
E - {
E - 'account_id': '0b872506f67b363803cd85cf9ff6807ebc1dc8a4521aa191386b4c5366d490d7',
E - 'created_index': 4,
E - 'credit_msat': 10000100001,
E - 'debit_msat': 0,
E - 'fees_msat': 100001,
E - 'payment_hash': '0ebfa5387de5fd12c15089833b0193fb6007e9f494ec24d479e327a96ac8e8c0',
E - 'primary_tag': 'routed',
E - },
E ]
```
We need to make sure anchor reaches bitcoind, otherwise it might mine
the commitment tx without it. This can happen in
test_coinmoves_unilateral_htlc_fulfill as well.
```
> check_chain_moves(l1, expected_chain1)
tests/test_coinmoves.py:844:
...
E Full diff:
E [
E {
E 'account_id': 'wallet',
E 'blockheight': 102,
E 'created_index': 1,
E 'credit_msat': 100000000000,
E 'debit_msat': 0,
E 'extra_tags': [],
E 'output_msat': 100000000000,
E 'primary_tag': 'deposit',
E 'utxo': 'fca99b85e58f8ae23e5c6872e0500784997deb98bfc92e43449206553a108db2:1',
E },
E {
E 'account_id': 'wallet',
E 'blockheight': 103,
E 'created_index': 2,
E 'credit_msat': 0,
E 'debit_msat': 100000000000,
E 'extra_tags': [],
E 'output_msat': 100000000000,
E 'primary_tag': 'withdrawal',
E 'spending_txid': 'c097ad8bde478396c961369b69c50a144fae3423f36af4554f3fb1dacfdff83f',
E 'utxo': 'fca99b85e58f8ae23e5c6872e0500784997deb98bfc92e43449206553a108db2:1',
E },
E {
E 'account_id': 'wallet',
E 'blockheight': 103,
E 'created_index': 3,
E 'credit_msat': 25000000,
E 'debit_msat': 0,
E 'extra_tags': [],
E 'output_msat': 25000000,
E 'primary_tag': 'deposit',
E 'utxo': 'c097ad8bde478396c961369b69c50a144fae3423f36af4554f3fb1dacfdff83f:1',
E },
E {
E 'account_id': '3ff8dfcfdab13f4f55f46af32334ae4f140ac5699b3661c9968347de8bad97c0',
E 'blockheight': 103,
E 'created_index': 4,
E 'credit_msat': 99970073000,
E 'debit_msat': 0,
E 'extra_tags': [
E 'opener',
E ],
E 'output_msat': 99970073000,
E 'peer_id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59',
E 'primary_tag': 'channel_open',
E 'utxo': 'c097ad8bde478396c961369b69c50a144fae3423f36af4554f3fb1dacfdff83f:0',
E },
E {
E - 'account_id': 'wallet',
E + 'account_id': '3ff8dfcfdab13f4f55f46af32334ae4f140ac5699b3661c9968347de8bad97c0',
E 'blockheight': 104,
E 'created_index': 5,
E - 'credit_msat': 0,
E - 'debit_msat': 25000000,
E - 'extra_tags': [],
E - 'output_msat': 25000000,
E - 'primary_tag': 'withdrawal',
E - 'spending_txid': '1b6fbf9887d6f9cce727fc8bf9f582a3353be682998d4bafc9691c9ed26897e7',
E - 'utxo': 'c097ad8bde478396c961369b69c50a144fae3423f36af4554f3fb1dacfdff83f:1',
E - },
E - {
E - 'account_id': 'wallet',
E - 'blockheight': 104,
E - 'created_index': 6,
E - 'credit_msat': Decimal('15579000.00000000'),
E - 'debit_msat': 0,
E - 'extra_tags': [],
E - 'output_msat': Decimal('15579000.00000000'),
E - 'primary_tag': 'deposit',
E - 'utxo': '1b6fbf9887d6f9cce727fc8bf9f582a3353be682998d4bafc9691c9ed26897e7:0',
E - },
E - {
E - 'account_id': '3ff8dfcfdab13f4f55f46af32334ae4f140ac5699b3661c9968347de8bad97c0',
E - 'blockheight': 104,
E - 'created_index': 7,
E 'credit_msat': 0,
E 'debit_msat': 49970073000,
E 'extra_tags': [],
E 'output_count': 5,
E 'output_msat': 99970073000,
E 'primary_tag': 'channel_close',
E 'spending_txid': 'a499419bfdce179727cffca45429151db47839b247d83f71837429f021ae6322',
E 'utxo': 'c097ad8bde478396c961369b69c50a144fae3423f36af4554f3fb1dacfdff83f:0',
E },
E {
E 'account_id': 'external',
E 'blockheight': 104,
E - 'created_index': 8,
E ? ^
E + 'created_index': 6,
E ? ^
E 'credit_msat': 330000,
E 'debit_msat': 0,
E 'extra_tags': [],
E 'originating_account': '3ff8dfcfdab13f4f55f46af32334ae4f140ac5699b3661c9968347de8bad97c0',
E 'output_msat': 330000,
E 'primary_tag': 'anchor',
E 'utxo': 'a499419bfdce179727cffca45429151db47839b247d83f71837429f021ae6322:0',
E },
E {
E 'account_id': 'external',
E 'blockheight': 104,
E - 'created_index': 9,
E ? ^
E + 'created_index': 7,
E ? ^
E 'credit_msat': 330000,
E 'debit_msat': 0,
E 'extra_tags': [],
E 'originating_account': '3ff8dfcfdab13f4f55f46af32334ae4f140ac5699b3661c9968347de8bad97c0',
E 'output_msat': 330000,
E 'primary_tag': 'anchor',
E 'utxo': 'a499419bfdce179727cffca45429151db47839b247d83f71837429f021ae6322:1',
E },
E {
E 'account_id': 'external',
E 'blockheight': 104,
E - 'created_index': 10,
E ? ^^
E + 'created_index': 8,
E ? ^
E 'credit_msat': 50000000000,
E 'debit_msat': 0,
E 'extra_tags': [],
E 'originating_account': '3ff8dfcfdab13f4f55f46af32334ae4f140ac5699b3661c9968347de8bad97c0',
E 'output_msat': 50000000000,
E 'primary_tag': 'to_them',
E 'utxo': 'a499419bfdce179727cffca45429151db47839b247d83f71837429f021ae6322:4',
E },
E ]
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It failed, because it got the message before connectd has processed
the updated allow list:
```
lightningd-2 2026-01-06T07:53:02.817Z INFO plugin-allow_even_msgs.py: Killing plugin: stopped by lightningd via RPC
...
lightningd-1 2026-01-06T07:53:02.820Z DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-lightningd: sendcustommsg id="-c:sendcustommsg#16" sending a custom even message (43690)
...
lightningd-1 2026-01-06T07:53:02.820Z 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-connectd: [OUT] aaaaffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffbb
lightningd-2 2026-01-06T07:53:02.823Z 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-connectd: [IN] aaaaffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffbb
...
lightningd-2 2026-01-06T07:53:02.823Z DEBUG connectd: Now allowing 0 custom message types
```
Resulting in:
``
l2.daemon.wait_for_log(r'\[IN\] {}'.format(msg))
> l1.daemon.wait_for_log('Invalid unknown even msg')
tests/test_misc.py:4673:
...
> raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E TimeoutError: Unable to find "[re.compile('Invalid unknown even msg')]" in logs.
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We don't explicitly save the return code in db, so we need to reconstruct it.
We didn't cover the "peer told us our onion was bad" corner case. But it's hardly
worth a changelog message, since users will never see this.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This showed up as a flake, where we "got lucky" and the sendpay resolved before waitsendpay was called. Instead, make this race explicit, so we can test it.
```
# FIXME: #define PAY_UNPARSEABLE_ONION 202
PAY_UNPARSEABLE_ONION = 202
> assert err.value.error['code'] == PAY_UNPARSEABLE_ONION
E assert 204 == 202
tests/test_misc.py:2152: AssertionError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
```
[gw0] [ 24%] PASSED tests/test_misc.py::test_hsm_capabilities
tests/test_connection.py::test_funding_cancel_race
Error: Process completed with exit code 143.
```
Seems like 100 nodes is too many!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The channel vanishes from listchannels when it's dying, *but* only when it gets deleted
do we consider moving the actual node_announcement. We have to wait until gossipd
has seen the 12 blocks, and move it if necessary.
```
E Full diff:
E {
E 'nodes': [
E - {
E - 'addresses': [],
E - 'alias': 'SILENTARTIST-e986cd2-modded',
E - 'color': '022d22',
E - 'features': '808898880a8a59a1',
E - 'last_timestamp': 1767572731,
E - 'nodeid': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59',
E - },
E {
E 'addresses': [],
E 'alias': 'HOPPINGFIRE-e986cd2-modded',
E 'color': '035d2b',
E 'features': '808898880a8a59a1',
E 'last_timestamp': 1767572731,
E 'nodeid': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d',
E },
E {
E 'addresses': [],
E 'alias': 'JUNIORFELONY-e986cd2-modded',
E 'color': '0382ce',
E 'features': '808898880a8a59a1',
E 'last_timestamp': 1767572731,
E 'nodeid': '0382ce59ebf18be7d84677c2e35f23294b9992ceca95491fcf8a56c6cb2d9de199',
E },
E {
E 'addresses': [],
E + 'alias': 'SILENTARTIST-e986cd2-modded',
E + 'color': '022d22',
E + 'features': '808898880a8a59a1',
E + 'last_timestamp': 1767572731,
E + 'nodeid': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59',
E + },
E + {
E + 'addresses': [],
E 'alias': 'JUNIORBEAM-e986cd2-modded',
E 'color': '0266e4',
E 'features': '808898880a8a59a1',
E 'last_timestamp': 1767572732,
E 'nodeid': '0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518',
E },
E ],
E }
tests/test_gossip.py:2390: AssertionError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
```
2026-01-05T00:11:22.0447771Z # Only up to one should succeed.
2026-01-05T00:11:22.0448201Z success = False
2026-01-05T00:11:22.0448571Z for c in completes:
2026-01-05T00:11:22.0448957Z try:
2026-01-05T00:11:22.0449322Z c.result(TIMEOUT)
2026-01-05T00:11:22.0449934Z num_complete += 1
2026-01-05T00:11:22.0450378Z > assert not success
2026-01-05T00:11:22.0451005Z E assert not True
2026-01-05T00:11:22.0451201Z
2026-01-05T00:11:22.0451331Z tests/test_connection.py:1596: AssertionError
```
We don't know *which* ones succeeded. Fix that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's not reliable:
```
# We should have deferred hook update at least once!
> l2.daemon.wait_for_log("UNUSUAL plugin-dep_b.py: Deferring registration of hook htlc_accepted until it's not in use.")
tests/test_plugin.py:2646:
...
if self.is_in_log(r):
print("({} was previously in logs!)".format(r))
> raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E TimeoutError: Unable to find "[re.compile("UNUSUAL plugin-dep_b.py: Deferring registration of hook htlc_accepted until it's not in use.")]" in logs.
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
