Commit Graph

17144 Commits

Author SHA1 Message Date
Rusty Russell
1102d8063e askrene: add optional layers to reservations.
We have the issue of aliases: xpay uses scids like 0x0x0 for
routehints and blinded paths, and then can apply reservations to them.  But
generally, reservations are *global*, so we need to differentiate.

Changelog-Added: Plugins: `askrene-reserve` and `askrene-unreserve` can take an optional `layer` inside `path` elements.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 13:52:54 +10:30
Rusty Russell
da4edd2811 pytest: test to demonstrate that reservations of "private" channels overlap.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 13:52:54 +10:30
Rusty Russell
6e4bc1dd6c askrene: neated flow array handling, by freeing flows we discard.
Pointed out by @Lagrang3; he's right, while it's a temporary leak the
way we use flows, it's still a trap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
69adf38784 askrene: remove now-unused bottleneck_idx from flow_max_deliverable.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
8c7ac33f88 askrene: implement reduce_num_flows in refine, using increase_flows().
Now we simply call it at the end.  We need to check it hasn't violated fee maxima, but
otherwise it's simple.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: `askrene` now handles limits on number of htlcs much more gracefully.
2025-11-17 10:56:18 +10:30
Rusty Russell
050b149e53 askrene: handle maxparts parameter values 1 and 0.
For 1, we use single-path.  For 0, reject.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
330f51aad7 askrene: make increase_flows function more generic.
Rewrite it, so it properly takes into account interactions between flows
by using reservations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
a8a6a44679 askrene: clean up renamed functions.
We added _noidx versions of the sort functions, but now they're the only ones, we can
rename them to the old names.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
7c7a4f8795 askrene: Remove index indirection from squash_flows, simplify sorting.
We don't need to convert to strings, we can compare directly.  This removes the final
use of the index arrays.

This of course changes the order of returned routes, which alters test_real_biases, since
that biases against the final channel in the *first* route.

Took me far too long to diagnose that!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
2735673f82 askrene: make increase_flows() use the raw flows array.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
e3dfc020da askrene: remove indexes from refine_flows except for increase_flows()
This removes the index array from code after increase_flows()m, so we use the flows
array directly.

The next step will be to make increase_flows() use the flows array, and remove the
index array indirection entirely.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
565a92e8fa askrene: use flows array directly in remove_excess.
We don't need the indexes array, we can use this directly.

We still set up the indexes array (for now) after we call this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
31d28bad86 askrene: remove max_deliverable cache from increase_flows.
Make it calculate on demand.  This will be useful when we call it from elsewhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
e120202120 askrene: fix use-after-free if remove_htlc_min_violations fails.
It can only fail on overflow, but if it did, the fail path frees working_ctx
and returns "error_message".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
b5b4dbf2c2 askrene: fix error path if we fail sanity checks.
We've already freed the working_ctx, and the fail path does that again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
b6c220457f askrene: remove overzealous cache of channel_data.
This is not worth optimizing that I can see.  Using a non-debug build I get
the following times for tests/test_askrene.py::test_real_data

Before:
	143 seconds

After:
	141 seconds.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
35f65c5d91 common: add amount_msat_deduct / amount_msat_deduct_sub.
I added amount_msat_accumulate for the "a+=b" case, but I was struggling
with a name for the subtractive equivalent.  After some prompting, ChatGPT
suggested deduct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
Rusty Russell
0b8239f3d6 lightningd: fix access to bcli plugin once it's freed.
```
2025-11-13T07:47:16.5733646Z Valgrind error file: valgrind-errors.46048
2025-11-13T07:47:16.5733739Z ==46048== Invalid read of size 8
2025-11-13T07:47:16.5733869Z ==46048==    at 0x2F2B47: strmap_add_ (strmap.c:69)
2025-11-13T07:47:16.5734014Z ==46048==    by 0x212066: plugin_request_send (plugin.c:2509)
2025-11-13T07:47:16.5734160Z ==46048==    by 0x18FA54: bitcoin_plugin_send (bitcoind.c:123)
2025-11-13T07:47:16.5734329Z ==46048==    by 0x190AA7: bitcoind_getrawblockbyheight_ (bitcoind.c:537)
2025-11-13T07:47:16.5734490Z ==46048==    by 0x1917E3: bitcoind_getfilteredblock_ (bitcoind.c:857)
2025-11-13T07:47:16.5734615Z ==46048==    by 0x1BE2DC: get_txout (gossip_control.c:106)
2025-11-13T07:47:16.5734750Z ==46048==    by 0x1BE88A: gossip_msg (gossip_control.c:212)
2025-11-13T07:47:16.5734860Z ==46048==    by 0x21BD60: sd_msg_read (subd.c:560)
2025-11-13T07:47:16.5734962Z ==46048==    by 0x2E416D: next_plan (io.c:60)
2025-11-13T07:47:16.5735074Z ==46048==    by 0x2E4DC3: do_plan (io.c:422)
2025-11-13T07:47:16.5735175Z ==46048==    by 0x2E4E2A: io_ready (io.c:439)
2025-11-13T07:47:16.5735294Z ==46048==    by 0x2E727E: io_loop (poll.c:455)
2025-11-13T07:47:16.5735464Z ==46048==  Address 0x5ec7518 is 216 bytes inside a block of size 288 free'd
2025-11-13T07:47:16.5735692Z ==46048==    at 0x484B27F: free (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
2025-11-13T07:47:16.5735798Z ==46048==    by 0x2F767C: del_tree (tal.c:456)
2025-11-13T07:47:16.5735902Z ==46048==    by 0x2F795D: tal_free (tal.c:532)
2025-11-13T07:47:16.5736018Z ==46048==    by 0x20C30D: plugin_kill (plugin.c:468)
2025-11-13T07:47:16.5736147Z ==46048==    by 0x20D0EE: plugin_conn_finish (plugin.c:853)
2025-11-13T07:47:16.5736265Z ==46048==    by 0x2E6AF6: destroy_conn (poll.c:246)
2025-11-13T07:47:16.5736397Z ==46048==    by 0x2E6B1A: destroy_conn_close_fd (poll.c:252)
2025-11-13T07:47:16.5736504Z ==46048==    by 0x2F6FC9: notify (tal.c:246)
2025-11-13T07:47:16.5736602Z ==46048==    by 0x2F75AA: del_tree (tal.c:437)
2025-11-13T07:47:16.5736707Z ==46048==    by 0x2F795D: tal_free (tal.c:532)
2025-11-13T07:47:16.5736812Z ==46048==    by 0x2E4FF6: io_close (io.c:496)
2025-11-13T07:47:16.5736920Z ==46048==    by 0x2E72A5: io_loop (poll.c:459)
```

Changelog-None: exposed by recent plugin changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 09:14:19 +10:30
Chandra Pratap
350090a8c2 fuzz-tests: Add coverage increasing inputs to seed corpora
Improvements in the fuzz-testing scheme of `fuzz-initial_channel`
led to the discovery of test inputs that result in greater code
coverage. Add these inputs to the test's seed corpus.
2025-11-16 15:17:30 +10:30
Chandra Pratap
84b95297c8 fuzz-tests: Add test for untested function
Currently, `fuzz-initial_channel` doesn't verify the function
`channel_update_fundinng()` in its target file,
`common/initial_channel.h`.

Add a test for it.
2025-11-16 15:17:30 +10:30
Chandra Pratap
0980d10442 fuzz-tests: Prevent memory leak in fuzz-initial_channel
Changelog-None: The current test can leak memory due to improper
cleanup in the case of an early return. Fix it.
2025-11-16 15:17:30 +10:30
ShahanaFarooqui
616fde5254 script: Add mtime constant for reproducible Fedora tarball
Changelog-Fixed: Fedora releases are also deterministic now.
2025-11-15 08:35:25 -08:00
ShahanaFarooqui
8fe749e881 gitignore: Add generated files to gitignore 2025-11-15 05:25:25 +05:30
Chandra Pratap
0cabd46c4f fuzz-tests: Add a seed corpus for the new test
Add a minimal input set as a seed corpus for the newly introduced
test. This leads to discovery of interesting code paths faster.
2025-11-14 11:50:16 +10:30
Chandra Pratap
a59d21dcd4 fuzz-tests: Add a wire test for wireaddr functions
Changelog-None: `towire_wireaddr()` and `fromwire_wireaddr()` in
`common/wireaddr.h` are responsible for marshalling/unmarshalling
BOLT #7 address descriptors.

Since these aren't tested by the existing wire fuzz tests, add a
roundtrip test for them. This has the added benefit of testing
`parse_wireaddr()` as well.
2025-11-14 11:50:16 +10:30
Rusty Russell
103cd1c412 common: fix bad formatting for DNS-type wireaddr.
Found by @Chand-ra fuzzing test!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-14 11:50:16 +10:30
Rusty Russell
0d93db77bc fuzz: when running as unit tests, allow -v and [corpus...] args.
Hacky parser, not a real one, but this is for devs, so they can clean
it up with ccan/opt themselves if the want to be fancy! 🎩

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-14 11:50:16 +10:30
Chandra Pratap
e95e5f970e fuzz-tests: Add a seed corpus for the new test
Add a minimal input set as a seed corpus for the newly introduced
test. This leads to discovery of interesting code paths faster.
2025-11-14 09:08:26 +10:30
Chandra Pratap
afdeec46c5 fuzz-tests: Add a test for peer_init_received()
Changelog-None: `peer_init_received()` in `connectd/peer_exchange_initmsg.{c, h}`
is responsible for handling `init` messages defined in BOLT #1. Since it deals
with untrusted input, add a test for it.
2025-11-14 09:08:26 +10:30
daywalker90
838909634e ci: add workflow to create a PR for crate bumps
Changelog-None
2025-11-14 09:02:35 +10:30
daywalker90
ff2b77a57f crates: centralize version management 2025-11-14 09:02:35 +10:30
Rusty Russell
b4eda94ed3 pyln-testing: introduce canned blocks support to bitcoind fixture.
We have to add a send_and_mine_block() for cases where we want to get
a txid and then mine it (for canned blocks, we mine it then figure out
which tx it was!).

And fix up out-by-one in saving blocks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:21:29 +10:30
Rusty Russell
845bb30f46 lightningd: add --dev-ignore-idb to not complain about bitcoind in initialblockdownload.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:21:29 +10:30
Rusty Russell
acbfaa00f4 topology: in deterministic mode, only return one best candidate for listincoming.
This ensures that bolt11/bolt12 selection of routehints/blinded paths is always the same.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:21:29 +10:30
Rusty Russell
8c2116a123 wallet: make utxo order deterministic if CLN_DEV_ENTROPY_SEED set.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:21:29 +10:30
Rusty Russell
633fd21e00 lightningd: introduce some changes for dual open id randomness even with CLN_DEV_ENTROPY_SEED.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:21:29 +10:30
Rusty Russell
8b9020d7b9 global: use clock_time in place of time_now().
Except for tracing, that sticks with time_now().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:21:29 +10:30
Rusty Russell
dc9b425035 common/clock_time: wrapper for time_now() so we can override it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:21:29 +10:30
Rusty Russell
522457a12b connectd, gossipd, pay, bcli: use timemono when solely measuring duration for timeouts.
This is immune to things like clock changes, and has the convenient side-effect that
it will *not* be overridden when we override time for developer purposes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:21:29 +10:30
Rusty Russell
806dc89cad gossipd: remove --dev-gossip-time setting, we'll use CLN_DEV_SET_TIME.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:21:29 +10:30
Rusty Russell
0b2b92ffe3 pseudorand: make the results in deterministic mode per-caller.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:21:29 +10:30
Rusty Russell
f8fd97fb5d global: replace randombytes_buf() with randbytes() wrapper.
This allows us to override it for deterministic results.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:21:29 +10:30
Rusty Russell
2086699b70 common: add randbytes() wrapper to override cryptographic entropy: $CLN_DEV_ENTROPY_SEED
Only in developer mode, ofc.

Notes:
1. We have to move the initialization before the lightningd main trace_start,
   since that uses pseudorand().
2. To make the results stable, we need to use per-caller values to randbytes().
   Otherwise external timing changes the call order.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:21:29 +10:30
Rusty Russell
7ff0239f6f lightningd: db migration to clean up any pending payments where theres no htlc.
Changelog-Fixed: JSON-RPC: `listpays`/`listsendpays` erroneously left `pending` in xpay are cleaned up.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:19:12 +10:30
Rusty Russell
e68e9bd259 pytest: add test that we fixup "pending" payments which don't actually have HTLCs.
And don't fix up a genuine pending one!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:19:12 +10:30
Dusty Daemon
c29db2458e splice: Update splice_info tlv to use odd version
Update the tlv to use the type referenced in the spec, which also makes marks the tlv as optional.

Changelog-EXPERIMENTAL: --experimental-splicing: if you have splicing enabled on a channel both nodes must upgrade in unison due to updating commitment_signeds splice_info tlv's type.
2025-11-13 15:37:31 +10:30
Joseph Goulden
1ead6791fd chore: nix flake update
Switched to nixpkgs-unstable
Replaced postgresql dependency with much smaller libpq.
Utilise new inputs.self.submodules feature to simplify flake use
Moved apps to a separate file
Changelog-None
2025-11-13 15:35:51 +10:30
Rusty Russell
f58079e559 CI: Remove non-working FreeBSD test.
It takes over an hour to run, because there's no FreeBSD wheel for the cryptography Python
package, and it builds it, in Rust, from scratch.

And eventually, takes over 125 minutes and times out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 15:33:02 +10:30
Dusty Daemon
a45189cc70 splice: Update to current spec
Updating splice related reestablish code to
https://github.com/lightning/bolts/pull/1289
and
https://github.com/lightning/bolts/pull/1160

Changelog-Changed: Breaking change -- if you have splicing enabled on a channel both nodes must upgrade in unison due to updating `channel_reestablish` for to new splice specifications
2025-11-13 15:17:19 +10:30
Rusty Russell
add398f5ea xpay: wait, if final node gives us an indication we're behind on blockheight.
This doesn't happen much in real life, but it's certainly possible, so do what pay does here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/8612
Changelog-Added: `xpay` will now wait if it suspects a payment failure is due to a height disagreement with the final node.
2025-11-13 15:15:57 +10:30