33 Commits

Author SHA1 Message Date
Rusty Russell
e8fd235d4e common: move gossip_store_wire.csv into common/ from gossipd/
It's used by common/gossip_store.c, which is used by many things other than
gossipd.  This file belongs in common.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2026-02-16 17:23:33 +10:30
Rusty Russell
6e5cb299dd global: remove unnecessary includes from C files.
Basically, `devtools/reduce-includes.sh */*.c`.

Build time from make clean (RUST=0) (includes building external libs):

Before:
	real    0m38.944000-40.416000(40.1131+/-0.4)s
	user    3m6.790000-17.159000(15.0571+/-2.8)s
	sys     0m35.304000-37.336000(36.8942+/-0.57)s
After:
	real    0m37.872000-39.974000(39.5466+/-0.59)s
	user    3m1.211000-14.968000(12.4556+/-3.9)s
	sys     0m35.008000-36.830000(36.4143+/-0.5)s

Build time after touch config.vars (RUST=0):

Before:
	real    0m19.831000-21.862000(21.5528+/-0.58)s
	user    2m15.361000-30.731000(28.4798+/-4.4)s
	sys     0m21.056000-22.339000(22.0346+/-0.35)s

After:
	real    0m18.384000-21.307000(20.8605+/-0.92)s
	user    2m5.585000-26.843000(23.6017+/-6.7)s
	sys     0m19.650000-22.003000(21.4943+/-0.69)s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-23 06:44:04 +10:30
Rusty Russell
e120f87083 Makefile: create a library containing common, wire and bitcoin objects.
This means we don't have to manually choose what to link against,
which is much of the complexity of our Makefiles: the compiler will
automatically use any object files it needs to link.

We already do this for ccan as libccan.a, now we have libcommon.a.

We don't link against it for *everything*, as some tests require their own
versions.

Notes:
1. I get rid of the weird plugins/test/Makefile2 (accidental commit?)
2. Many tests change due to update-mocks.
3. In some places I added the missing dependency on the Makefile itself, though most are in the next
   patch.

Before:
	Total program size:     221366528
	Total tests size:       364243856

After:
	Total program size:     190733656
	Total tests size:       337880888

Build time from make clean (RUST=0) (includes building external libs):

Before:
	real    0m38.227000-44.245000(41.8222+/-1.6)s
	user    3m2.105000-33.696000(23.1442+/-8.4)s
	sys     0m35.054000-42.269000(39.7231+/-2)s
After:
	real    0m38.944000-40.416000(40.1131+/-0.4)s
	user    3m6.790000-17.159000(15.0571+/-2.8)s
	sys     0m35.304000-37.336000(36.8942+/-0.57)s

Build time after touch config.vars (RUST=0):

Before:
	real    0m18.928000-22.776000(21.5084+/-1.1)s
	user    2m8.613000-36.567000(27.7281+/-7.7)s
	sys     0m20.458000-23.436000(22.3963+/-0.77)s

After:
	real    0m19.831000-21.862000(21.5528+/-0.58)s
	user    2m15.361000-30.731000(28.4798+/-4.4)s
	sys     0m21.056000-22.339000(22.0346+/-0.35)s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

rusty@rusty-Framework:~/devel/cvs/lightni
2025-10-23 06:44:04 +10:30
Rusty Russell
907f7c6698 unit tests: update all the gossmaps to have the GOSSIP_STORE_COMPLETED_BIT set.
Mostly this meant running them, then running devtools/convert-gossmap and replacing the code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-01 13:29:33 +09:30
Chandra Pratap
5365b44e1b plugins/test: Add a test to trigger the bug
Add a test in `plugins/test/run-route-calc.c` that reproduces the
runtime error when the fix is not applied.
2025-09-22 10:52:59 +09:30
Rusty Russell
5d5741e681 libplugin: correctly wrap notifications we send in the notification name.
All the core notifications changed over to wrapping the notification
fields in an object with the name of the notification, but notifications
from plugins were missed.

Changelog-Added: Plugins: `channel_hint_update`, `pay_failure` and `pay_success` notifications now have objects of the same name containing the expected fields.
Changelog-Deprecated: Plugins: `channel_hint_update`, `pay_failure` and `pay_success` notification fields outside the same-named object.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-18 10:01:07 +09:30
Matt Whitlock
9112c1d518 common/json_parse_simple: make convenience functions inline
json_tok_streq(…) and json_get_member(…) are convenience wrappers for
json_tok_strneq(…) and json_get_membern(…) respectively. Unfortunately, using
them incurs a performance penalty in the common case where they are called with
a string literal argument because the compiler is unable to substitute a
compile-time constant in place of the buried call to strlen(…).

For example,

	json_get_member(buf, tok, "example");

…will have worse performance than…

	json_get_membern(buf, tok, "example", strlen("example"));

…because the former is forced to scan over "example" at run-time to count its
length whereas the latter is able to elide the strlen(…) call at compile time.

Hoist these convenience functions up into common/json_parse_simple.h and mark
them as inline so that the compiler can elide the strlen(…) call in the common
case of calling these functions with a string literal argument.

Changelog-None
2025-08-14 17:53:39 +09:30
Rusty Russell
4c23cf296c libplugin: correctly mark that plugin_notification_end STEALS the stream.
And also slightly generalize: plugin_notification_start() can take any
tal ptr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 16:52:44 +09:30
Rusty Russell
cdb74434f2 trace: don't copy strings where we don't need to.
1. trace_span_start() is always called with a string literal, so
   no copy needed (and we can use a macro to enforce this).
2. trace_span_tag() name and value are always longer-lived than
   the span, so no need to copy these either.

Before:
	real	0m18.524000-19.100000(18.7674+/-0.21)s
	user	0m16.171000-16.833000(16.424+/-0.26)s
	sys	0m2.259000-2.400000(2.337+/-0.059)s

After:
	real	0m16.421000-18.407000(17.8128+/-0.72)s
	user	0m14.242000-16.041000(15.5382+/-0.67)s
	sys	0m2.179000-2.363000(2.273+/-0.061)s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-04-23 13:45:18 +09:30
Lagrang3
35f899a588 fetch max total htlc value from listpeerchannel
The parameter max_htlc_value_in_flight_msat stablished by peers on
channel opening (BOLT02) can now be retrived from the
gossmods_from_listpeerchannels API.

Adapted the corresponding callback functions in renepay and askrene to
take into account that value as a constraint to the value we can send
through a channel.

Changelog-Add: fetch max_htlc_value_in_flight_msat from gossmods_listpeerchannels API

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-21 17:03:36 -06:00
Rusty Russell
307dbe3e62 tests: put proper checksums into test gossip_store files.
We're about to test them in gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
fdfc7ce62f gossmap: add (and use) logging hook.
Default goes to stderr for LOG_UNUSUAL and higher.

We have to whitelist more cases in map_catchup so we don't spam the logs
with perfectly-expected (but ignored) messages though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Jesse de Wit
087a29b0b3 libplugin-pay: trace payment_continue
Changelog-Added: Plugins: `pay` now has tracing support for various payment steps.
2024-11-13 13:15:52 +01:00
Christian Decker
f36be4b006 plugin: Add tracing support for send_outreq
The `send_outreq` function is a good place to suspend and resume
traces, since these are usually the places where we hand off control
back to the `io_loop`. This assumes that we do not continue doing
heavy liftin after we have queued an `outreq` call, but that is most
likely the case anyway. This frees us from having to track suspensions
whenever we call the RPC from a plugin.
2024-11-13 13:15:52 +01:00
Rusty Russell
c797b6fb20 libplugin: add method string to jsonrpc callbacks, implement generic helpers.
Without knowing what method was called, we can't have useful general logging
methods, so go through the pain of adding "const char *method" everywhere,
and add:

1. ignore_and_complete - we're done when jsonrpc returned
2. log_broken_and_complete - we're done, but emit BROKEN log.
3. plugin_broken_cb - if this happens, fail the plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-07 17:04:35 +10:30
Rusty Russell
9f92af9fab libplugin: use NULL to imply "take prefix from cmd".
This avoids jsonrpc_request_start() double-evaluating its cmd arg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-07 17:04:35 +10:30
Rusty Russell
c5099b1647 libplugin: clean up API.
When we used to allow cmd to be NULL, we had to hand the plugin
everywhere.  We no longer do.

1. Various jsonrpc_ functions no longer need the plugin arg.
2. send_outreq no longer needs a plugin arg.
3. The init function takes a command, not a plugin.
4. Remove command_deprecated_in_nocmd_ok.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-07 17:04:35 +10:30
Rusty Russell
6a3e9d91f9 libplugin: insist on always having a non-NULL command context.
And remove command_done() which was used when there was no
cmd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-07 17:04:35 +10:30
Rusty Russell
ec8293d215 libplugin-pay: always use a non-NULL struct command.
This means we replace p->cmd with an auxillary command after we've
finished, so we have a valid command to use.

It also means we weave `struct command_result` returns back through
all the callers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-07 17:04:35 +10:30
Rusty Russell
4ee9d1d2f2 gossmap: include cltv_expiry_delta in gossmap_chan_get_update_details for completeness.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-10-15 09:58:04 +10:30
Jesse de Wit
cc1362ead3 libplugin-pay: use map for channel hints
For nodes with many channels this is a tremendous improvement in pay
performance. PR #7611 improves payment performance from 15 seconds to
13.5 seconds on one of our nodes. This commit improves payment
performance from 13.5 seconds to 5.7 seconds.

Changelog-Fixed: Improved pathfinding speed for large nodes.
2024-10-07 15:16:46 +02:00
Christian Decker
50a0321759 pay: Use the global channel_hint_set and remember across payments 2024-10-07 14:05:47 +02:00
Christian Decker
904eb3795c pay: Subscribe to the channel_hint_update notifications 2024-10-07 14:05:47 +02:00
Rusty Russell
a60063e763 common/gossmods_listpeerchannels: include channel capacity in callback.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-10-04 11:27:53 +09:30
Rusty Russell
3253623785 gossmods_from_listpeerchannels: use correct type for cltv_delta.
Doesn't matter now, but will with the next change where we want to
pass a pointer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-10-04 11:27:53 +09:30
Christian Decker
6a53cf288b libplugin: Add primitive to serialize channel_hint to JSON 2024-08-09 13:59:07 +09:30
Christian Decker
82afa8d38c pay: Add a pre-flight check for the spendable balance
Changelog-Added: pay: The pay plugin now checks whether we have enough spendable capacity before computing a route, returning a clear error message if we don't
2024-08-08 12:20:44 -07:00
Rusty Russell
1f9ddd24b2 libplugin: support filtering of outgoing commands.
This is crude, handing a raw JSON string, but it works.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Christian Decker
ff0b7488eb pay: Implement offset and chainlag computation when paying
This actually uses the `chainlag` to make unsynced payments possible.
2024-05-08 23:18:27 -05:00
Lagrang3
bb31856b80 remove is_local variable from listpeerchannels cb
is_local, ie. "is this side of the channel ours?" is not needed since we
can determine that predicate by evaluating
`scidd->dir == node_id_idx(self, peer)`
2024-04-03 10:07:13 +10:30
Lagrang3
56ac5eebef add more fields to gossmods_listpeerchannels
Add spendable/receivable and is_local fields to the callback function
used in gossmods_from_listpeerchannels. This allows to do more fine
grained use of the listpeerchannels call.
The first use case is renepay, for which we need to ignore the htlc_max
of the local channels only.
2024-04-03 10:07:13 +10:30
Rusty Russell
9450d46db1 bitcoin/short_channel_id: pass by copy everywhere.
It's a u64, we should pass by copy.  This is a big sweeping change,
but mainly mechanical (change one, compile, fix breakage, repeat).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 13:51:48 +10:30
Rusty Russell
65efa2ab18 common, pay: actually test Dijkstra and route finding.
Set up a simple line of channel pairs, where one should be preferred
over the other for our various reasons.  Make sure this works, both
using a low-level call to Dijkstra and at a higher level as the pay
plugin does.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-07 14:09:14 +01:00