Commit Graph

1973 Commits

Author SHA1 Message Date
daywalker90
bf37d41c7e clnrest: add more valid request and response types
Changelog-Added: clnrest can now return successful responses as xml, yaml, or form-encoded in addition to json defined in the 'Accept' header. The same goes for request types defined in the 'Content-type' header.
2025-07-25 08:18:44 -07:00
daywalker90
2e7181d04f wss-proxy: replaced by a rust version
Changelog-Changed: wss-proxy.py was replaced by a rust version with support for multiple `wss-bind-addr`. If you install CLN from pre-compiled binaries you must remove the old wss-proxy directory first before installing CLN, usually
it is located in `/usr/local/libexec/c-lightning/plugins/wss-proxy`. If you compile from source `make` will take care of this automatically.
2025-07-24 12:42:06 -07:00
Lagrang3
420cff9350 askrene: mcf: trade granularity for performance
Speed in getroutes up by setting the granularity to 1000

Amount (sats) | speedup
-----------------------
          100 | 1.00
         1000 | 1.00
        10000 | 1.06
       100000 | 1.31
      1000000 | 2.64

Worst runtime of getroutes

Amount (sats) | before (ms) | after (ms)
--------------------------------------
          100 | 1507        | 761
         1000 | 2129        | 1214
        10000 | 1632        | 1043
       100000 | 2004        | 1150
      1000000 | 27170       | 3289

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-19 10:09:17 +09:30
Lagrang3
b7f3d7bcc4 askrene: prune un-used arcs
From the multiple arcs that derive from the same channel we consider
only those with the smallest cost such that the payment amount and HTLC
max can fit in their combined capacity, ie. we prune high cost arcs that
surely will never be used by the optimal solution.

This reduces the number of arcs in the graph approximately from 8 arcs
per channel to approximately 2 arcs per channel.

No pruning.
amount:		100 	1000 	10000 	100000 	1000000
channels:	104741	106163	106607	106654	106666
arcs:		837928	849304	852856	853232	853328

Prune, limit the channel capacity by its HTLC max
amount:		100 	1000 	10000 	100000 	1000000
channels:	104741	106163	106607	106654	106666
arcs:		255502	259314	260538	260676	260704

Prune, limit the channel capacity to the payment amount
amount:		100 	1000 	10000 	100000 	1000000
channels:	104741	106163	106607	106654	106666
arcs:		209482	216270	228618	295450	432468

Prune, limit the channel capacity to the payment amount and its HTLC max
amount:		100 	1000 	10000 	100000 	1000000
channels:	104741	106163	106607	106654	106666
arcs:		209480	212324	213242	215726	228018

This produces a slight speedup for MCF computations:

Amount (sats) | speedup
-----------------------
          100 | 1.89
         1000 | 1.77
        10000 | 1.25
       100000 | 1.25
      1000000 | 1.18

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-19 10:09:17 +09:30
Lagrang3
25c88447fa askrene: add algorithm for single path routing
Changelog-Added: askrene: an optimal single-path solver has been added, it can be called using the developer option --dev_algorithm=single-path or by adding the layer "auto.no_mpp_support"

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-18 15:13:41 +09:30
Lagrang3
47b0f67d04 askrene: add internal API for single-path routes
The single path solver uses the same probability cost and fee cost
estimation of minflow. Single path routes computed this way are
suboptimal with respect to the MCF solution but still are optimal among
any other single path. Computationally is way faster than MCF, therefore
for some trivial payments it should be prefered.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-18 15:13:41 +09:30
Lagrang3
2a6eab2843 askrene: refactor MCF
Refactor MCF solver: remove structs linear_network and residual_network.
Prefer passing raw data to the helper functions.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-18 15:13:41 +09:30
Lagrang3
34cf3334f8 askrene: add a dev parameter to switch algorithm
Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-11 12:14:45 +09:30
Rusty Russell
4500818312 askrene: fix up getroutes_info struct.
It was originally a wrapper for JSON param() results, but now it's a more
generic struct.  So make it clear that the fields are not optional.  This
means a manual assignment in the initial population of this struct, but
all the users are now far clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:14:45 +09:30
Rusty Russell
2a5239589e askrene: move get_routes into do_getroutes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:14:45 +09:30
Rusty Russell
71525173a0 askrene: extract getroutes json formatting function.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:14:45 +09:30
Rusty Russell
98e30f2c42 askrene: extracet convert_flows_to_routes into its own function.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:14:45 +09:30
Rusty Russell
2a8334b06e plugins/askrene: extract apply_layers into its own function.
Simple refactoring.

Stolen entirely from Eduardo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:14:45 +09:30
Rusty Russell
f1fa498663 askrene: houst struct getroutes_info higher to clarify following patches.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:14:45 +09:30
Lagrang3
88bb1636bc askrene: refactor get_routes ...
Move the feature tuning algorithm to mcf.c, ie. the loops for searching
a good mu and delay_feefactor to satisfy the problem constraints.

We are looking to set the stage for an execution logic that allows for
multiple choices of routing algorithms, mainly for experimenting without
breaking the default working code.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-11 12:14:45 +09:30
Lagrang3
266a5f9e85 askrene: add runtime of getroutes to the logs
Add log information about the runtime of getroutes.

Changelog-None: askrene: add runtime of getroutes to the logs

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-10 23:37:06 +09:30
daywalker90
07e2a77be7 cln-plugin: don't panic if notification handler returns error
Changelog-None
2025-07-10 23:33:26 +09:30
Rusty Russell
89c4201918 plugins/sql: fix doc formatting for tables with multiple indices.
Before vs after:

   - `forwards` indexed by `in_channel and in_htlc_id` (see lightning-listforwards(7))
   - `forwards` indexed by `in_channel` and `in_htlc_id` (see lightning-listforwards(7))

And:
   - `htlcs` indexed by `short_channel_id and id` (see lightning-listhtlcs(7))
   - `htlcs` indexed by `short_channel_id` and `id` (see lightning-listhtlcs(7))

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-09 14:13:31 +09:30
Rusty Russell
50e4aabe18 commando: remove old commando rune commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: `commando-rune`, `commando-listrunes`, `commando-blacklist` (deprecated v23.08, disabled v25.05).
2025-06-25 09:06:31 +09:30
Rusty Russell
09e03e98ef deprecations: update v25.11 to v25.12.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-06-24 09:57:24 +09:30
Alex Myers
2b8b709bfd meta: update changelog for v25.05
Changelog-None
2025-06-16 13:10:33 -05:00
daywalker90
40d319b7be python: raise minimum supported python version to 3.9
Changelog-None
2025-06-12 13:51:18 +02:00
Alex Myers
8d6ae149ff release: update version for 25.05rc1
Changelog-None
2025-05-19 14:12:55 -05:00
Rusty Russell
44972b2e15 chanbackup: unbias our random peer selection.
We picked one node and iterated.  This means we would only sent to 1,
not 2 nodes if we picked the last (common if there were only a few
peers).  But it also means we would generally send to the same pair of
nodes, which isn't great for redundancy.

Rework to be more random.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
49ccb0eeb8 chanbackup: don't send our backups to peers if they would be overlength.
They will in fact get truncated, and never restore.  Large nodes should be using some real
backup strategy!

For this reason, we split "peer_backup" support into send vs store support, so we can
turn off send of our own without disabling storing/sending theirs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
c5cde13af6 chanbackup: only send channel backups to max 2 peers at a time.
This, too, is vital for large nodes, which don't want a significant
traffic increase.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
90bbd8aedc chanbackup: use peer table instead of calling "listpeerchannels".
This is much better for very large nodes with lots of channel activity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
4eafc1a2e1 chanbackup: cache available peers.
We already get the connected hook, so in there we can add to a hash
table of suitable peers.  Rather than subscribe to disconnection, we
simply remove the peer if a sendcustommsg fails.

This does make after_send_scb_single() a bit more complex, since it
needs this specific node_id: we use a `struct info` per node and a
pointer to a shared "idx" reference counter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
bda52c315c chanbackup: use the cache instead of calling listdatastore every time.
Now actually use our cache.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
dc2c3668e6 chanbackup: use local memory to store peer backups.
By keeping a local hash table, we won't have to look up every time.
We still write to the datastore when it changes, and we need to
initialize it at plugin start.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
2d821d6012 libplugin: fix jsonrpc_set_datastore_binary/jsonrpc_set_datastore_string with NULL cb.
This should complete the command, otherwise we get an assertion on
line 1090.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
adf7663fef chanbackup: only store backups for peers with current/previous channels.
This seems fair.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
cdce113d57 chanbackup: use modern plugin_data.
This avoids globals, which is a bit neater.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
f590962be2 chanbackup: less logging in preparation for default enable.
We should not send a message on unknown custom message receipt (we
expect this!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
e8690021ee chanbackup: remove unnecessary allocation.
after_listpeers() doesn't use the argument, so we don't need to allocate anything.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
9357954bd9 chanbackup: add spec quotes now we have this in the spec.
This is the only way to track spec changes in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Lagrang3
81417958d4 askrene-inform-channel: change order of operations
Change order of operations to ensure that askrene-inform-channel with
inform=constrained computes the correct upper bound on the liquidity:

    upperBound = reserves + amount - 1

With the previous order of operations in the case amount=0 we would get:

    upperBound = reserves

instead of
    upperBound = reserves - 1

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-05-13 19:14:57 +09:30
Dusty Daemon
0615c156f5 splice: Splice script should not abort on sign
The result of splice_signed can fail for many reasons that are non-critical (already in mempool for instance).

Don’t abort channels in this case as that causes a force close.
2025-05-13 14:52:15 +09:30
Rusty Russell
0824d12849 bcli: don't try asking non-full nodes for blocks.
Suggested-by: @NicolasDorier
Closes: https://github.com/ElementsProject/lightning/issues/8078
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: we no longer try asking for blocks from bitcoind's peers if they don't store blocks.
2025-05-12 10:38:25 +09:30
Rusty Russell
9ce3f5dde4 askrene: fix API breakage, add tests.
We cannot add new parameters in the middle, since we accept parameters by JSON
array as well as by dicts.  In fact, this broke tests, but due to unrelated
breakage in the GitHub "Automerge" functionality, it got applied as
556e38c838 ("askrene-bias-channel: bias call add
up.").

Also add tests, and a better Changelog line.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `askrene-bias-channel` now has a `relative` option to add, rather than replace, a channel bias.
2025-05-11 11:25:40 +09:30
Lagrang3
556e38c838 askrene-bias-channel: bias call add up.
The channel bias feature is not being used yet by any plugin, so this
hopefully doesn't break any working code.
When askrene-bias-channel is called the bias quantity is added on top of
any previous biased already present on that channel instead of
overwriting it.

Changelog-Changed: askrene-bias-channel: bias call add up.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-05-08 12:28:57 +09:30
Rusty Russell
c81ec4fe76 lightningd: deprecate (undocumented!) "unknown" old_state field in channel_state_changed notification.
Reported-by: daywalker90
Changelog-Deprecated: JSON-RPC: channel_state_changed notification field `old_state` value "unknown" (it will be omitted, instead)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-08 10:49:31 +09:30
Rusty Russell
74e7264d54 bitcoin: make input witness weight calculation explicit.
This is inspired by a patch from @whitslack, which overlapped with this series.
Most importantly, there was only one call to bitcoin_tx_simple_input_weight(),
and it is better to be explicit with that one.

This also changes our funder calculation to assume our own input is taproot,
which it is likely to be given we've defaulted to taproot for outputs for
change addresses since 23.08.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-06 12:27:53 +09:30
Rusty Russell
9af8c64ff4 txprepare: fix all amount when other amounts also specified.
This is legal!  And we actually do this in tests, but we didn't check the psbt
was spendable (the next patch does, indirectly, by testing feerate):

```
        # Discard prep4 and get all funds again
        l1.rpc.txdiscard(prep5['txid'])
        # You can have one which is all, but not two.
        prep5 = l1.rpc.txprepare([{addr: Millisatoshi(amount * 3 * 1000)},
                                  {addr: 'all'}])
        # Feerate should be ~ as we asked for
>       assert normal_feerate_perkw - 1 < feerate_from_psbt(bitcoind, l1, prep5['psbt']) < normal_feerate_perkw + 1
E       AssertionError: assert (7500 - 1) < -1091803.9574935874
```

Changelog-Fixed: JSON-RPC: `txprepare` with `all` as well as one or more non-all amount fields now produces a valid PSBT.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-06 12:27:53 +09:30
daywalker90
459c756040 cln-plugin: trace level logging support
Changelog-None
2025-05-05 09:31:22 -07:00
Lagrang3
47babbf661 askrene: add LOG_TRACE to every rpc entry point
This could be helpful for debugging the interaction of renepay/xpay with
askrene.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-05-02 13:59:13 -07:00
Lagrang3
0de781dc6d renepay: log renepay steps with LOG_TRACE
Previously it was flagged as LOG_DBG, but produces very verbose log
files.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-05-02 13:59:13 -07:00
Lagrang3
be6a140777 libplugin: fix LOG_TRACE for plugins
A log event LOG_TRACE submitted by a plugin was being logged as
**BROKEN** by lightningd before this commit.

Changelog-Fixed: plugins can now log events under the LOG_TRACE flag.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-05-02 13:59:13 -07:00
Alex Myers
a4e0091565 poetry: update all pyproject.toml files for poetry 2.0.1
and run poetry lock

Changelog-None
2025-05-02 13:39:33 -07:00
Peter Neuroth
ae226cbf7a lsps: Add additional documentation
Adds some additional documentation to the json-rpc v2.0 messages
and error types.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-05-02 13:34:21 -07:00