Commit Graph

16470 Commits

Author SHA1 Message Date
daywalker90
2e7181d04f wss-proxy: replaced by a rust version
Changelog-Changed: wss-proxy.py was replaced by a rust version with support for multiple `wss-bind-addr`. If you install CLN from pre-compiled binaries you must remove the old wss-proxy directory first before installing CLN, usually
it is located in `/usr/local/libexec/c-lightning/plugins/wss-proxy`. If you compile from source `make` will take care of this automatically.
2025-07-24 12:42:06 -07:00
Rusty Russell
e398c63ea5 doc: fix typo and missing ws: prefix documentation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-23 13:52:25 -07:00
Lagrang3
420cff9350 askrene: mcf: trade granularity for performance
Speed in getroutes up by setting the granularity to 1000

Amount (sats) | speedup
-----------------------
          100 | 1.00
         1000 | 1.00
        10000 | 1.06
       100000 | 1.31
      1000000 | 2.64

Worst runtime of getroutes

Amount (sats) | before (ms) | after (ms)
--------------------------------------
          100 | 1507        | 761
         1000 | 2129        | 1214
        10000 | 1632        | 1043
       100000 | 2004        | 1150
      1000000 | 27170       | 3289

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-19 10:09:17 +09:30
Lagrang3
b7f3d7bcc4 askrene: prune un-used arcs
From the multiple arcs that derive from the same channel we consider
only those with the smallest cost such that the payment amount and HTLC
max can fit in their combined capacity, ie. we prune high cost arcs that
surely will never be used by the optimal solution.

This reduces the number of arcs in the graph approximately from 8 arcs
per channel to approximately 2 arcs per channel.

No pruning.
amount:		100 	1000 	10000 	100000 	1000000
channels:	104741	106163	106607	106654	106666
arcs:		837928	849304	852856	853232	853328

Prune, limit the channel capacity by its HTLC max
amount:		100 	1000 	10000 	100000 	1000000
channels:	104741	106163	106607	106654	106666
arcs:		255502	259314	260538	260676	260704

Prune, limit the channel capacity to the payment amount
amount:		100 	1000 	10000 	100000 	1000000
channels:	104741	106163	106607	106654	106666
arcs:		209482	216270	228618	295450	432468

Prune, limit the channel capacity to the payment amount and its HTLC max
amount:		100 	1000 	10000 	100000 	1000000
channels:	104741	106163	106607	106654	106666
arcs:		209480	212324	213242	215726	228018

This produces a slight speedup for MCF computations:

Amount (sats) | speedup
-----------------------
          100 | 1.89
         1000 | 1.77
        10000 | 1.25
       100000 | 1.25
      1000000 | 1.18

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-19 10:09:17 +09:30
Lagrang3
fae176f4cc common/amount: add ceil division operation on msat
Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-19 10:09:17 +09:30
Lagrang3
73f03318bd askrene: update the docs on auto.no_mpp_support
Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-18 15:13:41 +09:30
Lagrang3
25c88447fa askrene: add algorithm for single path routing
Changelog-Added: askrene: an optimal single-path solver has been added, it can be called using the developer option --dev_algorithm=single-path or by adding the layer "auto.no_mpp_support"

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-18 15:13:41 +09:30
Lagrang3
47b0f67d04 askrene: add internal API for single-path routes
The single path solver uses the same probability cost and fee cost
estimation of minflow. Single path routes computed this way are
suboptimal with respect to the MCF solution but still are optimal among
any other single path. Computationally is way faster than MCF, therefore
for some trivial payments it should be prefered.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-18 15:13:41 +09:30
Lagrang3
2a6eab2843 askrene: refactor MCF
Refactor MCF solver: remove structs linear_network and residual_network.
Prefer passing raw data to the helper functions.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-18 15:13:41 +09:30
Rusty Russell
725c417198 pytest: add debug print to diagnose flake in test_peer_anchor_push
Sometimes the feerate is 5 too high.  But without seeing the actual txs it's
looking at, it's hard to know why.

```
actual_feerate = 13005.66942869603, expected_feerate = 13000

    def check_feerate(nodes, actual_feerate, expected_feerate):
        # Feerate can't be lower.
        assert actual_feerate > expected_feerate - 2
        if actual_feerate >= expected_feerate + 2:
            if any([did_short_sig(n) for n in nodes]):
                return
        # Use assert as it shows the actual values on failure
>       assert actual_feerate < expected_feerate + 2
E       AssertionError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:24:35 +09:30
Rusty Russell
1b89ea6776 pytest: avoid bad gossip flake in test_htlc_no_force_close
l3 closes with l2 unilaterally, but l2 sends l1 the channel_announce after
that, when it has never seen it before.  The solution is to wait until
l1 has seen the channel before we start the rest of the test:

```
E           ValueError:
E           Node errors:
E            - lightningd-2: had warning messages
E            - lightningd-1: had bad gossip messages
...
lightningd-2 2025-07-09T07:55:59.778Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-connectd: peer_in WIRE_WARNING
lightningd-2 2025-07-09T07:55:59.778Z INFO    0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-connectd: Received WIRE_WARNING: WARNING: channel_announcement: no unspent txout 103x1x0
lightningd-1 2025-07-09T07:55:59.781Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-channeld-chan#1: billboard: Channel ready for use.
lightningd-1 2025-07-09T07:55:59.782Z TRACE   gossipd: channel_announcement: got reply for 103x1x0...
lightningd-1 2025-07-09T07:55:59.782Z TRACE   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-gossipd: Bad gossip order: channel_announcement: no unspent txout 103x1x0
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:24:35 +09:30
Rusty Russell
44ed1bcd7f pytest: really fix flake in test_reconnect_signed
```
>       l1.rpc.fundchannel(l2.info['id'], CHANNEL_SIZE)

tests/test_connection.py:667:
...
>           raise RpcError(method, payload, resp['error'])
E           pyln.client.lightning.RpcError: RPC call failed: method: fundchannel, payload: {'id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 'amount': 50000, 'announce': True}, error: {'code': -1, 'message': 'Disconnected', 'data': {'id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 'method': 'openchannel_update'}}
```

What happens here is complicated.

1. dualopend never processes the WIRE_COMMITMENT_SIGNED message, meaning it doesn't consider it worth
   trying to reconnect.
2. Normally, on disconnect, we give subds time to process packets and then notice the disconnect.
3. But, if we can another connection, we terminate the (old) subds immediately.
4. When lightningd transitions the channel from DUALOPEND_OPEN_INIT to DUALOPEND_OPEN_COMMIT_READY,
   it tells connectd this peer is important.
5. Normally, this causes a reconnect one second after hangup.
6. However, if we've already disconnected, it makes connectd reconnect immediately.  This causes
   the test flake, if dualopend hasn't processed the message, and thus changed the state to
   DUALOPEND_OPEN_COMMITTED: we don't reconnect and instead fail the fundchannel() call.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:24:35 +09:30
Rusty Russell
3e679156ff pytest: fix flake in test_splicing_rbf
We get a unilateral close if the blocks come too fast:

```
>       l2.daemon.wait_for_log(r'CHANNELD_AWAITING_SPLICE to CHANNELD_NORMAL')

tests/test_splicing.py:96:
...
>                   raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E                   TimeoutError: Unable to find "[re.compile('CHANNELD_AWAITING_SPLICE to CHANNELD_NORMAL')]" in logs.
```

Here:

```
lightningd-2 2025-07-08T12:08:30.430Z DEBUG   lightningd: Adding block 114: 542499dfc63bc94c0ba0a3d81bdaff07ab73ba9aebddd4c2ae4b2e0c1e8bd15f
...
lightningd-2 2025-07-08T12:08:30.459Z UNUSUAL 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer permanent failure in CHANNELD_AWAITING_SPLICE: Fulfilled HTLC 1 RCVD_REMOVE_REVOCATION cltv11 4 hit deadline (reason=protocol)
lightningd-2 2025-07-08T12:08:30.469Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-channeld-chan#1: Status closed, but not exited. Killing
lightningd-2 2025-07-08T12:08:30.475Z INFO    0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: State changed from CHANNELD_AWAITING_SPLICE to AWAITING_UNILATERAL
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:24:35 +09:30
Lagrang3
34cf3334f8 askrene: add a dev parameter to switch algorithm
Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-11 12:14:45 +09:30
Rusty Russell
4500818312 askrene: fix up getroutes_info struct.
It was originally a wrapper for JSON param() results, but now it's a more
generic struct.  So make it clear that the fields are not optional.  This
means a manual assignment in the initial population of this struct, but
all the users are now far clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:14:45 +09:30
Rusty Russell
2a5239589e askrene: move get_routes into do_getroutes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:14:45 +09:30
Rusty Russell
71525173a0 askrene: extract getroutes json formatting function.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:14:45 +09:30
Rusty Russell
98e30f2c42 askrene: extracet convert_flows_to_routes into its own function.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:14:45 +09:30
Rusty Russell
2a8334b06e plugins/askrene: extract apply_layers into its own function.
Simple refactoring.

Stolen entirely from Eduardo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:14:45 +09:30
Rusty Russell
f1fa498663 askrene: houst struct getroutes_info higher to clarify following patches.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-11 12:14:45 +09:30
Lagrang3
88bb1636bc askrene: refactor get_routes ...
Move the feature tuning algorithm to mcf.c, ie. the loops for searching
a good mu and delay_feefactor to satisfy the problem constraints.

We are looking to set the stage for an execution logic that allows for
multiple choices of routing algorithms, mainly for experimenting without
breaking the default working code.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-11 12:14:45 +09:30
ShahanaFarooqui
cea6c81181 doc: App development updated with standard connection URI format
Changelog-None.
2025-07-11 10:51:26 +09:30
Lagrang3
266a5f9e85 askrene: add runtime of getroutes to the logs
Add log information about the runtime of getroutes.

Changelog-None: askrene: add runtime of getroutes to the logs

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-10 23:37:06 +09:30
daywalker90
07e2a77be7 cln-plugin: don't panic if notification handler returns error
Changelog-None
2025-07-10 23:33:26 +09:30
ShahanaFarooqui
87d862992a doc: Remove c-lightning-REST documentation
Changelog-None
2025-07-10 15:51:42 +09:30
Rusty Russell
89c4201918 plugins/sql: fix doc formatting for tables with multiple indices.
Before vs after:

   - `forwards` indexed by `in_channel and in_htlc_id` (see lightning-listforwards(7))
   - `forwards` indexed by `in_channel` and `in_htlc_id` (see lightning-listforwards(7))

And:
   - `htlcs` indexed by `short_channel_id and id` (see lightning-listhtlcs(7))
   - `htlcs` indexed by `short_channel_id` and `id` (see lightning-listhtlcs(7))

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-09 14:13:31 +09:30
Chandra Pratap
de5e4fe9ca fuzz-tests: Add a seed corpus for the new test
Add a minimal input set as a seed corpus for the newly introduced
test. This leads to discovery of interesting code paths faster.
2025-07-08 14:47:36 +09:30
Chandra Pratap
61bf4ffc35 fuzz-tests: Add differential test for HMAC-SHA256
Changelog-None: Add a differential fuzz test for
HMAC-SHA256, similar to those for SHA256 and RIPEMD160,
to verify CCAN’s implementation against OpenSSL’s.
2025-07-08 14:47:36 +09:30
Alex Myers
0e080a1c22 devtools/credit: add aliases of previous namers
It was suggesting past namers as new namers.

Also fixed the count extraction so that @21M4TW would be
properly credited.

Changelog-None
2025-07-08 12:56:17 +09:30
Chandra Pratap
0651b37ae4 fuzz-tests: Add coverage-increasing inputs to seed corpora
Change in the fuzz-testing scheme of fuzz-base32-64 led to
the discovery of test inputs that result in greater in
code-coverage. Add these inputs to the test's seed corpus.
2025-07-08 12:54:56 +09:30
Chandra Pratap
e721b0a89a fuzz-tests: Enhance b64_encode() validation with roundtrip decoding
Changelog-None: Currently, fuzz testing for b64_encode() merely
encodes input and frees the result, providing no real verification
of its behavior.

Introduce a new b64_decode() function (modeled after b32_decode())
and update the fuzz test to perform a roundtrip—encoding followed
by decoding—to ensure that b64_encode() correctly preserves the
original data.
2025-07-08 12:54:56 +09:30
Dusty Daemon
b4d0da5ff4 logs: A basic javascript log viewer
A basic javascript tool for filtering through large CLN log files.

Changelog-Added: A new tool for rendering CLN log files in the browser.
2025-07-08 12:24:24 +09:30
Matt Whitlock
cc0f66f07b hsmtool: implement new "derivetoremote" method
This method has a similar purpose as "guesstoremote" but is for use when
the channel's database ID is known. It produces the private key that can
spend the to_remote output of the peer's commitment transaction. It
assumes the channel was negotiated with option_static_remotekey unless
the optional per-commitment point argument is provided.

Changelog-Added: hsmtool has a new `derivetoremote` method.
2025-07-08 12:21:24 +09:30
Erick Cestari
6724db65de BOLT11: Make payment secret field ('s') mandatory
Make the payment secret field ('s') mandatory for BOLT11 payment requests,
implementing the requirement specified in BOLT11 spec PR #1242
(https://github.com/lightning/bolts/pull/1242).

This security enhancement prevents payment probing attacks by requiring
all invoices to include payment secrets. Changes include:

1. Adding validation in bolt11_decode_nosig() to reject invoices without
   the 's' field
2. Adding payment secrets to all test vectors
3. Updating expected encoded values in test cases to include payment secrets
4. Adding a specific test case that verifies proper rejection of invoices
   missing the payment secret field

Changelog-Changed: Made payment secret ('s' field) mandatory in BOLT11 payment requests for improved security.
2025-07-07 11:18:37 +09:30
Rusty Russell
66b6c111c2 pytest: fix up bolt11 tests to include s field.
It's about to be made compulsory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-07 11:18:37 +09:30
Rusty Russell
2b023be022 bolt11: don't accept wrong-length p, h, s or n fields.
Spec got stricter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-07 11:18:37 +09:30
Rusty Russell
17b3fb5196 common/bolt11: meet the new tighter parsing requirements.
These checks are a SHOULD, but implementing them helps avoid anyone
making such weird things in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-07 11:18:37 +09:30
Rusty Russell
c0ff1b5768 lightningd: update comments about known preimages for missing HTLC outputs.
Doesn't change anything for us, since we will already fulfull the incoming
HTLCs if we can, but good to note.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-07 11:18:37 +09:30
Rusty Russell
9298e5c212 wire: use real peer_storage messages now their format is fixed.
They weren't formatted correctly for bolts/tools/extract-formats.py
until this commit, so we had to patch them in manually.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-07 11:18:37 +09:30
Rusty Russell
0cd6efe585 wire: update splice patch to produce expected peer_wire.csv.
Dusty patched peer_wire.csv directly, breaking regeneration `make extract-peer-csv`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-07 11:18:37 +09:30
Rusty Russell
d86b88d409 channeld: remove experimental-upgrade-protocol.
It was already disabled by Dusty due to a number conflict with splicing, and
the proposal probably needs updating to use quiescence now that is merged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: The non-functional `experimental-upgrade-protocol` config option.
2025-07-07 11:18:37 +09:30
Rusty Russell
c50bd381ec doc: recommend injectpaymentonion in place of sendonion.
Among other things, injectpaymentonion insists on knowing the destination
amount_msat.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-06-30 13:23:26 +09:30
Rusty Russell
89f411ca79 doc: be explicit (with tests!) on when we don't know amount_msat for listsendpays.
Sangbida and I traced back through ancient history: when the pay plugin was introduced
in 0.9.0 (2019!) it already used the amount_msat parameter (then called `msatoshi`),
so this case effectively "never happens".

But we added a test for it just in case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-06-30 13:23:26 +09:30
ShahanaFarooqui
bd743d741e doc: Change commando plugin title
Changelog-None.
2025-06-27 11:08:16 +09:30
ShahanaFarooqui
d52e3c394b doc: Update image on App Development page 2025-06-27 11:08:16 +09:30
ShahanaFarooqui
94bdad19f0 doc: Update json-rpc-api categories due to updated RPC links 2025-06-27 11:08:16 +09:30
ShahanaFarooqui
e5ec688bf4 doc: Fix links to avoid 404 error 2025-06-26 14:21:56 +09:30
ShahanaFarooqui
adbe634453 doc: Update Commando plugin details and link 2025-06-26 14:21:56 +09:30
ShahanaFarooqui
2d73186a1e doc: Move deprecated features page from API reference list to Developer's guide 2025-06-26 14:21:56 +09:30
ShahanaFarooqui
5cc28a84eb docs: Fix script used to update versions in point release instructions
Changelog-None.
2025-06-26 13:58:51 +09:30