We use a wrapper around the MCF solver that takes care of finding the
best linearization parameters and fixing the flow values to meet the
htlc_min and htlc_max constraints.
We have reworked the current implementation and made it a bit more
similar to renepay's version.
Out of 50000 simulated payment situations distributed accross payment
amounts of 1e2, 1e3, 1e4, 1e5 and 1e6 sats, we find that 133 failed
cases in the master branch turn to success with the current changes,
while only 3 success cases in the master are not solved by the changes.
master
+-------+------+
| S | F |
+---+-------+------+
| S | 46329 | 133 |
changes +---+-------+------+
| F | 3 | 3535 |
+---+-------+------+
Out of the 133 cases that flipped from failure to success the failed
reasons were:
122 -> "Could not find route without excessive cost"
5 -> "We couldn't quite afford it"
5 -> "Amount *msat below minimum"
1 -> tripped an HTLC min check
Changelog-None.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
by a small amount if the deliver amount is less than the requested
amount by X.
This step saves runtime by avoiding calling an extra MCF
and it helps us solve a small percentage of cases where the only
available routes have HTLCmin that is bigger than X.
Changelog-None
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Disable channels with HTLC min violations so that we don't hit them
twice when computing routes.
Changelog-None
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Try a new way to refine flows,
ie. reduce excess due to MCF accuracy and HTLC max constraints
after hop amounts are computed with fees included.
Changelog-None.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
By policy, our stub hsmd accepts everything: openingd is supposed to
sort this out (or use VLS for a real HSM implementation!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Config: the node no longer crashes if you set `watchtime-blocks` to 0 (which is fine for testing: don't do this on mainnet!).
Improvements in the fuzz-testing scheme of
`fuzz-bolt12-invrequest-decode` led to the discovery of test inputs
that result in greater in code coverage.
Add these inputs to the test's seed corpus.
Changelog-None: Currently, the `BOLT #12` invrequest parsing test
only tests the invrequest decode function. Add a test for the
encoding function as well by making the test roundtrip.
When installing core lightning on macOS I found that the debug symbols were not being preserved leading to "no debug info in
Mach-O executable". This is because the .dSYM files, which contain the debug information, is generated in the build directory and could not be found by the installed binaries.
Changes:
- Add -fno-standalone-debug flag to CDEBUGFLAGS on macOS to reduce
dependency on absolute source paths
- Modify install-program target to copy .dSYM bundles alongside
binaries for BIN_PROGRAMS, PKGLIBEXEC_PROGRAMS, and PLUGINS
Testing:
1. ./configure --reconfigure
2. make install PREFIX=/tmp/lightning-install
3. make clean (removes all .dSYM files from build directory)
4. /tmp/lightning-install/bin/lightningd --help
5. Verified stack traces now work correctly without "no debug info" errors
On macOS, libbacktrace was failing to find debug information due to:
1. Debug symbols not being properly linked with dsymutil
2. Apple Clang 17.0.0 generating DWARF 5 which libbacktrace couldn't parse
In this commit we address both issues:
Debug symbol accessibility:
- Add dsymutil integration in Makefile to properly link debug symbols
- Use -fno-standalone-debug to embed debug info inline in executable
DWARF format compatibility:
- Force -gdwarf-4 instead of default DWARF 5 to avoid "DW_FORM_addrx value out of range" errors
Changelog-added: libbacktrace works with macOS
Changelog-Added: clnrest can now return successful responses as xml, yaml, or form-encoded in addition to json defined in the 'Accept' header. The same goes for request types defined in the 'Content-type' header.
Changelog-Changed: wss-proxy.py was replaced by a rust version with support for multiple `wss-bind-addr`. If you install CLN from pre-compiled binaries you must remove the old wss-proxy directory first before installing CLN, usually
it is located in `/usr/local/libexec/c-lightning/plugins/wss-proxy`. If you compile from source `make` will take care of this automatically.
From the multiple arcs that derive from the same channel we consider
only those with the smallest cost such that the payment amount and HTLC
max can fit in their combined capacity, ie. we prune high cost arcs that
surely will never be used by the optimal solution.
This reduces the number of arcs in the graph approximately from 8 arcs
per channel to approximately 2 arcs per channel.
No pruning.
amount: 100 1000 10000 100000 1000000
channels: 104741 106163 106607 106654 106666
arcs: 837928 849304 852856 853232 853328
Prune, limit the channel capacity by its HTLC max
amount: 100 1000 10000 100000 1000000
channels: 104741 106163 106607 106654 106666
arcs: 255502 259314 260538 260676 260704
Prune, limit the channel capacity to the payment amount
amount: 100 1000 10000 100000 1000000
channels: 104741 106163 106607 106654 106666
arcs: 209482 216270 228618 295450 432468
Prune, limit the channel capacity to the payment amount and its HTLC max
amount: 100 1000 10000 100000 1000000
channels: 104741 106163 106607 106654 106666
arcs: 209480 212324 213242 215726 228018
This produces a slight speedup for MCF computations:
Amount (sats) | speedup
-----------------------
100 | 1.89
1000 | 1.77
10000 | 1.25
100000 | 1.25
1000000 | 1.18
Changelog-None
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Changelog-Added: askrene: an optimal single-path solver has been added, it can be called using the developer option --dev_algorithm=single-path or by adding the layer "auto.no_mpp_support"
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
The single path solver uses the same probability cost and fee cost
estimation of minflow. Single path routes computed this way are
suboptimal with respect to the MCF solution but still are optimal among
any other single path. Computationally is way faster than MCF, therefore
for some trivial payments it should be prefered.
Changelog-None.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Refactor MCF solver: remove structs linear_network and residual_network.
Prefer passing raw data to the helper functions.
Changelog-None
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Sometimes the feerate is 5 too high. But without seeing the actual txs it's
looking at, it's hard to know why.
```
actual_feerate = 13005.66942869603, expected_feerate = 13000
def check_feerate(nodes, actual_feerate, expected_feerate):
# Feerate can't be lower.
assert actual_feerate > expected_feerate - 2
if actual_feerate >= expected_feerate + 2:
if any([did_short_sig(n) for n in nodes]):
return
# Use assert as it shows the actual values on failure
> assert actual_feerate < expected_feerate + 2
E AssertionError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
l3 closes with l2 unilaterally, but l2 sends l1 the channel_announce after
that, when it has never seen it before. The solution is to wait until
l1 has seen the channel before we start the rest of the test:
```
E ValueError:
E Node errors:
E - lightningd-2: had warning messages
E - lightningd-1: had bad gossip messages
...
lightningd-2 2025-07-09T07:55:59.778Z DEBUG 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-connectd: peer_in WIRE_WARNING
lightningd-2 2025-07-09T07:55:59.778Z INFO 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-connectd: Received WIRE_WARNING: WARNING: channel_announcement: no unspent txout 103x1x0
lightningd-1 2025-07-09T07:55:59.781Z DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-channeld-chan#1: billboard: Channel ready for use.
lightningd-1 2025-07-09T07:55:59.782Z TRACE gossipd: channel_announcement: got reply for 103x1x0...
lightningd-1 2025-07-09T07:55:59.782Z TRACE 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-gossipd: Bad gossip order: channel_announcement: no unspent txout 103x1x0
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
```
> l1.rpc.fundchannel(l2.info['id'], CHANNEL_SIZE)
tests/test_connection.py:667:
...
> raise RpcError(method, payload, resp['error'])
E pyln.client.lightning.RpcError: RPC call failed: method: fundchannel, payload: {'id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 'amount': 50000, 'announce': True}, error: {'code': -1, 'message': 'Disconnected', 'data': {'id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 'method': 'openchannel_update'}}
```
What happens here is complicated.
1. dualopend never processes the WIRE_COMMITMENT_SIGNED message, meaning it doesn't consider it worth
trying to reconnect.
2. Normally, on disconnect, we give subds time to process packets and then notice the disconnect.
3. But, if we can another connection, we terminate the (old) subds immediately.
4. When lightningd transitions the channel from DUALOPEND_OPEN_INIT to DUALOPEND_OPEN_COMMIT_READY,
it tells connectd this peer is important.
5. Normally, this causes a reconnect one second after hangup.
6. However, if we've already disconnected, it makes connectd reconnect immediately. This causes
the test flake, if dualopend hasn't processed the message, and thus changed the state to
DUALOPEND_OPEN_COMMITTED: we don't reconnect and instead fail the fundchannel() call.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We get a unilateral close if the blocks come too fast:
```
> l2.daemon.wait_for_log(r'CHANNELD_AWAITING_SPLICE to CHANNELD_NORMAL')
tests/test_splicing.py:96:
...
> raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E TimeoutError: Unable to find "[re.compile('CHANNELD_AWAITING_SPLICE to CHANNELD_NORMAL')]" in logs.
```
Here:
```
lightningd-2 2025-07-08T12:08:30.430Z DEBUG lightningd: Adding block 114: 542499dfc63bc94c0ba0a3d81bdaff07ab73ba9aebddd4c2ae4b2e0c1e8bd15f
...
lightningd-2 2025-07-08T12:08:30.459Z UNUSUAL 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer permanent failure in CHANNELD_AWAITING_SPLICE: Fulfilled HTLC 1 RCVD_REMOVE_REVOCATION cltv11 4 hit deadline (reason=protocol)
lightningd-2 2025-07-08T12:08:30.469Z DEBUG 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-channeld-chan#1: Status closed, but not exited. Killing
lightningd-2 2025-07-08T12:08:30.475Z INFO 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: State changed from CHANNELD_AWAITING_SPLICE to AWAITING_UNILATERAL
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It was originally a wrapper for JSON param() results, but now it's a more
generic struct. So make it clear that the fields are not optional. This
means a manual assignment in the initial population of this struct, but
all the users are now far clearer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Move the feature tuning algorithm to mcf.c, ie. the loops for searching
a good mu and delay_feefactor to satisfy the problem constraints.
We are looking to set the stage for an execution logic that allows for
multiple choices of routing algorithms, mainly for experimenting without
breaking the default working code.
Changelog-None
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Add log information about the runtime of getroutes.
Changelog-None: askrene: add runtime of getroutes to the logs
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Before vs after:
- `forwards` indexed by `in_channel and in_htlc_id` (see lightning-listforwards(7))
- `forwards` indexed by `in_channel` and `in_htlc_id` (see lightning-listforwards(7))
And:
- `htlcs` indexed by `short_channel_id and id` (see lightning-listhtlcs(7))
- `htlcs` indexed by `short_channel_id` and `id` (see lightning-listhtlcs(7))
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-None: Add a differential fuzz test for
HMAC-SHA256, similar to those for SHA256 and RIPEMD160,
to verify CCAN’s implementation against OpenSSL’s.
Change in the fuzz-testing scheme of fuzz-base32-64 led to
the discovery of test inputs that result in greater in
code-coverage. Add these inputs to the test's seed corpus.
