bookkeeper used to generate these as channel events, now lightningd does.
We also add a "journal" event, which we will need later too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We still output the fields, they're just always the currency of the node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `bookkeeper` now explicitly assumes every transaction is in the same currency as the node (true unless you added manually)
We're going to get rid of this concept, but the main change is that the
account_get_balance API can be drastically simplified:
account_get_credit_debit() accesses the raw fields, never fails, but
returns the a flag which tells us if the account doesn't actually have
any events.
The one place we care about the balance, calculate by hand. Then
account_get_balance() (and struct account_balance) can simply be
moved to th test.
Subtly, without the "GROUP BY" clause, you always get one row, even if
there are no rows (but the SUM are null).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means we can keep a pointer to the channel directly, *or* a string.
This avoids gratuitous formatting (on creation) and lookups (later).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
json_tok_streq(…) and json_get_member(…) are convenience wrappers for
json_tok_strneq(…) and json_get_membern(…) respectively. Unfortunately, using
them incurs a performance penalty in the common case where they are called with
a string literal argument because the compiler is unable to substitute a
compile-time constant in place of the buried call to strlen(…).
For example,
json_get_member(buf, tok, "example");
…will have worse performance than…
json_get_membern(buf, tok, "example", strlen("example"));
…because the former is forced to scan over "example" at run-time to count its
length whereas the latter is able to elide the strlen(…) call at compile time.
Hoist these convenience functions up into common/json_parse_simple.h and mark
them as inline so that the compiler can elide the strlen(…) call in the common
case of calling these functions with a string literal argument.
Changelog-None
Requested-by: Michael at Boltz
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `xpay` now publishes `pay_part_start` and `pay_part_end` notifications on every payment send attempt.
Changelog-Added: Plugins: `sql` also supports functions `json_object(key1, value1, ...)` to construct JSON objects and `json_group_array(value)` to aggregate rows into JSON array.
Security Considerations
- No new SQL injection risks: Functions only process explicitly provided column values (no arbitrary string parsing).
- Explicit column requirements: Wildcards (*) are not supported, all fields must be named (e.g., json_object('peer_id', id)).
- Permission-bound data access: Functions adhere to the same table/row permissions as the underlying query.
Performance Impact
- Optimized native execution: Leverages SQLite’s built-in JSON1 extension (when available) for efficiency.
- Moderate CPU overhead: Complex nesting may impact performance on large datasets but still faster than application-layer JSON conversion.
Now you can grep for 'sqlite3 version' and see where we would like
to update.
Debian 11 (Bullseye) and Ubuntu 20.04 (Focal) ship with SQLite 3.31.1.
RHEL 9 ships with 3.34.1. Fedora 38+ uses SQLite 3.40+.
Unfortunately, RHEL8 ships with 3.26.0, and is still on maintenance Support
(security fixes, no new features): runs until May 31, 2029.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We use a wrapper around the MCF solver that takes care of finding the
best linearization parameters and fixing the flow values to meet the
htlc_min and htlc_max constraints.
We have reworked the current implementation and made it a bit more
similar to renepay's version.
Out of 50000 simulated payment situations distributed accross payment
amounts of 1e2, 1e3, 1e4, 1e5 and 1e6 sats, we find that 133 failed
cases in the master branch turn to success with the current changes,
while only 3 success cases in the master are not solved by the changes.
master
+-------+------+
| S | F |
+---+-------+------+
| S | 46329 | 133 |
changes +---+-------+------+
| F | 3 | 3535 |
+---+-------+------+
Out of the 133 cases that flipped from failure to success the failed
reasons were:
122 -> "Could not find route without excessive cost"
5 -> "We couldn't quite afford it"
5 -> "Amount *msat below minimum"
1 -> tripped an HTLC min check
Changelog-None.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
by a small amount if the deliver amount is less than the requested
amount by X.
This step saves runtime by avoiding calling an extra MCF
and it helps us solve a small percentage of cases where the only
available routes have HTLCmin that is bigger than X.
Changelog-None
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Disable channels with HTLC min violations so that we don't hit them
twice when computing routes.
Changelog-None
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Try a new way to refine flows,
ie. reduce excess due to MCF accuracy and HTLC max constraints
after hop amounts are computed with fees included.
Changelog-None.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Changelog-Added: clnrest can now return successful responses as xml, yaml, or form-encoded in addition to json defined in the 'Accept' header. The same goes for request types defined in the 'Content-type' header.
Changelog-Changed: wss-proxy.py was replaced by a rust version with support for multiple `wss-bind-addr`. If you install CLN from pre-compiled binaries you must remove the old wss-proxy directory first before installing CLN, usually
it is located in `/usr/local/libexec/c-lightning/plugins/wss-proxy`. If you compile from source `make` will take care of this automatically.
From the multiple arcs that derive from the same channel we consider
only those with the smallest cost such that the payment amount and HTLC
max can fit in their combined capacity, ie. we prune high cost arcs that
surely will never be used by the optimal solution.
This reduces the number of arcs in the graph approximately from 8 arcs
per channel to approximately 2 arcs per channel.
No pruning.
amount: 100 1000 10000 100000 1000000
channels: 104741 106163 106607 106654 106666
arcs: 837928 849304 852856 853232 853328
Prune, limit the channel capacity by its HTLC max
amount: 100 1000 10000 100000 1000000
channels: 104741 106163 106607 106654 106666
arcs: 255502 259314 260538 260676 260704
Prune, limit the channel capacity to the payment amount
amount: 100 1000 10000 100000 1000000
channels: 104741 106163 106607 106654 106666
arcs: 209482 216270 228618 295450 432468
Prune, limit the channel capacity to the payment amount and its HTLC max
amount: 100 1000 10000 100000 1000000
channels: 104741 106163 106607 106654 106666
arcs: 209480 212324 213242 215726 228018
This produces a slight speedup for MCF computations:
Amount (sats) | speedup
-----------------------
100 | 1.89
1000 | 1.77
10000 | 1.25
100000 | 1.25
1000000 | 1.18
Changelog-None
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Changelog-Added: askrene: an optimal single-path solver has been added, it can be called using the developer option --dev_algorithm=single-path or by adding the layer "auto.no_mpp_support"
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
The single path solver uses the same probability cost and fee cost
estimation of minflow. Single path routes computed this way are
suboptimal with respect to the MCF solution but still are optimal among
any other single path. Computationally is way faster than MCF, therefore
for some trivial payments it should be prefered.
Changelog-None.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Refactor MCF solver: remove structs linear_network and residual_network.
Prefer passing raw data to the helper functions.
Changelog-None
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
It was originally a wrapper for JSON param() results, but now it's a more
generic struct. So make it clear that the fields are not optional. This
means a manual assignment in the initial population of this struct, but
all the users are now far clearer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Move the feature tuning algorithm to mcf.c, ie. the loops for searching
a good mu and delay_feefactor to satisfy the problem constraints.
We are looking to set the stage for an execution logic that allows for
multiple choices of routing algorithms, mainly for experimenting without
breaking the default working code.
Changelog-None
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Add log information about the runtime of getroutes.
Changelog-None: askrene: add runtime of getroutes to the logs
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Before vs after:
- `forwards` indexed by `in_channel and in_htlc_id` (see lightning-listforwards(7))
- `forwards` indexed by `in_channel` and `in_htlc_id` (see lightning-listforwards(7))
And:
- `htlcs` indexed by `short_channel_id and id` (see lightning-listhtlcs(7))
- `htlcs` indexed by `short_channel_id` and `id` (see lightning-listhtlcs(7))
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
