Adds an async safe JSON-RPC V2 client for a generic transport layer. The
transport layer we will use later on are BOLT8 lightning messages.
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
fundwallet() actually mines a block, putting our count out. If we see
both blocks at once, we will say "52" blocks instead of "51":
```
2025-05-02T05:28:40.5315155Z have_forgotten = l2.daemon.is_in_log(
2025-05-02T05:28:40.5315650Z r"UNUSUAL {}-chan#1: Forgetting channel: It has been 51 blocks without the funding transaction ".format(l1.info['id'])
2025-05-02T05:28:40.5316105Z )
2025-05-02T05:28:40.5316263Z
2025-05-02T05:28:40.5316417Z if dopay:
2025-05-02T05:28:40.5316616Z assert not have_forgotten
2025-05-02T05:28:40.5317056Z assert set([c['peer_id'] for c in l2.rpc.listpeerchannels()["channels"]]) == set([l1.info['id'], l3.info['id']])
2025-05-02T05:28:40.5317477Z else:
2025-05-02T05:28:40.5317662Z > assert have_forgotten
2025-05-02T05:28:40.5317887Z E assert None
```
```
0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Forgetting channel: It has been 52 blocks without the funding transaction 0bb0579df6b1d983dda49dad47513afc71696c9d5bea3c8b955ba4b76bb053de getting deeply confirmed. We are fundee and can forget channel without loss of funds.
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were supposed to put the sqlite db in a different directory, but
the test was wrong! So occasionally we would crash with:
```
Failed to commit DB transaction: Failed to commit a transaction: disk I/O error
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We try to do a JSON response if Python is not present, but it assumes sed. We should cleanly
exit on errors.
Before:
```
$ PATH=/tmp ./plugins/wss-proxy/wss-proxy
Something
./plugins/wss-proxy/wss-proxy: 12: sed: not found
{"jsonrpc":"2.0","id":,"result":{"disable":"No python3 binary found"}}
```
After:
```
$ PATH=/tmp ./plugins/wss-proxy/wss-proxy
something
./plugins/wss-proxy/wss-proxy: 12: sed: not found
```
Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This finally happened on my local build machine, so I tracked it down using
py-spy, `apt-get install python3-dbg` and `py-local`.
Turns out the dev-memleak command was hanging, and the processes were stuck in
SIGSTOP. There are only two places we send that, and sure enough, this was
the test which was running at the time.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
plugin_log inside after_send_scb_single was logging after sending peer storage
to each peer which could lead to spam in logs for big nodes, hence we should reduce
the log level to log_trace for it.
Changelog-Fixed: Suppress logs from chanbackup
This may be useful for their recovery, though they should see the spend onchain.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: We now reply to `channel_reestablish` even on long-closed channels.
This was always false. peer_start_channeld was called in various places
with the argument "NULL" instead of "false", which unfortunately compilers
didn't complain about :(
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Michael of Boltz told me this long ago, that when fees spiked some of their clients' opens got stuck. After two weeks their node forgot them, and when fees came back down the opens still failed. Unfortunately, I can't see an issue for this!
We can give some grace: we don't want to waste too many resources, but 100 channels is nothing.
The test needs to be adjusted to have *two* slow open channels, now.
Changelog-Changed: Protocol: we won't forget still-opening channels after 2016 blocks, unless there are more than 100.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We're going to start loading them into memory for nicer responses if
people try to reestablish closed channels, but we don't care about ones
which were never actually opened. We could add a new state, but easier
to simply remove them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We are supposed to allocate of the ctx we're passed, not tmpctx.
Doesn't matter for now, because we don't use this result with anything which outlives tmpctx,
but we're going to:
```
==47574==ERROR: AddressSanitizer: heap-use-after-free on address 0x6040005a8f38 at pc 0x55d3c584d252 bp 0x7ffddfb1b090 sp 0x7ffddfb1b088
READ of size 8 at 0x6040005a8f38 thread T0
#0 0x55d3c584d251 in json_add_closed_channel /home/runner/work/lightning/lightning/lightningd/closed_channel.c:27:3
#1 0x55d3c584ca5a in json_listclosedchannels /home/runner/work/lightning/lightning/lightningd/closed_channel.c:118:3
#2 0x55d3c58c0cbe in command_exec /home/runner/work/lightning/lightning/lightningd/jsonrpc.c:808:8
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that documentation says invoice expiries can batch, but that's no
longer true, so delete it. Usually, we miss a number because the
change is too fast.
This adds the wait interface, but it doesn't actually fire until the next
commit, which wires it into the db code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `wait` now supports the `htlcs` (`listhtlcs`) subsystem.
We're not going to increment one at a time for bulk deletion of htlcs
when a channel closes. There could be millions of HTLCs!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It makes the schema simpler, and indeed, expressable by GRPC.
Changelog-Added: JSON-RPC: `wait` now has separate `invoices`, `forwards` and `sendpays` objects for each subsystem.
Changelog-Deprecated: JSON-RPC: `wait` reply `details` object: use subsytem specific object instead.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The last argument is a pointer, but we were handing `false`. Which, for
terrible historic reasons, gets treated as NULL.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
notleak() doesn't work for lightningd since the first span is created before
memleak (or anything else!) is initialized, so we have to mark it manually.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This doesn't happen very often, but can with autoclean.
However, we rarely traverse to the end, since we always expect to find
what we're looking for, and we fill from the front. So even a large
array (unless it's used) is fine.
Subtle: when we injected a parent, we used "active_spans" as the (arbitrary)
key. That can now change with reallocation, and so if that memory were reused
we could have a key clash. So we use "&active_spans" which doesn't change.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We don't ever actually close the remote span (we don't have its key,
after all), and we keep a pointer to the parent.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is much faster to give 64 bits of data, and we don't need
cryptographic randomness.
This brings us back to 413ns per trace.
Before:
real 0m5.819000-6.472000(6.2064+/-0.26)s
user 0m3.779000-4.101000(3.956+/-0.12)s
sys 0m2.040000-2.431000(2.2496+/-0.15)s
After:
real 0m3.981000-4.247000(4.1276+/-0.11)s
user 0m3.979000-4.245000(4.126+/-0.11)s
sys 0m0.000000-0.002000(0.001+/-0.00063)s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: lightingd: trimmed overhead of tracing infrastructure.
There's an EBPF limit anyway, so stick with a 512-byte buffer.
This brings us back to 621ns per trace:
Before:
real 0m13.441000-14.592000(14.2686+/-0.43)s
user 0m11.265000-12.289000(11.9626+/-0.37)s
sys 0m2.175000-2.381000(2.3048+/-0.072)s
After:
real 0m5.819000-6.472000(6.2064+/-0.26)s
user 0m3.779000-4.101000(3.956+/-0.12)s
sys 0m2.040000-2.431000(2.2496+/-0.15)s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Avoids allocations. Also assume that name and value parameters
outlive the trace span, so don't copy.
Before:
real 0m16.421000-18.407000(17.8128+/-0.72)s
user 0m14.242000-16.041000(15.5382+/-0.67)s
sys 0m2.179000-2.363000(2.273+/-0.061)s
After:
real 0m13.441000-14.592000(14.2686+/-0.43)s
user 0m11.265000-12.289000(11.9626+/-0.37)s
sys 0m2.175000-2.381000(2.3048+/-0.072)s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. trace_span_start() is always called with a string literal, so
no copy needed (and we can use a macro to enforce this).
2. trace_span_tag() name and value are always longer-lived than
the span, so no need to copy these either.
Before:
real 0m18.524000-19.100000(18.7674+/-0.21)s
user 0m16.171000-16.833000(16.424+/-0.26)s
sys 0m2.259000-2.400000(2.337+/-0.059)s
After:
real 0m16.421000-18.407000(17.8128+/-0.72)s
user 0m14.242000-16.041000(15.5382+/-0.67)s
sys 0m2.179000-2.363000(2.273+/-0.061)s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Testing parenting handling revealed several issues:
1. By calling "trace_span_start" when CLN_TRACEPARENT is set produces a bogus
entry, for which the span_id is overwritten so we never end it.
2. We don't need to close the remote parent when we close the first child: in
fact, this causes the remaining traces to be detached from the parent!
3. Suspension should return current to the parent, not to NULL.
Now the traces balance as we expect.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I added this debugging because the next test revealed a mismatch, so
I wanted to see where it was happening.
The comment in lightningd suggests it's possible, but I can't see any
code which suspends in the lightningd io_loop, so I cannot see how
this is triggered.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
With an average runtime of 18.7674, this implies 1876ns
per trace, which is far in excess of the 370ns claimed in
doc/developers-guide/tracing-cln-performance.md.
We also add a tag in there, so we measure that!
Results on my laptop:
real 0m18.524000-19.100000(18.7674+/-0.21)s
user 0m16.171000-16.833000(16.424+/-0.26)s
sys 0m2.259000-2.400000(2.337+/-0.059)s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Introduced the use of UPDATE FROM syntax in SQLite queries,
which is not supported in versions prior to 3.33.0.
This causes issues on systems with older SQLite versions,
as reported in issue #8231. Rewrite the query in
migrate_convert_old_channel_keyidx() to use a subquery
with IN clause instead of UPDATE FROM, ensuring compatibility with
older SQLite versions.
Changelog-Fixed: db: replace UPDATE FROM syntax for SQLite compat
Fixes 68f3649d6b
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
```
cargo build --quiet --example cln-plugin-startup
error: failed to run custom build command for `cln-grpc v0.4.0 (/home/runner/work/lightning/lightning/cln-grpc)`
Caused by:
process didn't exit successfully: `/home/runner/work/lightning/lightning/target/debug/build/cln-grpc-1c0900b8d6f448d4/build-script-build` (exit status: 101)
--- stdout
cargo:rerun-if-changed=proto/node.proto
cargo:rerun-if-changed=proto
--- stderr
thread 'main' panicked at cln-grpc/build.rs:7:10:
called `Result::unwrap()` on an `Err` value: Custom { kind: NotFound, error: "Could not find `protoc`. If `protoc` is installed, try setting the `PROTOC` environment variable to the path of the `protoc` binary. Try installing `protobuf-compiler` or `protobuf` using your package manager. It is also available at https://github.com/protocolbuffers/protobuf/releases For more information: https://docs.rs/prost-build/#sourcing-protoc" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
gmake: *** [cln-rpc/Makefile:15: target/debug/examples/cln-plugin-startup] Error 101
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
