Commit Graph

17336 Commits

Author SHA1 Message Date
Peter Neuroth
4a1c922017 plugins: lsps: remove anyhow from tlvs
Remove anyhow as a dependency from the tlv module. This allows for a
cleaner error handling

Changelog-None

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
c7bbf84c83 plugins: lsps: remove cln dependencies from core module and clean up service
The core module shouldn't depend on anything from the cln_adapters
module. This ensures that we can switch out interfaces in the future
(e.g a different database than cln's datastore).

This change made it necessary to refactor the service plugin which
allowed us to clean it up on the go

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
30268ad945 plugins: lsps: move feature helpers to core module
This removes the util.rs module and leaves us with proto, core and
cln_adapters

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
f678a70610 plugins: lsps: remove lsps2 module
Separate concerns of the lsps2 modules into their respective layers:
core for tlv, as it is lightning related, not specifically
core-lightning. cln_adapters for core-lightning related types.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
fa62c64de7 plugins: lsps: split up handlers
This commit separates the business logic from core-lighting behaviour
and moves the core-lightning related implementations into the
cln_adapters module.

Splits up tests as well

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
e001652ab1 plugins: lsps: add own trait for lightning provider
Clean up the CLN interface

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
a05c2c9c2c plugins: lsps: add own trait for datastore provider
This now only makes it easier to test, but also more modular. It allows
us to use any kind of storage, not only the datastore.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
38c84be25e plugins: lsps: add own trait for blockheight provider
The ClnApi trait is way overloaded. We break things down to make it more
modular on the service side.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
38b3d14eb9 plugins: lsps: add own trait for lsps offer provider
The cln api trait is really cluttered, we try to clean it up a little
bit.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
e50f5ee863 plugins: lsps: refactor service hook
We have some shared behavior and can use the MessageSender for the
service hook as well

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
462ca844dd plugins: lsps: replace heavy transport
We replace the legacy transport by a slim custommsg hook that makes use
of the lightweight multiplexed transport and avoids deserialization and
serialization through this extra layer

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
d5bcde09ae plugins: lsps: implement ClnSender for bolt8 transport
Implement the MessageSender trait for Bolt8 transport via core-lightning

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
3cf99c358b plugins: lsps: add slim multiplexing transport
We can separate concerns here by adding a lightweight multiplexing
transport client. This helps us to clean up any direct dependencies on
CLN

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
7401585088 plugins: lsps: move client transport to cln_adapters
cln_adapters is the layer that holds concrete implementations that are
related to core-lightning.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
9adc8cdfed plugins: lsps: slim down lsps service
This commit replaces the lsps service. Therefore we add the cln_adapters
module that handles the communication with core-lightning. We add a
lightweight LspsService that wraps the router and allows handlers to
register themselves to the service/router.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
461065aff7 plugins: lsps: add router compatible lsps2 service handler
This commit copys and adapts the lsps2 service handlers to match the
slim router handlers

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
ffc05ddcf0 plugins: lsps: add slim generic json-rpc router
This router is very lightweight and can handle routing for the lsps
service. It avoids allocations by just borrowing from the request. This
is much cleaner than what we have now and reduces the amount of
serializations by a lot.

Also it supports typed handlers which will remove serialization overhead
from the handlers in future commits

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
1159adb0e2 plugin: lsps: move json-rpc server to core module
This removes the jsonrpc module

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
b9172239d0 plugins: lsps: switch to typed transport
There is no need to communicate via a generic JsonRpcClient if all
messages are well defined and we can use the typed transport to remove a
whole lot of overhead and just pass the ResponseObjects directly to the
Transport layer

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
1bb76f84e2 plugins: lsps: add request to transport trait
Introducing "request" to the transport trait that uses the RequestObject
and JsonRpcResponse directly reduces some overhead and unnecessary
conversions. Why do we implement a generic transport over &str when we
already know that we are dealing with JSON-RPC 2.0 here.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
2546642c32 plugins: lsps: move id generation into jsonrpcrequest
There is no need to generate the id in some client when we can actually
generate it on a lower layer as we know by LSPS0 how we have to generate
it.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
5ed743e10d plugins: lsps: add lsps0 encoding and decoding
In order to seperate concerns, this adds the basic encoding of an lsps0
frame to the proto module.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
0927179fe1 plugins: lsps: replace owned types by references
Improves flexibility in transport implementations.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
e08756cd57 plugins: lsps: use transport specific error
instead of using std::error, this is more specific and keeps the errors
"type-safe"

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
e591323ac8 plugins: lsps: remove peer_id from transport
As we pass the peer_id through the transport methods, there is no need
to store the peer_id in the Bolt8Transport struct anymore

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
0f624ed67f plugins: lsps: add peer_id to transport
Frankly, transport without a target doesn't make sense, so this commit
adds a PublicKey (from secp256k1) as the target to the Transport trait.
It can easily be replaced by a common PeerId without changing the api
for external implementations if needed in the future

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
6e868443e2 plugins: lsps: move transport to core module
This commit is part of a series of commits to make the lsps plugin crate
layered and modular for efficient reuse in external projects.

The core module is responsible for common business logic and interfaces
and acts as the intermediate layer for an actual implementation

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
5274556518 plugins: lsps: refactor client to use jsonrpcresponse
This makes it easier for client implementations to separate transport or
parser related issues from actuall json-rpc error responses. Also this
helps to segregate the individual responsibilities of the crates
further.

This further alows us to remove the error enum from proto/jsonrpc
completely as this now only deals as a from/to-wire module

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
bad42c711b plugin: lsps: simplify json-rpc response object
We can use an enum as a more idiomatic rust style to create and retreive
json-rpc responses. The actual response data is kept in the body enum
while the surrounding only holds the identifier.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
67f11b62e6 plugins: lsps: refactor jsonrpc error
In order to create a sane dependecy structure for this crate, this
commit cleans up the jsonrpc error enum and shifts responsibilities
towards the transport client

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
9c474a6d5c plugins: lsps: move transport error to client module
The error belongs to the transport definition, not into the
wire/protocol module.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
a51f6e1ebb plugins: lsps: remove debug print from library module
This commit is part of a series that refactor the lsp plugin crate into
a modularized crate with the goal to separate the actual plugin runtime
as much as possible from library code to make it accessible for 3rd
party plugin implementations.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
b7be5ef3a2 plugins: lsps: refactor lsps2 and lsps0 error enums
Split them up how they are actually defined.

This commit is part of a series that refactor the lsp plugin crate into
a modularized crate with the goal to separate the actual plugin runtime
as much as possible from library code to make it accessible for 3rd
party plugin implementations.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
b88dc5972f plugins: lsps: move lsps2 model to proto module
This commit is part of a series that refactor the lsp plugin crate into
a modularized crate with the goal to separate the actual plugin runtime
as much as possible from library code to make it accessible for 3rd
party plugin implementations.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
ba93a7ca62 plugins: lsps: move primitives into lsps0
Those primitives are actually defined in lsps0

This commit is part of a series that refactor the lsp plugin crate into
a modularized crate with the goal to separate the actual plugin runtime
as much as possible from library code to make it accessible for 3rd
party plugin implementations.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
5847cd443f plugins: lsps: move jsonrpc into proto module
This commit is part of a series that refactor the lsp plugin crate into
a modularized crate with the goal to separate the actual plugin runtime
as much as possible from library code to make it accessible for 3rd
party plugin implementations.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
bd9fd581f1 plugins: lsps: move feature types into proto module
This commit is part of a series that refactor the lsp plugin crate into
a modularized crate with the goal to separate the actual plugin runtime
as much as possible from library code to make it accessible for 3rd
party plugin implementations.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
a94885ba7f plugins: lsps: move errors into proto module
This commit is part of a series that refactor the lsp plugin crate into
a modularized crate with the goal to separate the actual plugin runtime
as much as possible from library code to make it accessible for 3rd
party plugin implementations.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
39c36a03dd plugins: lsps: move lsps0 model into proto module
This commit is part of a series that refactor the lsp plugin crate into
a modularized crate with the goal to separate the actual plugin runtime
as much as possible from library code to make it accessible for 3rd
party plugin implementations.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Peter Neuroth
06fe54c467 plugins: lsps: move primitives into proto module
This commit is part of a series that refactor the lsp plugin crate into
a modularized crate with the goal to separate the actual plugin runtime
as much as possible from library code to make it accessible for 3rd
party plugin implementations.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-12-17 05:56:01 +10:30
Christian Decker
0b6f739f35 refactor: use GROUP variable for artifact names instead of sanitizing NAME
Add a GROUP matrix variable to valgrind and sanitizer test configurations
with simple numeric values. Use matrix.GROUP in artifact names instead of
sanitizing the descriptive NAME, which contains special characters.

This is cleaner and more maintainable than string replacement logic.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2025-12-16 13:04:33 +01:00
Christian Decker
612c5260ba fix: sanitize artifact names to handle special characters in matrix vars
The valgrind and sanitizer test matrix names contain parentheses and slashes
(e.g., 'Valgrind (01/10)', 'ASan/UBSan (01/12)') which are invalid in artifact
filenames. Replace these characters with hyphens to ensure reliable artifact uploads.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2025-12-16 13:04:33 +01:00
Christian Decker
c7d7a007b8 CI: fix env context reference in job-level env sections.
GitHub Actions doesn't allow referencing workflow-level env variables
from job-level env sections. Fixed by:
- Replacing ${{ env.PYTEST_OPTS_BASE }} with full values in job-level env
- Keeping ${{ env.PYTEST_OPTS_BASE }} in step-level env (which is valid)

This resolves the "Unrecognized named-value: 'env'" errors.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-16 13:04:33 +01:00
Christian Decker
82b7b39d86 CI: add artifact upload for pytest JUnit XML reports.
Adds upload-artifact steps after each test run to preserve test
results as GitHub artifacts. Each job uploads its report.xml with
a unique name based on the job and matrix configuration.

Artifacts are uploaded even when tests fail (if: always()) to
ensure test results are available for debugging.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-16 13:04:33 +01:00
Christian Decker
bca2db632c CI: consolidate GLOBAL_PYTEST_OPTS into PYTEST_OPTS_BASE.
Replaces GLOBAL_PYTEST_OPTS with PYTEST_OPTS_BASE that includes all
common pytest options: --reruns=10, -vvv, --junit-xml=report.xml,
--timeout=1800, and --durations=10.

Removes redundant option specifications from individual jobs and
simplifies pytest command invocations by removing direct references
to GLOBAL_PYTEST_OPTS (now inherited via PYTEST_OPTS).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Changelog-None
2025-12-16 13:04:33 +01:00
ShahanaFarooqui
546272048b tools: Add script to update stable tag on Dockerhub
Changelog-None: Script for Docker tag only.
2025-12-15 22:10:35 -08:00
21M4TW
dbfd352a22 Changelog-Fixed: Replacing sed by $(SED) in Makefile 2025-12-15 15:39:12 -08:00
daywalker90
3c2d399647 docs: update backup install instructions
Changelog-None
2025-12-15 11:38:57 -08:00
Rusty Russell
98a188b394 doc: document the direction field in listpeerchannels.
We mentioned it in the schema (so it was allowed), but we didn't document it!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-12-12 14:20:08 +01:00
Rusty Russell
cef11ea361 lightningd: fix db constraint error when fixing up old blocks.
It's very hard to reproduce, since we only consider UTXOs we are watching, but scanning
from the first block we know about is wrong, because we don't care about blocks which
we only put in the db in response to old gossip queries.

I'm not sure how Sjors got into the state where they see their own UTXO spend in a block
they don't have in the database, but we shouldn't crash:

```
2025-12-07T11:25:18.163Z **BROKEN** lightningd: Error executing statement: wallet/wallet.c:4913: UPDATE outputs SET spend_height = ?,  status = ? WHERE prev_out_tx = ? AND prev_out_index = ?: FOREIGN KEY constraint failed
2025-12-07T11:25:18.163Z **BROKEN** lightningd: Error executing statement: wallet/wallet.c:4913: UPDATE outputs SET spend_height = ?,  status = ? WHERE prev_out_tx = ? AND prev_out_index = ?: FOREIGN KEY constraint failed
2025-12-07T11:25:18.179Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v25.12)
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: common/daemon.c:46 (send_backtrace) 0x5e263feb9292
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: common/daemon.c:83 (crashdump) 0x5e263feb92e1
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f64532f
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f69eb2c
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f64527d
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f6288fe
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/log.c:1054 (fatal_vfmt) 0x5e263fe38e10
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: wallet/db.c:95 (db_error) 0x5e263fe6f7ef
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: db/utils.c:326 (db_fatal) 0x5e263feaa797
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: db/utils.c:200 (db_exec_prepared_v2) 0x5e263feaab85
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: wallet/wallet.c:4924 (wallet_outpoint_spend) 0x5e263fe7de2f
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/chaintopology.c:929 (topo_update_spends) 0x5e263fe0af81
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/chaintopology.c:1532 (fixup_scan_block) 0x5e263fe0c268
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/bitcoind.c:503 (getrawblockbyheight_callback) 0x5e263fe09280
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:701 (plugin_response_handle) 0x5e263fe5bf2b
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:790 (plugin_read_json) 0x5e263fe610e7
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:60 (next_plan) 0x5e263fef3629
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:422 (do_plan) 0x5e263fef3afa
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:439 (io_ready) 0x5e263fef3bb7
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:470 (io_loop) 0x5e263fef55c5
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x5e263fe308e9
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1492 (main) 0x5e263fe363ce
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f62a1c9
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f62a28a
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x5e263fe06bf4
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff
```

Changelog-Fixes: lightningd: crash on fixup scan with old blocks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-12-11 12:16:42 +10:30