Commit Graph

17286 Commits

Author SHA1 Message Date
Rusty Russell
22f7a620e6 lightningd: fix crash on fixup scan if block unavailable.
```
lightningd: FATAL SIGNAL 11 (version v25.12rc3-1-g498c5b6)
0x5cc2f620ce4c send_backtrace
        common/daemon.c:38
0x5cc2f620cee8 crashdump
        common/daemon.c:83
0x7e3ac1e4532f ???
        ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x5cc2f615f186 fixup_scan_block
        lightningd/chaintopology.c:1531
0x5cc2f615c22c getrawblockbyheight_callback
        lightningd/bitcoind.c:484
0x5cc2f61aee87 plugin_response_handle
        lightningd/plugin.c:701
0x5cc2f61b4043 plugin_read_json
        lightningd/plugin.c:790
0x5cc2f6248d8b next_plan
        ccan/ccan/io/io.c:60
0x5cc2f624925c do_plan
        ccan/ccan/io/io.c:422
0x5cc2f6249319 io_ready
        ccan/ccan/io/io.c:439
0x5cc2f624ad24 io_loop
        ccan/ccan/io/poll.c:470
0x5cc2f618381a io_loop_with_timers
        lightningd/io_loop_with_timers.c:22
0x5cc2f61892ff main
```

This happens intermittantly on in a few tests:

tests/test_invoices.py::test_invoice_botched_migration
tests/test_pay.py::test_pay_bolt11_metadata
tests/test_runes.py::test_id_migration

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: lightningd: potential crash on startup if bitcoind isn't up-to-date.
2025-12-10 14:43:45 +10:30
Madeline Paech
b270d915d3 updated pull request for next cln release timeline 2025-12-09 19:11:04 +10:30
Lakshya Singh
62300da7b8 chore: remove rustfmt check in configure
rustfmt isn’t needed for production builds and cargo is sufficient

[ Also, rustfmt --version exits with status 1 due to deprecation --RR ]
Changelog-None
2025-12-09 13:50:16 +10:30
Christian Decker
686ce98906 git: Remove generated binary 2025-12-08 16:37:02 +01:00
Christian Decker
16735c95b7 chore: Downgrade tool diff 2025-12-08 16:37:02 +01:00
Christian Decker
8db09f6d11 Add GitHub Pages documentation site with orchestrated workflows
This commit creates a comprehensive documentation publishing system that
combines coverage reports, Python API docs, and project documentation into
a unified GitHub Pages site.

Changes:
- Update coverage-nightly.yaml to support workflow_call trigger
- Update python-docs-nightly.yaml to support workflow_call trigger
- Add docs-nightly.yaml workflow for project documentation
- Add publish-site.yaml orchestrator workflow

The publish-site workflow:
- Triggers all three documentation workflows in parallel
- Collects artifacts from each workflow
- Organizes them into a unified site structure:
  - / (root) - Beautiful landing page with navigation
  - /docs/ - Project documentation
  - /python/ - Python API reference (pdoc3)
  - /coverage/ - Code coverage reports
- Deploys to GitHub Pages with proper permissions
- Runs nightly at 5 AM UTC, after all other workflows complete

Each workflow can be:
- Triggered manually via workflow_dispatch
- Called from other workflows via workflow_call
- Run on schedule (coverage: 2 AM, python-docs: 3 AM, docs: 4 AM)

The site includes:
- Modern, responsive landing page with gradient design
- Navigation cards for each documentation section
- 404 error page
- .nojekyll file to prevent Jekyll processing
- Automatic timestamp updates

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Changelog-None
2025-12-08 16:37:02 +01:00
Christian Decker
5c9d3884bb Add Python API documentation generation with pdoc3
This commit adds automated Python API documentation generation for all
workspace packages using pdoc3:

- Add contrib/api/generate-python-docs.py script to generate docs
- Add Makefile targets: python-docs and python-docs-clean
- Add GitHub Actions workflow for nightly documentation generation
- Documents 5 packages: pyln.client, pyln.proto, pyln.grpc, pyln.testing, pyln.spec.bolt7
- Creates beautiful index page with cards linking to each package
- Stores generated docs as artifacts with 90-day retention
- Add pdoc3 and markdown to dev dependencies

Bug fix:
- Fix pyln-client version.py: __all__ must contain strings, not class objects
  This was causing "TypeError: attribute name must be string, not 'type'" in pdoc3

Documentation is generated to docs/python/ which is excluded from version control.
Run 'make python-docs' to generate locally, or download from nightly workflow artifacts.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-08 16:37:02 +01:00
Christian Decker
4b9cffe183 Add comprehensive coverage infrastructure with clang source-based coverage
This commit introduces a modern coverage infrastructure for Core Lightning:

- Migrate from ad-hoc coverage script to integrated Makefile targets
- Add LLVM source-based coverage support with per-test profraw organization
- Integrate coverage collection into pytest framework via TailableProc
- Add GitHub Actions workflow for nightly coverage reports
- Add Taskfile.yml for convenient task automation
- Add codecov.yml for Codecov integration
- Add comprehensive coverage documentation in COVERAGE.md
- Update contributor workflow docs with new coverage script path
- Add coverage data files to .gitignore (*.profraw, *.profdata)
- Remove obsolete contrib/clang-coverage-report.sh
- Remove obsolete tests/conftest.py (now using pyln-testing markers)
- Update pyproject.toml to include pyln-testing in main dependencies

The new infrastructure automatically collects coverage data when CLN_COVERAGE_DIR
is set, organizing profraw files by test name for granular analysis.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-08 16:37:02 +01:00
Matt Whitlock
7a1a363330 test_renepay.py: remove data dump into /tmp
Changelog-None
2025-12-08 17:47:26 +10:30
Christian Decker
5c0827e417 ci: Add 2 shards to the UBSAN / ASAN runs 2025-12-06 12:32:38 +01:00
Christian Decker
2a2def32ab ci: Remove nix check from the critical path in ci.yaml 2025-12-06 12:32:38 +01:00
Christian Decker
5c749fe4ee ci: Mark some tests as rerun 2025-12-06 12:32:38 +01:00
Christian Decker
fa31e6e4c2 git: Ignore two more binaries 2025-12-06 12:32:38 +01:00
Christian Decker
7c28fdb689 ci: Run with rerunfailures rather than flaky 2025-12-06 12:32:38 +01:00
Christian Decker
287abfbd90 ci: Add a simple plugin to report test results to our falkiness tracker
Changelog-None
2025-12-06 12:32:38 +01:00
ShahanaFarooqui
d18efbb0b7 doc: Add instructions to run CLN node with remote_hsmd_socket 2025-12-05 10:25:19 -08:00
ShahanaFarooqui
a5268e5780 ci: Add new version-vls tag for Docker image
Changelog-Added: Added a new `version-vls` tag with includes  VLS's `remote_hsmd_socket` binary with the Core Lightning image.
2025-12-05 10:25:19 -08:00
ShahanaFarooqui
be471e598f docker: Add vls remote_hsmd_socket binary
This Dockerfile builds the VLS binaries during the builder stage and copies only the `remote_hsmd_socket` binary into the `lightningd-vls-signer` target.
2025-12-05 10:25:19 -08:00
ShahanaFarooqui
c03288cff7 doc: Update release instructions 2025-12-04 16:29:39 -08:00
ShahanaFarooqui
bc64d92b23 tools: Capture Shasums for current version only
Fixes #8753.

Changelog-None: Release process fix.
2025-12-04 16:29:39 -08:00
Madeline Paech
ed439772a8 release branch for 25.12 release 2025-12-04 10:44:56 +10:30
ShahanaFarooqui
af5d02a9fa gitignore: Ignore lightning-downgrade and check-bolt tools
Changelog-None.
2025-12-03 06:24:04 -08:00
Rusty Russell
e02f76bdd8 CHANGELOG: update for 25.12rc3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-29 10:32:47 +10:30
Dusty Daemon
377802ce9c contrib: Update startup_regtest for taproot
Wallet address default to taproot now — update the startup_regtest.sh script to use taproot addresses.

Changelog-None
2025-11-29 08:35:49 +10:30
Matt Whitlock
6db80536cc tests/test_misc.py: fix another "AF_UNIX path too long"
Also, avoid resource leak by explicitly closing socket.

Changelog-None
2025-11-29 08:35:09 +10:30
wqxoxo
3c08a8bdb9 xpay: fix misleading "route hint" label in error messages
describe_scidd() was labeling any channel as "the invoice's route hint"
when there was a single-hop route hint, even if the error occurred on
an unrelated intermediate channel.

Now we check the channel actually matches the route hint's
short_channel_id before using that label.

Fixes: #8252
Changelog-Fixed: xpay: error messages no longer incorrectly label intermediate channels as "the invoice's route hint".
2025-11-29 08:33:37 +10:30
Rusty Russell
9d4a150b7f pytest: add test that we correctly scan back for missed UTXOs.
I took the node from the original test which missed the UTXO, and
added some more blocks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-29 08:18:20 +10:30
Rusty Russell
6c60375651 lightningd: scan back to seek missing UTXOs.
We only do this once, and not on new nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-29 08:18:20 +10:30
Rusty Russell
1e66414b9e wallet: make sure we re-watch outpoints after blocks are rolled back.
At startup, we load the outpoints to watch, *then* roll back 15
blocks.  If there were things in those blocks we wanted to watch, we
no longer do!

1. We load the utxoset into memory: everything in the utxoset table
   which has spendheight null.
2. We roll back 15 blocks to re-read.  Deleting a block from the
   database causes the utxo spentheights referring to it to be set
   to null.
3. We roll forward, but we didn't update the in-memory utxoset,
   so we're not watching those utxos which are spent.

The main symptom of this is that we spam peers with obsolete gossip
(if we get sent a channel announcement for a closed channel, we can
think it isn't spent yet).  But it could *also* mean we don't notice
onchain txs, if we restart at the wrong time!

Changelog-Fixed: lightningd: we could miss tx spends which happened in the past blocks when we restarted.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-29 08:18:20 +10:30
Rusty Russell
8caffde0fa pytest: test for watching utxos over restart reorg.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-29 08:18:20 +10:30
ShahanaFarooqui
534e3ae8f8 ci: Remove lowdown and configure from the sign release job step
Changelog-None: Fixes draft release job creation in CI.
2025-11-28 20:27:02 +10:30
Rusty Russell
7e9ec3ef2d tools: remove accidentally-committed obsolete hsmtool binary.
And keep it in the .gitignore, as others may have old ones lying around too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 15:45:06 +10:30
Madeline Paech
c59d9beab3 second release candidate for 25.12 2025-11-28 10:48:43 +10:30
Rusty Russell
385b6a9e2f doc: generate documentation for askrene-bias-node.
I noticed it was missing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
640888aa91 CI: add test that we can downgrade the node and run it under v25.09.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
39a92116a8 doc: add a manual page.
In particular, document when downgrades are not possible.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
4d8f1d33e9 pytest: add failing downgrade test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
a45c907200 lightningd-downgrade: downgrade askrene's layer info too.
If they had a channel bias, and ran xpay, it will update the bias
to a v2 bias (with a timestamp).  We must downgrade that, or the
older version won't load!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
eb83a205b9 askrene: move datastore wire functions into separate file for lightning-downgrade to use.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
2a066b92fc wallet: separate datastore access functions for the lightning-downgrade tool to access.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
9981e238a3 tools/lightning-downgrade: tool to downgrade (offline) v25.12 to v25.09.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: tools: `lightningd-downgrade` can downgrade your database from v25.12 to v25.09 if something goes wrong.
2025-11-28 09:16:53 +10:30
Rusty Russell
76ad32ec88 check-bolt: move to devtools, clean up tools/
And leave $(TOOLS) as user-visible tools.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
23f2141683 global: use lightning-hsmtool not hsmtool.
When installed, the name is `lightning-hsmtool`.  We actually copy
`tools/hsmtool` to `tools/lightning-hsmtool` but that's a silly step
which we should get rid of.

So:
1. Make sure our documentation always refers to it as lightning-hsmtool.
2. Make sure our tests invoke it as `lightning-hsmtool`.
3. Rename the C file.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
6dcf4e7bdd wallet: separate migration table into its own source file.
This will make it easier to share with the downgrade tool.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
378293d268 wallet: add optional sql and functions to downgrade db.
This will allow a downgrade tool to attempt to downgrade a node to the
previous version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
f052408832 wire: add explicit-length fromwire_peektype variant.
Useful for continuous streams.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
fc23b538cd lightningd: print UNUSUAL log message if a command is very slow.
We don't expect an internal command to take 5 seconds to service
without explicitly pausing: if it does, log at a higher level.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-27 15:04:20 +10:30
Rusty Russell
acc41ddc0c connectd: don't log at INFO level for known issue.
We get spammed by this, because we somehow missed occasional channel closes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-27 15:04:20 +10:30
ShahanaFarooqui
ff0ee6dfa0 doc: Fix mdx format errors generated due to Readme v2 migrations
Changelog-None: Documentation fixes only.
2025-11-27 14:10:18 +10:30
ShahanaFarooqui
edfb64c736 contrib: Remove litecoin keys as CLN has recently stopped supporting litecoin 2025-11-27 14:10:18 +10:30