470 Commits

Author SHA1 Message Date
Rusty Russell
ca2d389920 devtools/gossipwith: don't count "padding" pings towards max-messages count.
We are about to use them to make our packet size constant, and this
will upset the tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2026-02-18 14:13:25 +10:30
Rusty Russell
facf24b6ee devtools/gossmap-compress: create latest gossip_store version
This saves gossipd from converting it:

```
lightningd-1 2026-02-02T00:50:49.505Z DEBUG   gossipd: Time to convert version 14 store: 890 msec
```

Reducing node startup time from 1.4 seconds to 0.5 seconds.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2026-02-16 17:23:33 +10:30
Rusty Russell
e8fd235d4e common: move gossip_store_wire.csv into common/ from gossipd/
It's used by common/gossip_store.c, which is used by many things other than
gossipd.  This file belongs in common.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2026-02-16 17:23:33 +10:30
Rusty Russell
b1055aa0ac gossip_store: add UUID entry at front of the store.
We also put this in the store_ended message, too: so you can
tell if the equivalent_offset there really refers to this new
entry (or if two or more rewrites have happened).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2026-02-16 17:23:33 +10:30
Rusty Russell
120c9d8ce4 devtools/gossmap-compress: generate better scids.
Our poor scid generation clashes badly with simplified hashing (the
next patch) leading to l1's startup time when using a generated map
moving from 4 seconds to 14 seconds.  Under CI it actually timed out
several tests.

Fixing our fake scids to be more "random" reduces it to 1.5 seconds.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2026-02-16 17:23:33 +10:30
Rusty Russell
e5e5998cd8 devtools: enhance dump-gossipstore to show some details of messages.
Not a complete decode, just the highlights (what channel was announced
or updated, what node was announced).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2026-02-16 17:23:33 +10:30
Se7enZ
5ca181ec89 devtools: Add custom include-order-fixer to pre-commit.
Also fixes some exising file spacing issues. Preserves whitespace and
comments.

Assisted by Cursor Auto.
2026-01-24 09:40:49 +10:00
Se7enZ
27482957a9 devtools: Add fix-style-errors script by @sangbida.
A helper tool to fix code style errors using `ruff` and `clang-format`
and correct spelling.
2026-01-24 09:40:49 +10:00
Christian Decker
686ce98906 git: Remove generated binary 2025-12-08 16:37:02 +01:00
Christian Decker
4b9cffe183 Add comprehensive coverage infrastructure with clang source-based coverage
This commit introduces a modern coverage infrastructure for Core Lightning:

- Migrate from ad-hoc coverage script to integrated Makefile targets
- Add LLVM source-based coverage support with per-test profraw organization
- Integrate coverage collection into pytest framework via TailableProc
- Add GitHub Actions workflow for nightly coverage reports
- Add Taskfile.yml for convenient task automation
- Add codecov.yml for Codecov integration
- Add comprehensive coverage documentation in COVERAGE.md
- Update contributor workflow docs with new coverage script path
- Add coverage data files to .gitignore (*.profraw, *.profdata)
- Remove obsolete contrib/clang-coverage-report.sh
- Remove obsolete tests/conftest.py (now using pyln-testing markers)
- Update pyproject.toml to include pyln-testing in main dependencies

The new infrastructure automatically collects coverage data when CLN_COVERAGE_DIR
is set, organizing profraw files by test name for granular analysis.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-08 16:37:02 +01:00
ShahanaFarooqui
af5d02a9fa gitignore: Ignore lightning-downgrade and check-bolt tools
Changelog-None.
2025-12-03 06:24:04 -08:00
Rusty Russell
76ad32ec88 check-bolt: move to devtools, clean up tools/
And leave $(TOOLS) as user-visible tools.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
35f65c5d91 common: add amount_msat_deduct / amount_msat_deduct_sub.
I added amount_msat_accumulate for the "a+=b" case, but I was struggling
with a name for the subtractive equivalent.  After some prompting, ChatGPT
suggested deduct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-17 10:56:18 +10:30
ShahanaFarooqui
8fe749e881 gitignore: Add generated files to gitignore 2025-11-15 05:25:25 +05:30
Rusty Russell
8b9020d7b9 global: use clock_time in place of time_now().
Except for tracing, that sticks with time_now().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:21:29 +10:30
Rusty Russell
565f7deec0 connectd: at disconnected, tell lightningd how long we were connected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-12 13:58:43 +10:30
Rusty Russell
82fff3c74b Remove litecoin support.
No idea if it works, we don't test it and nobody runs it.  I guess not.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Config: non-functioning litecoin support (who knew we even had that?)
2025-10-26 20:31:45 +10:30
Rusty Russell
42f9361375 ccan: update to get improved grab_file API, and adapt code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-26 12:37:58 +10:30
Rusty Russell
c821e41085 common: assert that they don't call setup_tmpctx() twice.
Otherwise, leaks will occur.  And fix up dualopend and devtool/route,
which do this!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-23 06:44:04 +10:30
Rusty Russell
6e5cb299dd global: remove unnecessary includes from C files.
Basically, `devtools/reduce-includes.sh */*.c`.

Build time from make clean (RUST=0) (includes building external libs):

Before:
	real    0m38.944000-40.416000(40.1131+/-0.4)s
	user    3m6.790000-17.159000(15.0571+/-2.8)s
	sys     0m35.304000-37.336000(36.8942+/-0.57)s
After:
	real    0m37.872000-39.974000(39.5466+/-0.59)s
	user    3m1.211000-14.968000(12.4556+/-3.9)s
	sys     0m35.008000-36.830000(36.4143+/-0.5)s

Build time after touch config.vars (RUST=0):

Before:
	real    0m19.831000-21.862000(21.5528+/-0.58)s
	user    2m15.361000-30.731000(28.4798+/-4.4)s
	sys     0m21.056000-22.339000(22.0346+/-0.35)s

After:
	real    0m18.384000-21.307000(20.8605+/-0.92)s
	user    2m5.585000-26.843000(23.6017+/-6.7)s
	sys     0m19.650000-22.003000(21.4943+/-0.69)s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-23 06:44:04 +10:30
Rusty Russell
bc44d3c629 devtools/reduce-includes.sh: don't remove our own .h from .c file includes.
Even if we would currently include it indirectly, we must include it directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-23 06:44:04 +10:30
Rusty Russell
f6a4e79420 global: remove unnecessary includes from headers.
Each header should only include the other headers it needs to compile;
`devtools/reduce-includes.sh */*.h` does this.  The C files then need
additional includes if they don't compile.

And remove the entirely useless wire/onion_wire.h, which only serves to include wire/onion_wiregen.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-23 06:44:04 +10:30
Rusty Russell
e120f87083 Makefile: create a library containing common, wire and bitcoin objects.
This means we don't have to manually choose what to link against,
which is much of the complexity of our Makefiles: the compiler will
automatically use any object files it needs to link.

We already do this for ccan as libccan.a, now we have libcommon.a.

We don't link against it for *everything*, as some tests require their own
versions.

Notes:
1. I get rid of the weird plugins/test/Makefile2 (accidental commit?)
2. Many tests change due to update-mocks.
3. In some places I added the missing dependency on the Makefile itself, though most are in the next
   patch.

Before:
	Total program size:     221366528
	Total tests size:       364243856

After:
	Total program size:     190733656
	Total tests size:       337880888

Build time from make clean (RUST=0) (includes building external libs):

Before:
	real    0m38.227000-44.245000(41.8222+/-1.6)s
	user    3m2.105000-33.696000(23.1442+/-8.4)s
	sys     0m35.054000-42.269000(39.7231+/-2)s
After:
	real    0m38.944000-40.416000(40.1131+/-0.4)s
	user    3m6.790000-17.159000(15.0571+/-2.8)s
	sys     0m35.304000-37.336000(36.8942+/-0.57)s

Build time after touch config.vars (RUST=0):

Before:
	real    0m18.928000-22.776000(21.5084+/-1.1)s
	user    2m8.613000-36.567000(27.7281+/-7.7)s
	sys     0m20.458000-23.436000(22.3963+/-0.77)s

After:
	real    0m19.831000-21.862000(21.5528+/-0.58)s
	user    2m15.361000-30.731000(28.4798+/-4.4)s
	sys     0m21.056000-22.339000(22.0346+/-0.35)s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

rusty@rusty-Framework:~/devel/cvs/lightni
2025-10-23 06:44:04 +10:30
Rusty Russell
559e3fc447 common: fix bolt12 quotes to bring them up-to-date.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-01 15:31:30 +09:30
Rusty Russell
54444e4337 BOLT12: Update recurrence to latest spec draft.
Changes:
* Fields renumbered to their draft values + billion.
* offer_recurrence now comes in compulsory or optional (backwards compat) flavors.
* `proportional_amount` is now inside `offer_recurrence_base` not `offer_recurrence_paywindow`.
* New field `invreq_recurrence_cancel`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Draft specification for recurring offers changed: old recurring offers will no longer work.
2025-10-01 15:31:30 +09:30
Rusty Russell
e7ea57e130 BOLT12: Remove start_any_period from recurrence_base.
Offer_absolute_expiry should be used if you want to require starting at the start.

Changelog-EXPERIMENTAL: Protocol: BOLT 12 recurrence `start_any_period` removed, use expiry if you need to restrict when they can start using the offer.
2025-10-01 15:31:30 +09:30
Rusty Russell
05005475e3 BOLT12: Remove years from recurrence.
Use months instead.

Changelog-EXPERIMENTAL: Protocol: BOLT 12 recurrence `years` removed; use 12 x months.
2025-10-01 15:31:30 +09:30
Rusty Russell
0e4b6ca109 gossip_store: wait for completed bit on reading.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-01 13:29:33 +09:30
Rusty Russell
d8db51d879 devtools: create conversion tool for old gossip stores.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-01 13:29:33 +09:30
Rusty Russell
9fb8870f92 gossip: add COMPLETED bit to mark records which are complete.
This should detect partial writes more robustly, since we make a
separate pwrite() call to update this flag after the record is written.

Previously we were playing a bit loose with synchronization assumptions,
which seemed to work on Linux ext4, but not so well elsewhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-01 13:29:33 +09:30
Rusty Russell
7aec8cec53 common: remove unused push bit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-01 13:29:33 +09:30
Matt Whitlock
41d31dcd19 avoid UB when calling ctype functions
The character classification functions in <ctype.h> are designed to
classify characters returned by <stdio.h> getchar() and friends, which
return characters as signed integers in the range 0 to 255 or EOF. The
behavior of the ctype functions is undefined if they are passed a value
outside of that range, which may happen if they are passed a char-typed
value and the system's char type is signed.

<ccan/str/str.h> defines some inline utility functions that perform the
necessary cast to coerce a char-typed argument into the allowed value
range. Call these wrappers instead of the bare ctype functions when
classifying char-typed characters.

Changelog-None
2025-08-18 14:03:20 +09:30
Rusty Russell
0938d544ed gossipwith: add flag to insist that we receive all messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 17:35:39 +09:30
Alex Myers
0e080a1c22 devtools/credit: add aliases of previous namers
It was suggesting past namers as new namers.

Also fixed the count extraction so that @21M4TW would be
properly credited.

Changelog-None
2025-07-08 12:56:17 +09:30
Rusty Russell
72a8fa20f7 gossipwith: add --handle-pings flag.
Required for long gossips, where we might get timed out for not replying to pings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 22:40:45 +09:30
Lagrang3
5f0c24b441 add devtool/bip137-verifysignature utility
To validate BIP137 signatures produced by core-lightning in tests.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-05-13 13:19:03 +09:30
Rusty Russell
7a276bbe09 common/utxo: use a real type for the UTXO, not a boolean is_p2sh.
To actually evaluate spend cost, we need to know whether it's taproot or not.
Using an enum (rather than making callers examine the script) means we can
ensure all cases are handled.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-06 12:27:53 +09:30
Rusty Russell
e832784cba BOLTS: update which fixes BIP353 quotes, missing invreq field in invoices.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-03-18 14:30:58 +10:30
Rusty Russell
10f333b041 devtools/bolt12-cli: fix interpretation of blindedpay, amounts.
The old blindedpay fields would have an entry per hop, but that was changed
to a single per-path entry.  The devtools hadn't caught up.

Similarly, it didn't like descriptionless offer fields in invreqs and invoices.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-26 14:26:47 -06:00
Rusty Russell
6d961f6b13 devtools: allow encode with 9 flag (for features).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-14 22:17:21 +10:30
Rusty Russell
c3362b057c BOLT12: remove -offers from bolt12 quotes, update them.
Typo fixes and wording changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
ba3e85bb43 decode: handle new bip353 fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
e440799b5e devtools: have dump-gossipstore print malformed warnings to stdout.
If they go to stderr, you can't associate them with the record they're
talking about.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
fdfc7ce62f gossmap: add (and use) logging hook.
Default goes to stderr for LOG_UNUSUAL and higher.

We have to whitelist more cases in map_catchup so we don't spam the logs
with perfectly-expected (but ignored) messages though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
607b14fe12 common/gossmap: remove open-by-fd.
We only use it in one place, and that was simply to share an fd between
gossipd writing and gossipd reading, which may be causing our zfs problem
anyway.

In fact, it fixes a race if we don't have HAVE_PWRITEV.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
f7714613bd devtools/gossipwith: filter for max-messages.
So not every message type counts: this is useful when we want to get a specific number
of a specific type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-01-27 11:07:04 +10:30
Dusty Daemon
a21ae33b2d channeld: Add dynamic funding_pubkeys to channel_txs
In anticipation of adding support for rotating funding pubkeys during a splice, `channel_txs` is updated to support specifying these manually instead of using the channel’s funding pubkeys.

Changelog-None
2024-11-21 14:15:36 +10:30
Rusty Russell
e38ef42041 devtools: bolt11-cli encode command to make fake invoices.
This will let us make invoices for the fake network.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
ef475db478 common: sphinx_path_new to take explicit len.
Useful if associated_data is not a tal pointer (xpay wants this).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
6303a96ee2 devtools/gossmap-compress: make fake nodeids same for 32/64 bits.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-06 21:17:36 +10:30