Commit Graph

634 Commits

Author SHA1 Message Date
Rusty Russell
6e5cb299dd global: remove unnecessary includes from C files.
Basically, `devtools/reduce-includes.sh */*.c`.

Build time from make clean (RUST=0) (includes building external libs):

Before:
	real    0m38.944000-40.416000(40.1131+/-0.4)s
	user    3m6.790000-17.159000(15.0571+/-2.8)s
	sys     0m35.304000-37.336000(36.8942+/-0.57)s
After:
	real    0m37.872000-39.974000(39.5466+/-0.59)s
	user    3m1.211000-14.968000(12.4556+/-3.9)s
	sys     0m35.008000-36.830000(36.4143+/-0.5)s

Build time after touch config.vars (RUST=0):

Before:
	real    0m19.831000-21.862000(21.5528+/-0.58)s
	user    2m15.361000-30.731000(28.4798+/-4.4)s
	sys     0m21.056000-22.339000(22.0346+/-0.35)s

After:
	real    0m18.384000-21.307000(20.8605+/-0.92)s
	user    2m5.585000-26.843000(23.6017+/-6.7)s
	sys     0m19.650000-22.003000(21.4943+/-0.69)s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-23 06:44:04 +10:30
Rusty Russell
f6a4e79420 global: remove unnecessary includes from headers.
Each header should only include the other headers it needs to compile;
`devtools/reduce-includes.sh */*.h` does this.  The C files then need
additional includes if they don't compile.

And remove the entirely useless wire/onion_wire.h, which only serves to include wire/onion_wiregen.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-23 06:44:04 +10:30
Rusty Russell
e120f87083 Makefile: create a library containing common, wire and bitcoin objects.
This means we don't have to manually choose what to link against,
which is much of the complexity of our Makefiles: the compiler will
automatically use any object files it needs to link.

We already do this for ccan as libccan.a, now we have libcommon.a.

We don't link against it for *everything*, as some tests require their own
versions.

Notes:
1. I get rid of the weird plugins/test/Makefile2 (accidental commit?)
2. Many tests change due to update-mocks.
3. In some places I added the missing dependency on the Makefile itself, though most are in the next
   patch.

Before:
	Total program size:     221366528
	Total tests size:       364243856

After:
	Total program size:     190733656
	Total tests size:       337880888

Build time from make clean (RUST=0) (includes building external libs):

Before:
	real    0m38.227000-44.245000(41.8222+/-1.6)s
	user    3m2.105000-33.696000(23.1442+/-8.4)s
	sys     0m35.054000-42.269000(39.7231+/-2)s
After:
	real    0m38.944000-40.416000(40.1131+/-0.4)s
	user    3m6.790000-17.159000(15.0571+/-2.8)s
	sys     0m35.304000-37.336000(36.8942+/-0.57)s

Build time after touch config.vars (RUST=0):

Before:
	real    0m18.928000-22.776000(21.5084+/-1.1)s
	user    2m8.613000-36.567000(27.7281+/-7.7)s
	sys     0m20.458000-23.436000(22.3963+/-0.77)s

After:
	real    0m19.831000-21.862000(21.5528+/-0.58)s
	user    2m15.361000-30.731000(28.4798+/-4.4)s
	sys     0m21.056000-22.339000(22.0346+/-0.35)s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

rusty@rusty-Framework:~/devel/cvs/lightni
2025-10-23 06:44:04 +10:30
Madeline Vibes
766db61370 Meta: Adding changelog and version update for 25.09.1 2025-10-21 08:17:35 +05:30
ShahanaFarooqui
ea2f7607b8 tools: Read the correct default-key from gpgconf
Workflow error `gpg: using "4129A994AA7E9852" is thrown due to incorrect gpg parsing. Update the awk parsing logic to properly locate and extract the key fingerprint within the gpgconf --list-options output structure, ensuring automated signing uses the correct key.

Changelog-None.
2025-10-20 16:07:29 +10:30
Sangbida Chaudhuri
7b1c4874ed makefile: use SED from configure in most places.
Some simple cases are left alone, but anything called from make uses $SED.
2025-10-17 10:40:17 +10:30
Rusty Russell
92a6961937 tools: I cannot spell recurrEnce.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-01 15:31:30 +09:30
Claudio Raimondi
13c5db51a9 Refactor Dockerfile, Add TODOs
all these changelogs only apply to the Docker image.

Changelog-Added: added verification of GPG keys for the bitcoin and litecoin tarballs.
Changelog-Fixed: fixed compilation on all target architectures; each had their own bugs (poetry, missing packages...).
Changelog-Fixed: fixed cargo cross compilation. it was mistakenly using QEMU before.
Changelog-Fixed: fixed CPU compatibility bug described in issue 8456
Changelog-Changed: improve build time by 8.8x
Changelog-Changed: improve image size by 2.07x

more detailed changelog can be found on the PR: https://github.com/ElementsProject/lightning/pull/8429
2025-09-28 20:56:07 +05:30
ShahanaFarooqui
e92203c18f ci: Update sqlite3 version and shasums for noble repro build
Changelog-Fixed: Reproducible build for Ubuntu noble by updating sqlite3 version and shasums.
2025-09-16 13:05:18 -05:00
Rusty Russell
a0eb503618 doc: make sure v prefix is always on the version.
1. Put v prefix before NEW_VERSION
2. Change the checkin line to a one-liner.
3. Have build-release.sh check for the v prefix (with --force-version= you could omit it).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-09-10 12:38:12 +05:30
Rusty Russell
8ba6522c80 tools/build-release.sh: don't assume you need sudo to run docker.
I don't!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-09-10 12:38:12 +05:30
Rusty Russell
b0a9b55e65 build-release.sh: do make -j inside docker image.
Defaults to nproc, but you can set MAKEPAR=N to override it.

Timings on my laptop are only a little better, because Rust.

Before:
	time tools/build-release.sh bin-Fedora bin-Ubuntu
	...
	real	33m17.104s
	user	0m4.259s
	sys	0m3.605s

After:
	time tools/build-release.sh bin-Fedora bin-Ubuntu
	...
	real	25m25.556s
	user	0m4.297s
	sys	0m3.743s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-09-10 12:38:12 +05:30
Rusty Russell
b77d648630 tools/build-release.sh: always append version to SHA256SUMS file.
Telling users to rename it is a poor idea, and if you have multiple releases in your release/ dir
it will get confusing.  So always append -v25.09 and update docs accordingly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-09-10 12:38:12 +05:30
Rusty Russell
ef25f4bfa4 tools: fix indentation.
Whitespace changes only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-09-10 12:38:12 +05:30
Rusty Russell
6182b9d3fc tools: insist on lowdown for zipfile.
On GitHub where we build the zipfile, it exists, so we don't check out
the submodule.  If you do, your zipfile won't match!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-09-10 12:38:12 +05:30
Rusty Russell
cbc38c789f Release: do docker make inside uv run.
Here's the error inside docker:

```
wiregen common/status_wiregen.h
Traceback (most recent call last):
  File "/build/tools/generate-wire.py", line 27, in <module>
    from mako.template import Template
ModuleNotFoundError: No module named 'mako'
rm external/build-x86_64-redhat-linux/libwally-core-build/src/secp256k1/libsecp256k1.la
make: *** [Makefile:328: common/status_wiregen.h] Error 1
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-09-02 14:29:41 +09:30
Rusty Russell
6da111280b Release: fix build inside Fedora
```
 => => naming to docker.io/library/fedora                                                                                                                                                                 0.0s
Inside docker: starting build
Cloning into '/build'...
done.
Note: switching to 'ca533a084d7a7636b099de7f6326f549c5251dfc'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

Downloading cpython-3.12.11-linux-x86_64-gnu (download) (29.9MiB)
 Downloading cpython-3.12.11-linux-x86_64-gnu (download)
Using CPython 3.12.11
Resolved 120 packages in 2ms
error: No virtual environment found for Python 3.12; run `uv venv` to create an environment, or pass `--system` to install into a non-virtual environment
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-09-02 14:29:41 +09:30
Madeline Paech
d7d0e3222b update the versions for 25.09 2025-09-01 13:13:38 +09:30
madelinevibes
81838fabf9 updates for rc4: name changes and uv 2025-08-28 15:59:52 +09:30
madelinevibes
c25f0fafff small changes of any reference to rc2 to rc3 2025-08-27 15:29:16 +09:30
Lagrang3
7e5cf41b4e htlc_wire: fix crash when adding an HTLC
In line channeld/channeld_wiregen.c:832 `*added+i` is not a tal object hence
the instruction in common/htlc_wire.c:200 `tal_arr(ctx, struct tlv_field, 0);` crashes CLN.
This is fixed by stating that added_htlc is a a varsize_type.

Logs:

2025-08-16T02:25:28.640Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v25.05-200-g79b959b)V
...
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:95 (call_error) 0x54f6bc
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:169 (check_bounds) 0x54f75a
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:178 (to_tal_hdr) 0x54f782
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:193 (to_tal_hdr_or_null) 0x54f7c7
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:471 (tal_alloc_) 0x54ffe4
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:517 (tal_alloc_arr_) 0x5500c4
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: common/htlc_wire.c:200 (fromwire_len_and_tlvstream) 0x48d63d
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: common/htlc_wire.c:234 (fromwire_added_htlc) 0x48dd23
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: channeld/channeld_wiregen.c:832 (fromwire_channeld_got_commitsig) 0x4c61fa
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:2377 (peer_got_commitsig) 0x4549cb
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/channel_control.c:1552 (channel_msg) 0x4140fe
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/subd.c:560 (sd_msg_read) 0x461513
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:60 (next_plan) 0x544885
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:422 (do_plan) 0x544cea
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:439 (io_ready) 0x544d9d
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:455 (io_loop) 0x54665d
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x42d220
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1487 (main) 0x43280f

gdb inspection:
830             *added = num_added ? tal_arr(ctx, struct added_htlc, num_added) : NULL;
831             for (size_t i = 0; i < num_added; i++)
832                     fromwire_added_htlc(&cursor, &plen, *added + i);
(gdb) p i
$3 = 1

Changelog-None: crash introduced this release.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
[ Added test, removed Changelog --RR ]
2025-08-27 14:14:31 +09:30
madelinevibes
fcd92febad change version 25.09rc2 for release
replace rc1
2025-08-21 16:12:27 +09:30
ShahanaFarooqui
e2e9ba3336 script: Poetry migration to uv for Fedora build
Fixes current error:
```
ERROR: Invalid requirement: 'Updating dependencies': Expected end or semicolon (after name and no valid version specifier)
```
2025-08-20 08:46:43 +09:30
Sangbida Chaudhuri
da7d3057ed Run version script for rc1.
Run version script
2025-08-19 16:46:29 +09:30
Alex Myers
79b5695835 reckless: reduce uv verbosity and avoid flooding output
This was overloading the reckless-rpc plugin input when outputting
json all in one shot. The verbosity was mostly dependency resolution
which wasn't all that helpful so call uv pip install as normal.

Changelog-None: bug introduced this release.
2025-08-18 14:04:48 +09:30
Alex Myers
b2ff500cce reckless: store source locations as correct type 2025-08-18 14:04:48 +09:30
Peter Neuroth
5c1fd782ec tools: Remove lockfiles from spell-checking
The rare case happened where a lockfile sha-sum contained a "Ctlv" which
spell-check complained about. Stupid lockfiles that don't know it is
actually "cltv"!

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Peter Neuroth
5e620cabce tools: Filter "highlight" case insensitive
There was a problem with a ‘highlight’ that was misunderstood as a
spelling mistake in lib-wally. Since ‘hightlight’ is already filtered
out, we simply instruct grep to ignore upper/lower case when filtering.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Alex Myers
ebbe5a274d reckless: add uv installer support for legacy projects
Those that only have a requirements.txt can be installed
with uv even if it's not managing the project requirements.
2025-08-12 09:28:21 +09:30
Alex Myers
1bb809a6e2 reckless: add uv python env installation method
uv is a python installation and package manager written in
rust.  We can use it to quickly install python package
dependencies and configure our plugin's python virtual environment.
To maintain consistency with our other reckless python
installations, the venv is still activated in a wrapper which then
imports the original python source.

Changelog-added: reckless can now install python plugins using the uv package manager.
2025-08-12 09:28:21 +09:30
Alex Myers
0364282eb2 reckless: correct direct install from local repo subdirectory 2025-08-12 09:28:21 +09:30
Alex Myers
bd26f726f1 reckless: fix installer search
Some installer procedures have more options for valid entypoint
names than others. We iterate through each of their first choices,
then their second choices, etc..
2025-08-12 09:28:21 +09:30
Rusty Russell
afffb534b4 tools: enforce minimum sqlite3 version number.
This is RHEL8's version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: build: we now require sqlite3 version 3.26 or above (released 2018-12-01).
2025-08-11 11:08:43 +09:30
ShahanaFarooqui
c1ad55c601 ci: Shasums updated for libsqlite in Ubuntu Noble
Changelog-None.
2025-07-29 16:28:38 -05:00
Matt Whitlock
cc0f66f07b hsmtool: implement new "derivetoremote" method
This method has a similar purpose as "guesstoremote" but is for use when
the channel's database ID is known. It produces the private key that can
spend the to_remote output of the peer's commitment transaction. It
assumes the channel was negotiated with option_static_remotekey unless
the optional per-commitment point argument is provided.

Changelog-Added: hsmtool has a new `derivetoremote` method.
2025-07-08 12:21:24 +09:30
Alex Myers
2b8b709bfd meta: update changelog for v25.05
Changelog-None
2025-06-16 13:10:33 -05:00
ShahanaFarooqui
ad2dc972de ci: Fix noble repro build by updating sqlite3 version and shasums
The reproducible build is currently failing on Ubuntu Noble, causing the daily `Repro Build Nightly` GitHub action to fail and triggering email notifications.

This update resolves the issue by adjusting the default sqlite3 version and checksums to match the packages now available in Noble, allowing the image to build successfully.

Changelog-None.
2025-06-05 17:38:55 -05:00
Alex Myers
8d6ae149ff release: update version for 25.05rc1
Changelog-None
2025-05-19 14:12:55 -05:00
Matt Whitlock
968bb63739 doc: properly handle ``preformatted blocks``
Lowdown requires a blank line before all preformatted blocks, or it doesn't
recognize them. `tools/md2man.sh` contained some ad-hoc efforts at fixing up
some locations where these required blank lines are absent from the output of
`tools/fromschema.py`, but it missed some. Instead of playing Whack-a-Mole, use
a blanket sed expression to ensure that a blank line precedes _every_ opening
```.

`esc_underscores(…)` in `tools/fromschema.py` did not work correctly on strings
containing an odd number of backticks, notably the ``` delimiters surrounding
preformatted text blocks. Specifically, it was dropping the last backtick since
none of the alternatives in the regex matched it. Add a new alternative that
matches a whole preformatted block as a single unit.

`output_member(…)` in `tools/fromschema.py` was passing each line of a member's
description through `esc_underscores(…)` individually, but that breaks
preformatted text blocks that are naturally multi-line and leads to mistakenly
escaping underscores inside such blocks. Rewrite the code to make use of the
`outputs(…)` utility function that joins all the provided lines together before
passing the whole text through `esc_underscores(…)`.

Drive-by fix a couple of flubbed preformatted blocks in schemas.

[ Added shellcheck suppression for md2man.sh --RR ]
Changelog-None
2025-05-15 16:06:08 +09:30
Alex Myers
70fa1a4b5a reckless: don't update a plugin if a specific tag was previously installed 2025-05-14 13:02:03 +09:30
Alex Myers
45d83aa01e reckless: don't return error if update is unnecessary 2025-05-14 13:02:03 +09:30
Alex Myers
666284d51f reckless: only proceed with update when appropriate 2025-05-14 13:02:03 +09:30
Alex Myers
cf767654ad reckless: return result from update 2025-05-14 13:02:03 +09:30
Alex Myers
eab84603ae reckless: provide user feedback at info level if enable fails 2025-05-14 13:02:03 +09:30
Alex Myers
741611b9a9 reckless: add update command
This updates all reckless-installed plugins with `reckless update` or
update individual plugins by passing the plugin names as arguments.

The metadata stored with the installed plugin is used to find the
plugin from the appropriate source (the same source is used as when
originally installed.)

Changelog-Added: Reckless: `reckless update` updates all reckless-installed plugins.
2025-05-14 13:02:03 +09:30
Alex Myers
f51fbc9459 reckless: refactor install
remove the duplicative search and extract the enable portion for use next.
2025-05-14 13:02:03 +09:30
Alex Myers
1401484ba0 reckless: store absolute paths in metadata 2025-05-14 13:02:03 +09:30
Alex Myers
e9d8397c72 reckless: handle a direct source in the form of a git repo url 2025-05-14 13:02:03 +09:30
Alex Myers
3e468be1ae reckless: accept a full local path as source+name
This allows installing a local plugin directly without having
to modify reckless sources.

Changelog-changed: Reckless can be passed a local file or directory for installation.
2025-05-14 13:02:03 +09:30
Alex Myers
884ab8e616 reckless: fix installation from local directories with subpaths
This could previously copy the parent directory of a plugin
into the installed reckless directory, which was unnecessary.
2025-05-14 13:02:03 +09:30