clnrest's `utoipa-swagger-ui` library has an indirect `rust-embed` dependency which by default includes timestamps in build. It results in non-deterministic build for clnrest. Using environment variable `SOURCE_DATE_EPOCH` with fixed value will enforce a consistent timestamp for builds.
Also adding the `--locked` flag to ensure the release build uses exact dependencies from Cargo.lock. The `--locked` flag is particularly important for deterministic builds as it prevents Cargo from updating the lockfile.
Fixes#8288.
Changelog-Fixed: Core lightning builds for Ubuntu Focal, Jammy and Noble are deterministic again.
1. We need to replace versions in __init__.py
2. We need to run uv after changing versions, so it updates uv.lock
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Extract package versions from pyproject.toml directly
instead of using poetry commands. Use `uv run` to execute flake8,
pytest and other Python tools consistently.
Add new make commands for uv builds
When installing core lightning on macOS I found that the debug symbols were not being preserved leading to "no debug info in
Mach-O executable". This is because the .dSYM files, which contain the debug information, is generated in the build directory and could not be found by the installed binaries.
Changes:
- Add -fno-standalone-debug flag to CDEBUGFLAGS on macOS to reduce
dependency on absolute source paths
- Modify install-program target to copy .dSYM bundles alongside
binaries for BIN_PROGRAMS, PKGLIBEXEC_PROGRAMS, and PLUGINS
Testing:
1. ./configure --reconfigure
2. make install PREFIX=/tmp/lightning-install
3. make clean (removes all .dSYM files from build directory)
4. /tmp/lightning-install/bin/lightningd --help
5. Verified stack traces now work correctly without "no debug info" errors
On macOS, libbacktrace was failing to find debug information due to:
1. Debug symbols not being properly linked with dsymutil
2. Apple Clang 17.0.0 generating DWARF 5 which libbacktrace couldn't parse
In this commit we address both issues:
Debug symbol accessibility:
- Add dsymutil integration in Makefile to properly link debug symbols
- Use -fno-standalone-debug to embed debug info inline in executable
DWARF format compatibility:
- Force -gdwarf-4 instead of default DWARF 5 to avoid "DW_FORM_addrx value out of range" errors
Changelog-added: libbacktrace works with macOS
Changelog-Changed: wss-proxy.py was replaced by a rust version with support for multiple `wss-bind-addr`. If you install CLN from pre-compiled binaries you must remove the old wss-proxy directory first before installing CLN, usually
it is located in `/usr/local/libexec/c-lightning/plugins/wss-proxy`. If you compile from source `make` will take care of this automatically.
These checks are a SHOULD, but implementing them helps avoid anyone
making such weird things in future.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Doesn't change anything for us, since we will already fulfull the incoming
HTLCs if we can, but good to note.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
They weren't formatted correctly for bolts/tools/extract-formats.py
until this commit, so we had to patch them in manually.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When the command string run through $(call VERBOSE,…) contains a single-quote
character followed by shell metacharacters, the shell executing the VERBOSE
template's $(ECHO) command will attempt to interpret those metacharacters. This
could be disastrous, such as if someone were to put in a Makefile recipe:
$(call VERBOSE, , $(ECHO) 'Will not `rm -rf ~`')
When run with V=1, which causes VERBOSE to be defined as…
VERBOSE = $(ECHO) '$(2)'; $(2)
…Make would evaluate the above call into:
echo 'echo 'Will not `rm -rf ~`''; echo 'Will not `rm -rf ~`'
And oops, there goes the neighborhood.
The real-world motivating case for this fix is the sed call in the recipe for
doc/index.rst in doc/Makefile. It contains a sed expression enclosed in single
quotes, containing parentheses. When run through VERBOSE with V=1, the single
quotes around the sed expression actually escape _out_ of the single-quoted
string that is intended to be the whole command line, and you get:
/bin/sh: -c: line 1: syntax error near unexpected token `('
The fix is for VERBOSE to escape any single quotes embedded in the command line
argument when echoing it:
VERBOSE = $(ECHO) '$(subst ','\'',$(2))'; $(2)
Note that this is still wrong, as it will not do the right thing if $(2) happens
to begin with a hyphen, but I didn't want to introduce a new "PRINTF" variable
(or do something unsavory like calling cat with a here-doc) to squash a bug that
currently has no known manifestations.
Changelog-None
Unfortunately a spec typo means the data fields are missing (PR pending),
so we still patch those in.
The message "your_peer_storage" got renamed to "peer_storage_retrieval",
and the option "want_peer_backup_storage" was removed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: `experimental-peer-storage` now only advertizes feature 43, not 41.
We now have to explicitly enable various deprecated commando commands, and now
when deprecations are disabled, we honour missing MPP option in bolt12 invoices.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I should have done this immediately after last release :(.
This turns various things off by default, even if deprecated APIs are enabled.
We remove the test of the to-be-removed params.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Config: `rest-port`, `rest-protocol`, `rest-host` and `rest-certs` disabled by default (use clnrest-*, or `i-promise-to-fix-broken-api-user=rest-port.clnrest-prefix` etc and PLEASE REPORT if you need this!)
Changelog-Deprecated: Config: `max-locktime-blocks` disabled by default (use `i-promise-to-fix-broken-api-user=max-locktime-blocks` and PLEASE REPORT if you need this!)
plugins/Makefile has target/${RUST_PROFILE}/cln-grpc depend on the
generated files via $(MSGGEN_GENALL), but cln-rpc/Makefile adds to
that variable, so needs to be included first.
Here's an example build error:
```
Combining schemas from /home/rusty/lightning-ltest/doc/schemas into /home/rusty/lightning-ltest/contrib/msggen/msggen/schema.json
Created /home/rusty/lightning-ltest/contrib/msggen/msggen/schema.json from 2 files
error: failed to run custom build command for `cln-grpc v0.3.0 (/home/rusty/lightning-ltest/cln-grpc)`
Caused by:
process didn't exit successfully: `/home/rusty/lightning-ltest/target/debug/build/cln-grpc-95489e3ba33c0ab3/build-script-build` (exit status: 101)
--- stdout
cargo:rerun-if-changed=proto/node.proto
cargo:rerun-if-changed=proto
--- stderr
thread 'main' panicked at cln-grpc/build.rs:7:10:
called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: "protoc failed: node.proto:134:52: \"AskreneageResponse\" is not defined.\nnode.proto:135:23: \"GetroutesRequest\" is not defined.\nnode.proto:135:50: \"GetroutesResponse\" is not defined.\nnode.proto:136:32: \"AskrenedisablenodeRequest\" is not defined.\nnode.proto:136:68: \"AskrenedisablenodeResponse\" is not defined.\nnode.proto:137:34: \"AskreneinformchannelRequest\" is not defined.\nnode.proto:137:72: \"AskreneinformchannelResponse\" is not defined.\nnode.proto:138:34: \"AskrenecreatechannelRequest\" is not defined.\nnode.proto:138:72: \"AskrenecreatechannelResponse\" is not defined.\nnode.proto:139:34: \"AskreneupdatechannelRequest\" is not defined.\nnode.proto:139:72: \"AskreneupdatechannelResponse\" is not defined.\nnode.proto:140:32: \"AskrenebiaschannelRequest\" is not defined.\nnode.proto:140:68: \"AskrenebiaschannelResponse\" is not defined.\nnode.proto:141:37: \"AskrenelistreservationsRequest\" is not defined.\nnode.proto:141:78: \"AskrenelistreservationsResponse\" is not defined.\nnode.proto:142:32: \"InjectpaymentonionRequest\" is not defined.\nnode.proto:142:68: \"InjectpaymentonionResponse\" is not defined.\nnode.proto:143:18: \"XpayRequest\" is not defined.\nnode.proto:143:40: \"XpayResponse\" is not defined.\nnode.proto:145:33: \"StreamBlockAddedRequest\" is not defined.\nnode.proto:145:74: \"BlockAddedNotification\" is not defined.\nnode.proto:146:40: \"StreamChannelOpenFailedRequest\" is not defined.\nnode.proto:146:88: \"ChannelOpenFailedNotification\" is not defined.\nnode.proto:147:36: \"StreamChannelOpenedRequest\" is not defined.\nnode.proto:147:80: \"ChannelOpenedNotification\" is not defined.\nnode.proto:148:30: \"StreamConnectRequest\" is not defined.\nnode.proto:148:68: \"PeerConnectNotification\" is not defined.\nnode.proto:149:32: \"StreamCustomMsgRequest\" is not defined.\nnode.proto:149:72: \"CustomMsgNotification\" is not defined.\n" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
make: *** [plugins/Makefile:305: target/debug/cln-grpc] Error 101
make: *** Waiting for unfinished jobs....
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When building from a release zipfile and there is no `git` present, the version number lacks a ‘v’.
This triggers db.c to fail it’s `is_released_version` check, which fails because the version not start with a ‘v’ — thereby the database upgrade is rejected.
Issue reported by TonyV on Discord:
“Hey guys, I just upgraded to the latest release and am now getting
```
Refusing to irreversibly upgrade db from version 219 to 261 in non-final version 24.11.1 (use --database-upgrade=true to override)
```
I have the db backed up... but am I good to make those changes without breaking my channels?”
Changelog-Changed: Fix for people upgrading using source release zip packages.
Currently on I see link failures like the following:
```bash
./configure
<snip>
checking for libsodium with IETF chacha20 variants... yes
<snip>
ar libccan.a
ld ccan/ccan/cdump/tools/cdump-enumstr
ld: library 'sodium' not found
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
The configure check passes under Clang here.
Changelog-Fixed: build: fix linking against libsodium on macOS.
Due to Darwin-arm64 conditional setting of `CPPFLAGS`, the subsequent
`CPPFLAGS +=` is resolved earlier on ARM macOS which results in empty
paths being used.
Changelog-None
Our gossmap_store uncompresser generates nodeids with well-known
privkeys, so we can decrypt and respond to HTLCs sent to such nodes.
By replacing channeld with a fake, we can connect a node to another
node, but then once the channel is established, allow payments to be
sent into the generated network, and respond appropriately.
This minimal version handles MPP timeouts, but doesn't insert any
delays or runtime capacity for channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-None: Testing only
