When installing core lightning on macOS I found that the debug symbols were not being preserved leading to "no debug info in
Mach-O executable". This is because the .dSYM files, which contain the debug information, is generated in the build directory and could not be found by the installed binaries.
Changes:
- Add -fno-standalone-debug flag to CDEBUGFLAGS on macOS to reduce
dependency on absolute source paths
- Modify install-program target to copy .dSYM bundles alongside
binaries for BIN_PROGRAMS, PKGLIBEXEC_PROGRAMS, and PLUGINS
Testing:
1. ./configure --reconfigure
2. make install PREFIX=/tmp/lightning-install
3. make clean (removes all .dSYM files from build directory)
4. /tmp/lightning-install/bin/lightningd --help
5. Verified stack traces now work correctly without "no debug info" errors
On macOS, libbacktrace was failing to find debug information due to:
1. Debug symbols not being properly linked with dsymutil
2. Apple Clang 17.0.0 generating DWARF 5 which libbacktrace couldn't parse
In this commit we address both issues:
Debug symbol accessibility:
- Add dsymutil integration in Makefile to properly link debug symbols
- Use -fno-standalone-debug to embed debug info inline in executable
DWARF format compatibility:
- Force -gdwarf-4 instead of default DWARF 5 to avoid "DW_FORM_addrx value out of range" errors
Changelog-added: libbacktrace works with macOS
Currently on I see link failures like the following:
```bash
./configure
<snip>
checking for libsodium with IETF chacha20 variants... yes
<snip>
ar libccan.a
ld ccan/ccan/cdump/tools/cdump-enumstr
ld: library 'sodium' not found
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
The configure check passes under Clang here.
Changelog-Fixed: build: fix linking against libsodium on macOS.
The current check fails when building with newer GCC, i.e:
```bash
error: ‘crypto_secretstream_xchacha20poly1305_init_push’ reading 32 bytes from a region of size 3 [-Werror=stringop-overread]
12 | crypto_secretstream_xchacha20poly1305_init_push(&crypto_state, header,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
13 | data);
```
This inadvertently results in the release binaries for Ubuntu 22.04 &
24.04, not having a runtime libsodium dependency, but instead using the
bundled lib.
It's not clear to me this is actually enough to fix (all) the release builds,
as the build containers need to have `libsodium-dev`, not just `libsodium`
in them, and it's not clear to me which packages are actually present
looking at the repro build scripts.
Changelog-Fixed: build: libsodium configure check fixed to work with newer GCC.
The test program has a leak, so address sanitizer complains and makes it
"fail" the zlib detection test!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We do this with typesafe_cb and it's so useful I'm not going to remove it. But clang 18 complains:
```
ccan/ccan/tal/tal.c:246:6: runtime error: call to function destroy_conn_close_fd through pointer to incorrect function type 'void (*)(void *)'
/home/rusty/devel/cvs/lightning/ccan/ccan/io/poll.c:251: note: destroy_conn_close_fd defined here
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
By default, UBSan reports runtime errors but does not stop execution.
We already abort in debug builds, and this commit makes us also abort in
regular builds when UBSan is enabled. Arguably, this is what users
expect when they enable UBSan, so it is a good default.
I know I've missed some UBSan bugs in the past because of this issue,
and dergoegge mentioned that this also happened to him.
I noticed this while working on a nix devShell to work on CLN. We are
blanked overriding the `pkg-config` search path, which can cause some
trouble. Specifically `nix` uses content addressable locations, and
macOS arm64 and x84_64 use separate `pkg-config` search paths, and by
overwriting it we can cause a mix of different architectures failing
the compilation
Changelog-Fixed: configure: We now respect the `PKG_CONFIG_PATH` environment variable
If you previously configured with `--enable-developer` we turn that into `--enable-debugbuild`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: build: `--enable-developer` arg to configure (and DEVELOPER variables): use `./configure --enable-debugbuild` and `developer` setting at runtime.
This controls debug flags for the build, rather than --developer,
which is going away.
I thought about making this flag control the RUST_PROFILE too, but
it seems that we want that set to "release" for CI, whereas for the
C code we want --enable-debugbuild.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Clang's coverage instrumentation [1] is the best I've seen, with
precision down to the expressions within a line of code. Add an option
to use this instrumentation for better coverage reports.
[1] https://clang.llvm.org/docs/SourceBasedCodeCoverage.html
Changelog-EXPERIMENTAL: Build: all experimental features are now runtime-enabled; no more ./configure --enable-experimental-features
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
By using fuzzer instrumentation for dependencies, we get more coverage
signal during fuzzing. This is useful when the fuzzer must figure out
how to take certain branches in a dependency.
In our case, the fuzz-bip32 target was failing to create a data buffer
that successfully passed fromwire_ext_key() parsing because the fuzzer
couldn't see what was happening inside libwally-core.
For example, if we use -fsanitize=undefined, we can't do unaligned
integer access, but since we didn't test with the sanitizer flags, we
didn't know this, and set `HAVE_UNALIGNED_ACCESS=1`.
Also, add -fno-sanitize-recover= in developer mode, so we actually
fail binaries if something is detected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The M1 Macs support both x86_64 and arm64 architectures, which forced
homebrew to use a different path for its storage (`/opt/homebrew/`
instead of `/usr/local`). If we don't adjust the path we'd mix x86_64
and arm64 libraries which can lead to weird compiler and linker
errors.
This patch just introduces `CPATH` and `LIBRARY_PATH` as suggested by
the homebrew team, and detects the current architecture automatically.
Changelog-Added: macos: Added m1 architecture support for macos
Mostly comments and docs: some places are actually paths, which
I have avoided changing. We may migrate them slowly, particularly
when they're user-visible.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We detect whether we have the rust tooling available (mainly `cargo`)
and enable or disable the rust libraries, plugins and examples when it
is enabled. Since the rest of the Makefiles assumes that executables
have an associated header and C source file, we also needed to add a
target that we can add non-C binaries to.
When looking for a pytest executable we should be looking for the ones
that `pip` installs in a virtualenv (`pytest` and `py.test`) before we
look for the ones that `apt` installs system-wide (`pytest3` and
`py.test3`) because these may not be part of the virtualenv that all
other packages are installed in.
This adds a new configuration, --enable-fuzzing (which is more than
welcome to be coupled with --enable-address-sanitizer), to pass the
fuzzer sanitizer argument when compiling objects. This allows libfuzzer
to actually be able "to fuzz" by detecting coverage and be smart when
mutating inputs.
As libfuzzer brings its own ~~fees~~ main(), we compile objects with
fsanitize=fuzzer-no-link, and special-case the linkage of the fuzz
targets.
A "lib" is added to abstract out the interface to the fuzzing tool used.
This allow us to use the same targets to fuzz using AFL, hongfuzz or w/e
by adding their entrypoints into libfuzz. (h/t to practicalswift who
introduced this for bitcoin-core, which i mimiced)
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
At least on my Ubuntu box, they're compatible. If they're not, we need
to disable regeneration altogether.
Fixes: #4075
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We alter the script to do the move to config.vars right at the end,
which is probably a good idea anyway.
Fixes: #4054
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
