Commit Graph

586 Commits

Author SHA1 Message Date
Alex Myers
3e468be1ae reckless: accept a full local path as source+name
This allows installing a local plugin directly without having
to modify reckless sources.

Changelog-changed: Reckless can be passed a local file or directory for installation.
2025-05-14 13:02:03 +09:30
Alex Myers
884ab8e616 reckless: fix installation from local directories with subpaths
This could previously copy the parent directory of a plugin
into the installed reckless directory, which was unnecessary.
2025-05-14 13:02:03 +09:30
Rusty Russell
2154289911 hsmd: rename simple_htlc to hsm_htlc, don't gratuitously dynamically allocate.
The renaming makes it clear that it's HSM specific.

And it has no pointers, so we can have an array instead of an array of pointers.

I tested this hadn't accidentally changed the wire format by disabling
version checks and using an old hsmd with the altered daemons and
running the test suite.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-06 12:27:53 +09:30
Rusty Russell
daf1560eb4 hsmd: make our private utxo type, to ensure binary compatibility.
I'm about to update our utxo type, but Christian spotted that this is
part of the ABI for the hsm.  So make that a private "hsm_utxo" type,
to insulate it from changes.

In particular, the HSM versions only contain the fields that the
hsm cares about, and the wire format is consistent (even though that
*did* include some of those fields, they are now dummies).

In the long term, this should be removed from the ABI: once we
no longer have "close_info" utxos, this information should already be
in the PSBT.

I tested this hadn't accidentally changed the wire format by disabling
version checks and using an old hsmd with the altered daemons and
running the test suite.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-06 12:27:53 +09:30
ShahanaFarooqui
2d703fce8e tools: Removed lightning- for schema filename before generating markdowns 2025-04-15 15:17:14 +09:30
ShahanaFarooqui
162fcb754d tools: Create config.vars before ./configure 2025-03-05 18:25:24 -06:00
Alex Myers
e96af42947 meta: update changelog for 25.02
Changelog-None
2025-02-24 16:06:41 -06:00
Aditya Sharma
fe02d2f1c5 scb_wire: Define new subtype 'modern_scb_chan' with 'scb_tlvs'
We define a new subtype 'modern_scb_chan' and a new tlvtype 'scb_tlvs' which includes
all the relevant information to create a penalty transaction when the peer tries to cheat.

Key Changes:
 - Rename the old format to 'legacy_scb_chan' and define a new type 'modern_scb_chan'
 - Include TLVs to 'modern_scb_chan'
 - Create a new msgtype 'static_chan_backup_with_tlvs'
 - Modify 'struct channel' to include 'struct modern_scb_chan'
 - Add these two types to 'varsize_types' in generate.py
2025-02-22 11:51:54 -06:00
Aditya Sharma
6df6789744 Enable Length-Prefixed TLVs through Generator
This change allows adding a length prefix to a serialized TLV. It will
be particularly useful for serializing all 'scb_chan' entries in
the 'emergency.recover' file.

Key Changes:
 - Removed the need to loop in towire_tlv and fromwire_tlv, so the if conditions have been modified accordingly.
 - During serialization, the length of the TLV is calculated before appending it, and it is stored in a temporary variable.
 - For fromwire_tlv, only a simple length adjustment is required, and no loop is needed here either.
2025-02-22 11:51:54 -06:00
Aditya Sharma
39f63eb27f tools/gen: Enable Subtypes to have TLVStreams
This change will allow subtypes in wiregen files to have tlvstreams.
Shifting tlv structs above subtypes in header_template is done to prevent
forward declaration.

Since generate-wire prepends 'tlv_' in tlvname, we
have to modify fromwire_subtype_field and towire_subtype_field in
impl_template to accommodate this.

Changelog-Added: This PR would turn our peers into watchtower and enable SCB to create penalty txn.
2025-02-22 11:51:54 -06:00
ShahanaFarooqui
54555ce5e7 doc: Add new createrune example
- Moved the `Usage` section further down in `createrune` and `commando-rune` for improved UX.
- Added a new example for creating a rune with `read-only` restrictions, extending it to allow only payments of `less than 100,000 sats per day` using the `pay` or `xpay` methods.
- Adjusted formatting by appending an extra space after the `dependentUpon` condition, fixing `[*start* [*end*]][*relist*]` to `[*start* [*end*]] [*relist*]`.
- Relocated `Examples` from the expandable section to a standard heading, as examples are now already placed at the end of the page.

Changelog-None.
2025-02-04 20:12:59 +10:30
ShahanaFarooqui
7f73d312e8 ci: Fix for reproducible noble build failure
Changelog-None.
2025-01-30 09:34:01 +10:30
ShahanaFarooqui
b1fbbb0405 ci: Version update for Fedora reproducible build
Closes #7902.

Changelog-None.
2025-01-28 14:50:45 +10:30
ShahanaFarooqui
24e0929c0e tools: Add testnet4 support for reckless and hsmtool
Changelog-Added: Support for Bitcoin `testnet4`
2025-01-13 15:45:19 -08:00
Se7enZ
79273220f4 build: Add architecture to Ubuntu repro artifacts ([#6228]).
Changelog-Changed: Added architecture identifier to Ubuntu release
artifacts. ([#6228])
2024-12-03 11:56:23 +10:30
Se7enZ
f789f9cb21 build: Allow for signing releases without building a zip.
Adds a `--without-zip` parameter default `false` which optionally
skips the archive, to allow for signing releases from CI.
2024-12-03 11:56:23 +10:30
Se7enZ
b9b81c6918 build: Matrix strategy for release builds ([#7776]).
Also modifies the `build-release.sh` script to optionally
accept specific distribution targets for Ubuntu releases.
2024-12-03 11:56:23 +10:30
Se7enZ
c99b72da1c build: Release build automation ([#7776]).
Changelog-None
2024-12-03 11:56:23 +10:30
Rusty Russell
b5d1acee45 wire: call unknown types "UNKNOWN X" not "INVALID X".
It's freaking people out when they see things like:

```
 2024-11-11T05:26:41.281Z DEBUG ...53c-connectd: peer_out INVALID 22859
```

Fixes: https://github.com/ElementsProject/lightning/issues/7802
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: connectd: log unknown messages as "UNKNOWN" not "INVALID" to avoid freaking people out.
2024-12-02 15:09:45 +10:30
Matt Whitlock
67d667db90 tools/headerversions.c: fix build without SQLite
Commit 531845971c broke the build without
SQLite because this code:

	new = tal_fmt(NULL, template,
	              IF_SQLITE3(sqlite3_libversion_number()));

preprocesses into:

	new = tal_fmt(NULL, template,
	              );

which has a syntax error. Fix it by moving the comma into the macro
argument.

Fixes: 531845971c
Changelog-None
2024-11-19 09:48:31 +10:30
Rusty Russell
38d229df3d tools/generate-wire.py: make sure TLV array fields are allocated off TLV.
Otherwise the whole thing cannot be tal_steal() onto a different parent.

Here's the difference in generated files:

   --- ./wire/onion_wiregen.c.pre	2024-10-23 12:26:09.023176933 +1030
   +++ ./wire/onion_wiregen.c	2024-10-23 12:26:52.434828303 +1030
   @@ -128,7 +128,7 @@
     	blinded_path->path = num_hops ? tal_arr(blinded_path, struct blinded_path_hop *, 0) : NULL;
    	for (size_t i = 0; i < num_hops; i++) {
    		struct blinded_path_hop * tmp;
   -		tmp = fromwire_blinded_path_hop(blinded_path, cursor, plen);
   +		tmp = fromwire_blinded_path_hop(blinded_path->path, cursor, plen);
    		tal_arr_expand(&blinded_path->path, tmp);
    	}
    
   --- ./wire/bolt12_wiregen.c.pre	2024-10-23 12:26:09.079176474 +1030
   +++ ./wire/bolt12_wiregen.c	2024-10-23 12:26:52.612826902 +1030
   @@ -316,7 +316,7 @@
    	r->offer_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->offer_paths, cursor, plen);
    		tal_arr_expand(&r->offer_paths, tmp);
    	}
    }
   @@ -729,7 +729,7 @@
    	r->offer_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->offer_paths, cursor, plen);
    		tal_arr_expand(&r->offer_paths, tmp);
    	}
    }
   @@ -1052,7 +1052,7 @@
    	r->invreq_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->invreq_paths, cursor, plen);
    		tal_arr_expand(&r->invreq_paths, tmp);
    	}
    }
   @@ -1385,7 +1385,7 @@
    	r->offer_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->offer_paths, cursor, plen);
    		tal_arr_expand(&r->offer_paths, tmp);
    	}
    }
   @@ -1708,7 +1708,7 @@
    	r->invreq_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->invreq_paths, cursor, plen);
    		tal_arr_expand(&r->invreq_paths, tmp);
    	}
    }
   @@ -1781,7 +1781,7 @@
    	r->invoice_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->invoice_paths, cursor, plen);
    		tal_arr_expand(&r->invoice_paths, tmp);
    	}
    }
   @@ -1808,7 +1808,7 @@
    	r->invoice_blindedpay = *plen ? tal_arr(r, struct blinded_payinfo *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_payinfo * tmp;
   -		tmp = fromwire_blinded_payinfo(r, cursor, plen);
   +		tmp = fromwire_blinded_payinfo(r->invoice_blindedpay, cursor, plen);
    		tal_arr_expand(&r->invoice_blindedpay, tmp);
    	}
    }
   @@ -1927,7 +1927,7 @@
    	r->invoice_fallbacks = *plen ? tal_arr(r, struct fallback_address *, 0) : NULL;
    	while (*plen != 0) {
    		struct fallback_address * tmp;
   -		tmp = fromwire_fallback_address(r, cursor, plen);
   +		tmp = fromwire_fallback_address(r->invoice_fallbacks, cursor, plen);
    		tal_arr_expand(&r->invoice_fallbacks, tmp);
    	}
    }
   
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
9af01b062c pytest: fix hsmtool which reports leak under address sanitizer.
Couldn't figure out why hsmtool.proc.wait(WAIT_TIMEOUT) returns 1?
hsmtool doesn't ever seem to exit status 1!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
niftynei
e66653fa1d hsmtool: fixup show usage parens + return 2024-11-15 12:48:44 +10:30
Rusty Russell
b7d77b49df hsmtool.c: extend generatehsm to allow command-line args.
Based on the patch by bstin <barry.github@capsmx.com>, which added a separate command,
this simply extends "generatehsm" to allow more options.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: hsmtool: generatehsm can run non-interactive, taking options on the cmdline.
2024-11-15 12:48:44 +10:30
Rusty Russell
ab6f405ae7 common/hsm_encryption: use const char * for errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-13 08:49:55 +10:30
Rusty Russell
dca3a8c72c hsmtool: fix NULL secp256k1_ctx when calling dumpcommitments
secp256k1_ctx is used by pubkey_from_node_id.  Don't try to pick and
choose where to initialize secp256k1_ctx, just always do it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-06 21:17:36 +10:30
Chris Guida
c7189f644a make: Change mv to mv -f when replacing tools/headerversions file
Whenever we build without `make clean` first, this file gets
overwritten. It asks the user for input because it is not run with
`-f`. This causes the build to hang until the user inputs `y`.

Here is an example build:

```
$ make -j7 && sudo make install
CC: cc -DCLN_NEXT_VERSION="v24.08" -DPKGLIBEXECDIR="/usr/local/libexec/c-lightning" -DBINDIR="/usr/local/bin" -DPLUGINDIR="/usr/local/libexec/c-lightning/plugins" -DCCAN_TAL_NEVER_RETURN_NULL=1 -Wall -Wundef -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wold-style-definition -Werror -Wno-maybe-uninitialized -Wshadow=local -std=gnu11 -g -fstack-protector-strong -Og -I ccan -I external/libwally-core/include/ -I external/libwally-core/src/secp256k1/include/ -I external/jsmn/ -I external/libbacktrace/ -I external/gheap/ -I external/build-x86_64-linux-gnu/libbacktrace-build -I external/libsodium/src/libsodium/include -I external/libsodium/src/libsodium/include/sodium -I external/build-x86_64-linux-gnu/libsodium-build/src/libsodium/include -I . -I/usr/local/include  -I/usr/include/postgresql    -DSHACHAIN_BITS=48 -DJSMN_PARENT_LINKS  -DCOMPAT_V052=1 -DCOMPAT_V060=1 -DCOMPAT_V061=1 -DCOMPAT_V062=1 -DCOMPAT_V070=1 -DCOMPAT_V072=1 -DCOMPAT_V073=1 -DCOMPAT_V080=1 -DCOMPAT_V081=1 -DCOMPAT_V082=1 -DCOMPAT_V090=1 -DCOMPAT_V0100=1 -DCOMPAT_V0121=1  -c -o
LD: cc   -Og  config.vars  -Lexternal/build-x86_64-linux-gnu -lwallycore -lsecp256k1 -ljsmn -lbacktrace -lsodium -L/usr/local/include -lm -lsqlite3   -L/usr/lib/x86_64-linux-gnu -lpq -o
mv: replace 'tools/headerversions', overriding mode 0755 (rwxr-xr-x)? cc lightningd/test/run-check_node_announcement.c
cc lightningd/test/run-find_my_abspath.c
cc lightningd/test/run-invoice-select-inchan.c
cc lightningd/test/run-jsonrpc.c
cc lightningd/test/run-log_filter.c
cc lightningd/test/run-log-pruning.c
cc lightningd/test/run-shuffle_fds.c
ld lightningd/test/run-find_my_abspath
ld lightningd/test/run-log-pruning
ld lightningd/test/run-check_node_announcement
ld lightningd/test/run-log_filter
ld lightningd/test/run-jsonrpc
ld lightningd/test/run-shuffle_fds
ld lightningd/test/run-invoice-select-inchan
^Cmake: *** [tools/Makefile:16: tools/headerversions] Interrupt
```

One workaround is to just know that you need to enter `y` here.
But the best solution is probably to update the Makefile like so.

Changelog-None
2024-10-14 15:00:36 +10:30
Rusty Russell
00e0b99d3a hsmtool: provide nodeid from hsm secret.
This allows tools to validate that it is accessing the correct hsm_secret for this node!

This is extremely important for backups: if they are using VLS, they need to back *that*
up instead, for example.

Changelog-Added: `hsmtool`: `getnodeid` command derives the node id from the hsm_secret, to verify it's the correct secret.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-25 12:07:40 -07:00
ShahanaFarooqui
1a71e15cf3 github: Read tag on HEAD in Github Actions
Running `git describe` in local setup correctly identifies the tag available on the tag. But `docker buildx` via actions does not identify it without `--tags` param.

Reference Issue: https://github.com/ElementsProject/lightning/issues/7626

Changelog-Fixed: Docker image created via github actions correctly reads the tag available on the HEAD.
2024-09-05 12:31:49 -07:00
ShahanaFarooqui
48fa438b15 script: Updating SHASUMS for updated zlib1g-dev in Ubuntu Noble
Changelog-Fixed: core lightning's reproducible build for Ubuntu v24.04 (noble).
2024-08-26 17:10:22 -07:00
ShahanaFarooqui
50faa7c122 script: Auto generate rpc examples for documentation
I will commit doc/schemas/lightning-*.json files separately to keep this commit easier to review.

- test-autogenerate-rpc-examples updates all example request & responses in doc/schemas/lightning-*.json files.

- Updated tools/fromschema.py to accommodate the sql JSON example requirement where it does not accept -o in the query but shell does (for queries containing the = sign).

Changelog-None.
2024-08-09 23:56:45 -07:00
Alex Myers
48258ecb31 reckless: correct logging levels
Some output was hidden under normal operation which should
not have been.
2024-08-08 11:33:39 +09:30
Alex Myers
fe58c731ed reckless: escape strings in log 2024-08-08 11:33:39 +09:30
Alex Myers
9a7481b532 reckless: handle failure to find entrypoint cleanly 2024-08-08 11:33:39 +09:30
Alex Myers
dff7b50040 reckless: handle other --json cases without crashing 2024-08-08 11:33:39 +09:30
Alex Myers
efd67b36f0 reckless: handle unresolvable situations in a --json friendly way 2024-08-08 11:33:39 +09:30
Alex Myers
4aef72648c reckless: accept json array arguments as input
Changelog-Added: Reckless: accepts json array input for command targets
2024-08-08 11:33:39 +09:30
Alex Myers
1ac6e25ffd reckless: don't polute stdout with python install
status if --json was requested.
2024-08-08 11:33:39 +09:30
Alex Myers
bb47bc1d4a reckless: all command functions return objects to enable json out
This more easily allows list output for commands accepting list
input, i.e., installing 3 plugins produces 3 outputs.
2024-08-08 11:33:39 +09:30
Alex Myers
a2e458047f reckless: add json output option
Also redirect config creation prompts to stderr in order to not interfere
with json output on stdout.

Changelog-Added: reckless provides json output with option flag -j/--json
2024-08-08 11:33:39 +09:30
Alex Myers
75d8d8b91f reckless: add logger class
This will allow redirection of json output in the following commits.
2024-08-08 11:33:39 +09:30
Alex Myers
40c24065f7 reckless: make options flags position independent
Changelog-Changed: Reckless option flags are now position independent.
2024-08-08 11:33:39 +09:30
Alex Myers
0a6c58133d reckless: add version information to reckless 2024-08-08 11:33:39 +09:30
Alex Myers
9cbaafb525 reckless: add installer for rust plugins
This follows the same structure that enables python virtual environments:
  reckless/
    <plugin_name>/
      <symlink to compiled bin>
      source/
        <clone of original source plugin dir>/

Changelog-Added: Reckless: added the ability to install rust plugins.
2024-08-08 11:33:39 +09:30
Alex Myers
ed2c0ae18f reckless: place source code in source/plugin_name/
rather than just source/.  This is required for cargo install, so let's just
use this paradigm globally.
2024-08-08 11:33:39 +09:30
Alex Myers
c1d46d6ef9 reckless: add type hints for InstInfo 2024-08-08 11:33:39 +09:30
Alex Myers
7387c9cb1f reckless: remove mimetype
The python mimetype package wasn't useful enough in indentifying
filetypes anyhow.
2024-08-08 11:33:39 +09:30
Rusty Russell
a1fdeee76b Makefile: clean up install path handling.
We had some weird code to try to do relative paths.  Instead, use absolute
ones and keep life simple.  Also rename "daemon_dir" to the clearer
"subdaemon_dir" as that's what it's used for.

We now need special magic to do "installcheck", which is a bit awkward,
but at least it's contained.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 18:32:56 +02:00
Rusty Russell
531845971c gossipd: drop zlib support.
This was removed from the spec on Apr 25, 2022.  We stopped ever sending them
in 0.12.0 (2022-08-23).  Now we remove receive support.

Changelog-Protocol: Removed support for zlib-compressed short-channel-ids in query responses.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-10 12:21:19 +09:30
Rusty Russell
fd9975c288 reckless: don't use time-based directories.
This always fails on my test machine, which runs many tests in parallel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30