Commit Graph

610 Commits

Author SHA1 Message Date
Alex Myers
79b5695835 reckless: reduce uv verbosity and avoid flooding output
This was overloading the reckless-rpc plugin input when outputting
json all in one shot. The verbosity was mostly dependency resolution
which wasn't all that helpful so call uv pip install as normal.

Changelog-None: bug introduced this release.
2025-08-18 14:04:48 +09:30
Alex Myers
b2ff500cce reckless: store source locations as correct type 2025-08-18 14:04:48 +09:30
Peter Neuroth
5c1fd782ec tools: Remove lockfiles from spell-checking
The rare case happened where a lockfile sha-sum contained a "Ctlv" which
spell-check complained about. Stupid lockfiles that don't know it is
actually "cltv"!

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Peter Neuroth
5e620cabce tools: Filter "highlight" case insensitive
There was a problem with a ‘highlight’ that was misunderstood as a
spelling mistake in lib-wally. Since ‘hightlight’ is already filtered
out, we simply instruct grep to ignore upper/lower case when filtering.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Alex Myers
ebbe5a274d reckless: add uv installer support for legacy projects
Those that only have a requirements.txt can be installed
with uv even if it's not managing the project requirements.
2025-08-12 09:28:21 +09:30
Alex Myers
1bb809a6e2 reckless: add uv python env installation method
uv is a python installation and package manager written in
rust.  We can use it to quickly install python package
dependencies and configure our plugin's python virtual environment.
To maintain consistency with our other reckless python
installations, the venv is still activated in a wrapper which then
imports the original python source.

Changelog-added: reckless can now install python plugins using the uv package manager.
2025-08-12 09:28:21 +09:30
Alex Myers
0364282eb2 reckless: correct direct install from local repo subdirectory 2025-08-12 09:28:21 +09:30
Alex Myers
bd26f726f1 reckless: fix installer search
Some installer procedures have more options for valid entypoint
names than others. We iterate through each of their first choices,
then their second choices, etc..
2025-08-12 09:28:21 +09:30
Rusty Russell
afffb534b4 tools: enforce minimum sqlite3 version number.
This is RHEL8's version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: build: we now require sqlite3 version 3.26 or above (released 2018-12-01).
2025-08-11 11:08:43 +09:30
ShahanaFarooqui
c1ad55c601 ci: Shasums updated for libsqlite in Ubuntu Noble
Changelog-None.
2025-07-29 16:28:38 -05:00
Matt Whitlock
cc0f66f07b hsmtool: implement new "derivetoremote" method
This method has a similar purpose as "guesstoremote" but is for use when
the channel's database ID is known. It produces the private key that can
spend the to_remote output of the peer's commitment transaction. It
assumes the channel was negotiated with option_static_remotekey unless
the optional per-commitment point argument is provided.

Changelog-Added: hsmtool has a new `derivetoremote` method.
2025-07-08 12:21:24 +09:30
Alex Myers
2b8b709bfd meta: update changelog for v25.05
Changelog-None
2025-06-16 13:10:33 -05:00
ShahanaFarooqui
ad2dc972de ci: Fix noble repro build by updating sqlite3 version and shasums
The reproducible build is currently failing on Ubuntu Noble, causing the daily `Repro Build Nightly` GitHub action to fail and triggering email notifications.

This update resolves the issue by adjusting the default sqlite3 version and checksums to match the packages now available in Noble, allowing the image to build successfully.

Changelog-None.
2025-06-05 17:38:55 -05:00
Alex Myers
8d6ae149ff release: update version for 25.05rc1
Changelog-None
2025-05-19 14:12:55 -05:00
Matt Whitlock
968bb63739 doc: properly handle ``preformatted blocks``
Lowdown requires a blank line before all preformatted blocks, or it doesn't
recognize them. `tools/md2man.sh` contained some ad-hoc efforts at fixing up
some locations where these required blank lines are absent from the output of
`tools/fromschema.py`, but it missed some. Instead of playing Whack-a-Mole, use
a blanket sed expression to ensure that a blank line precedes _every_ opening
```.

`esc_underscores(…)` in `tools/fromschema.py` did not work correctly on strings
containing an odd number of backticks, notably the ``` delimiters surrounding
preformatted text blocks. Specifically, it was dropping the last backtick since
none of the alternatives in the regex matched it. Add a new alternative that
matches a whole preformatted block as a single unit.

`output_member(…)` in `tools/fromschema.py` was passing each line of a member's
description through `esc_underscores(…)` individually, but that breaks
preformatted text blocks that are naturally multi-line and leads to mistakenly
escaping underscores inside such blocks. Rewrite the code to make use of the
`outputs(…)` utility function that joins all the provided lines together before
passing the whole text through `esc_underscores(…)`.

Drive-by fix a couple of flubbed preformatted blocks in schemas.

[ Added shellcheck suppression for md2man.sh --RR ]
Changelog-None
2025-05-15 16:06:08 +09:30
Alex Myers
70fa1a4b5a reckless: don't update a plugin if a specific tag was previously installed 2025-05-14 13:02:03 +09:30
Alex Myers
45d83aa01e reckless: don't return error if update is unnecessary 2025-05-14 13:02:03 +09:30
Alex Myers
666284d51f reckless: only proceed with update when appropriate 2025-05-14 13:02:03 +09:30
Alex Myers
cf767654ad reckless: return result from update 2025-05-14 13:02:03 +09:30
Alex Myers
eab84603ae reckless: provide user feedback at info level if enable fails 2025-05-14 13:02:03 +09:30
Alex Myers
741611b9a9 reckless: add update command
This updates all reckless-installed plugins with `reckless update` or
update individual plugins by passing the plugin names as arguments.

The metadata stored with the installed plugin is used to find the
plugin from the appropriate source (the same source is used as when
originally installed.)

Changelog-Added: Reckless: `reckless update` updates all reckless-installed plugins.
2025-05-14 13:02:03 +09:30
Alex Myers
f51fbc9459 reckless: refactor install
remove the duplicative search and extract the enable portion for use next.
2025-05-14 13:02:03 +09:30
Alex Myers
1401484ba0 reckless: store absolute paths in metadata 2025-05-14 13:02:03 +09:30
Alex Myers
e9d8397c72 reckless: handle a direct source in the form of a git repo url 2025-05-14 13:02:03 +09:30
Alex Myers
3e468be1ae reckless: accept a full local path as source+name
This allows installing a local plugin directly without having
to modify reckless sources.

Changelog-changed: Reckless can be passed a local file or directory for installation.
2025-05-14 13:02:03 +09:30
Alex Myers
884ab8e616 reckless: fix installation from local directories with subpaths
This could previously copy the parent directory of a plugin
into the installed reckless directory, which was unnecessary.
2025-05-14 13:02:03 +09:30
Rusty Russell
2154289911 hsmd: rename simple_htlc to hsm_htlc, don't gratuitously dynamically allocate.
The renaming makes it clear that it's HSM specific.

And it has no pointers, so we can have an array instead of an array of pointers.

I tested this hadn't accidentally changed the wire format by disabling
version checks and using an old hsmd with the altered daemons and
running the test suite.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-06 12:27:53 +09:30
Rusty Russell
daf1560eb4 hsmd: make our private utxo type, to ensure binary compatibility.
I'm about to update our utxo type, but Christian spotted that this is
part of the ABI for the hsm.  So make that a private "hsm_utxo" type,
to insulate it from changes.

In particular, the HSM versions only contain the fields that the
hsm cares about, and the wire format is consistent (even though that
*did* include some of those fields, they are now dummies).

In the long term, this should be removed from the ABI: once we
no longer have "close_info" utxos, this information should already be
in the PSBT.

I tested this hadn't accidentally changed the wire format by disabling
version checks and using an old hsmd with the altered daemons and
running the test suite.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-06 12:27:53 +09:30
ShahanaFarooqui
2d703fce8e tools: Removed lightning- for schema filename before generating markdowns 2025-04-15 15:17:14 +09:30
ShahanaFarooqui
162fcb754d tools: Create config.vars before ./configure 2025-03-05 18:25:24 -06:00
Alex Myers
e96af42947 meta: update changelog for 25.02
Changelog-None
2025-02-24 16:06:41 -06:00
Aditya Sharma
fe02d2f1c5 scb_wire: Define new subtype 'modern_scb_chan' with 'scb_tlvs'
We define a new subtype 'modern_scb_chan' and a new tlvtype 'scb_tlvs' which includes
all the relevant information to create a penalty transaction when the peer tries to cheat.

Key Changes:
 - Rename the old format to 'legacy_scb_chan' and define a new type 'modern_scb_chan'
 - Include TLVs to 'modern_scb_chan'
 - Create a new msgtype 'static_chan_backup_with_tlvs'
 - Modify 'struct channel' to include 'struct modern_scb_chan'
 - Add these two types to 'varsize_types' in generate.py
2025-02-22 11:51:54 -06:00
Aditya Sharma
6df6789744 Enable Length-Prefixed TLVs through Generator
This change allows adding a length prefix to a serialized TLV. It will
be particularly useful for serializing all 'scb_chan' entries in
the 'emergency.recover' file.

Key Changes:
 - Removed the need to loop in towire_tlv and fromwire_tlv, so the if conditions have been modified accordingly.
 - During serialization, the length of the TLV is calculated before appending it, and it is stored in a temporary variable.
 - For fromwire_tlv, only a simple length adjustment is required, and no loop is needed here either.
2025-02-22 11:51:54 -06:00
Aditya Sharma
39f63eb27f tools/gen: Enable Subtypes to have TLVStreams
This change will allow subtypes in wiregen files to have tlvstreams.
Shifting tlv structs above subtypes in header_template is done to prevent
forward declaration.

Since generate-wire prepends 'tlv_' in tlvname, we
have to modify fromwire_subtype_field and towire_subtype_field in
impl_template to accommodate this.

Changelog-Added: This PR would turn our peers into watchtower and enable SCB to create penalty txn.
2025-02-22 11:51:54 -06:00
ShahanaFarooqui
54555ce5e7 doc: Add new createrune example
- Moved the `Usage` section further down in `createrune` and `commando-rune` for improved UX.
- Added a new example for creating a rune with `read-only` restrictions, extending it to allow only payments of `less than 100,000 sats per day` using the `pay` or `xpay` methods.
- Adjusted formatting by appending an extra space after the `dependentUpon` condition, fixing `[*start* [*end*]][*relist*]` to `[*start* [*end*]] [*relist*]`.
- Relocated `Examples` from the expandable section to a standard heading, as examples are now already placed at the end of the page.

Changelog-None.
2025-02-04 20:12:59 +10:30
ShahanaFarooqui
7f73d312e8 ci: Fix for reproducible noble build failure
Changelog-None.
2025-01-30 09:34:01 +10:30
ShahanaFarooqui
b1fbbb0405 ci: Version update for Fedora reproducible build
Closes #7902.

Changelog-None.
2025-01-28 14:50:45 +10:30
ShahanaFarooqui
24e0929c0e tools: Add testnet4 support for reckless and hsmtool
Changelog-Added: Support for Bitcoin `testnet4`
2025-01-13 15:45:19 -08:00
Se7enZ
79273220f4 build: Add architecture to Ubuntu repro artifacts ([#6228]).
Changelog-Changed: Added architecture identifier to Ubuntu release
artifacts. ([#6228])
2024-12-03 11:56:23 +10:30
Se7enZ
f789f9cb21 build: Allow for signing releases without building a zip.
Adds a `--without-zip` parameter default `false` which optionally
skips the archive, to allow for signing releases from CI.
2024-12-03 11:56:23 +10:30
Se7enZ
b9b81c6918 build: Matrix strategy for release builds ([#7776]).
Also modifies the `build-release.sh` script to optionally
accept specific distribution targets for Ubuntu releases.
2024-12-03 11:56:23 +10:30
Se7enZ
c99b72da1c build: Release build automation ([#7776]).
Changelog-None
2024-12-03 11:56:23 +10:30
Rusty Russell
b5d1acee45 wire: call unknown types "UNKNOWN X" not "INVALID X".
It's freaking people out when they see things like:

```
 2024-11-11T05:26:41.281Z DEBUG ...53c-connectd: peer_out INVALID 22859
```

Fixes: https://github.com/ElementsProject/lightning/issues/7802
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: connectd: log unknown messages as "UNKNOWN" not "INVALID" to avoid freaking people out.
2024-12-02 15:09:45 +10:30
Matt Whitlock
67d667db90 tools/headerversions.c: fix build without SQLite
Commit 531845971c broke the build without
SQLite because this code:

	new = tal_fmt(NULL, template,
	              IF_SQLITE3(sqlite3_libversion_number()));

preprocesses into:

	new = tal_fmt(NULL, template,
	              );

which has a syntax error. Fix it by moving the comma into the macro
argument.

Fixes: 531845971c
Changelog-None
2024-11-19 09:48:31 +10:30
Rusty Russell
38d229df3d tools/generate-wire.py: make sure TLV array fields are allocated off TLV.
Otherwise the whole thing cannot be tal_steal() onto a different parent.

Here's the difference in generated files:

   --- ./wire/onion_wiregen.c.pre	2024-10-23 12:26:09.023176933 +1030
   +++ ./wire/onion_wiregen.c	2024-10-23 12:26:52.434828303 +1030
   @@ -128,7 +128,7 @@
     	blinded_path->path = num_hops ? tal_arr(blinded_path, struct blinded_path_hop *, 0) : NULL;
    	for (size_t i = 0; i < num_hops; i++) {
    		struct blinded_path_hop * tmp;
   -		tmp = fromwire_blinded_path_hop(blinded_path, cursor, plen);
   +		tmp = fromwire_blinded_path_hop(blinded_path->path, cursor, plen);
    		tal_arr_expand(&blinded_path->path, tmp);
    	}
    
   --- ./wire/bolt12_wiregen.c.pre	2024-10-23 12:26:09.079176474 +1030
   +++ ./wire/bolt12_wiregen.c	2024-10-23 12:26:52.612826902 +1030
   @@ -316,7 +316,7 @@
    	r->offer_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->offer_paths, cursor, plen);
    		tal_arr_expand(&r->offer_paths, tmp);
    	}
    }
   @@ -729,7 +729,7 @@
    	r->offer_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->offer_paths, cursor, plen);
    		tal_arr_expand(&r->offer_paths, tmp);
    	}
    }
   @@ -1052,7 +1052,7 @@
    	r->invreq_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->invreq_paths, cursor, plen);
    		tal_arr_expand(&r->invreq_paths, tmp);
    	}
    }
   @@ -1385,7 +1385,7 @@
    	r->offer_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->offer_paths, cursor, plen);
    		tal_arr_expand(&r->offer_paths, tmp);
    	}
    }
   @@ -1708,7 +1708,7 @@
    	r->invreq_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->invreq_paths, cursor, plen);
    		tal_arr_expand(&r->invreq_paths, tmp);
    	}
    }
   @@ -1781,7 +1781,7 @@
    	r->invoice_paths = *plen ? tal_arr(r, struct blinded_path *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_path * tmp;
   -		tmp = fromwire_blinded_path(r, cursor, plen);
   +		tmp = fromwire_blinded_path(r->invoice_paths, cursor, plen);
    		tal_arr_expand(&r->invoice_paths, tmp);
    	}
    }
   @@ -1808,7 +1808,7 @@
    	r->invoice_blindedpay = *plen ? tal_arr(r, struct blinded_payinfo *, 0) : NULL;
    	while (*plen != 0) {
    		struct blinded_payinfo * tmp;
   -		tmp = fromwire_blinded_payinfo(r, cursor, plen);
   +		tmp = fromwire_blinded_payinfo(r->invoice_blindedpay, cursor, plen);
    		tal_arr_expand(&r->invoice_blindedpay, tmp);
    	}
    }
   @@ -1927,7 +1927,7 @@
    	r->invoice_fallbacks = *plen ? tal_arr(r, struct fallback_address *, 0) : NULL;
    	while (*plen != 0) {
    		struct fallback_address * tmp;
   -		tmp = fromwire_fallback_address(r, cursor, plen);
   +		tmp = fromwire_fallback_address(r->invoice_fallbacks, cursor, plen);
    		tal_arr_expand(&r->invoice_fallbacks, tmp);
    	}
    }
   
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
9af01b062c pytest: fix hsmtool which reports leak under address sanitizer.
Couldn't figure out why hsmtool.proc.wait(WAIT_TIMEOUT) returns 1?
hsmtool doesn't ever seem to exit status 1!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
niftynei
e66653fa1d hsmtool: fixup show usage parens + return 2024-11-15 12:48:44 +10:30
Rusty Russell
b7d77b49df hsmtool.c: extend generatehsm to allow command-line args.
Based on the patch by bstin <barry.github@capsmx.com>, which added a separate command,
this simply extends "generatehsm" to allow more options.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: hsmtool: generatehsm can run non-interactive, taking options on the cmdline.
2024-11-15 12:48:44 +10:30
Rusty Russell
ab6f405ae7 common/hsm_encryption: use const char * for errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-13 08:49:55 +10:30
Rusty Russell
dca3a8c72c hsmtool: fix NULL secp256k1_ctx when calling dumpcommitments
secp256k1_ctx is used by pubkey_from_node_id.  Don't try to pick and
choose where to initialize secp256k1_ctx, just always do it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-06 21:17:36 +10:30