Add TLV field to hsmd_init_reply_v4 to communicate the HSM secret type
(mnemonic vs legacy) from HSM to lightningd. This allows lightningd to
automatically determine whether to use BIP86 or BIP32 derivation without
needing separate address types.
This commit is updating hsmtool and exposesecrets to use the new pattern for storing the secret, which is the secret_data and secret_len, to support both 64 byte and 32 byte seeds.
In the case where we receive a taproot utxo we want to be able to tell if it was derived using a BIP32 seed or a BIP86 seed. Considering we will only be supporting BI86 type wallet addresses for mnemonics we can check if the out secret is 64 bytes long and if it is we can use our BIP86 for the withdrawal.
RIP to this commit there's a good chance a lot of this code doesn't even make this into the final PR. Pour one out for the fallen lines of code.
This commit is doing the rest of the derivation. There was a significant overlap between the bip32_pubkey derivation and the bip86_pubkey derivation so that has been refactored in one place.
BIP86 wants the full 64-byte BIP32 seed (from BIP39). This wires up BIP86
support so the HSM derives the hardened base m/86'/0'/0' inside the box,
and exposes helpers:
• derive_bip86_base_key() // m/86'/0'/0'
• bip86_key(index) // m/86'/0'/0'/0/index
Spoiler: derive_bip86_base_key() and bip86_key() now live in libhsmd.c as they will later be used to check the derived wallet address against hsmd's derivation, this is just to sanity check that we haven't had an accidental bit flip while we have generated this address.
Here's some *foreshadowing* for what's to come. Here's what we're aiming for with our derivation flow:
Derivation split (hardened vs unhardened)
========================================
┌───────────────┐
│ HSM │ (secrets live here)
│ │
│ BIP39 → seed (64B)
│ ↓
│ m/86'/0'/0' ← derive hardened base (private)
│ ↓ (neuter)
│ BIP86 base xpub ← public-only + chain code
│ ↓
│ [send once over wire]
└───────────────┘
│
▼
┌───────────────────────┐
│ lightningd / wallet │
│ │
│ local (unhardened) derivations:
│ /0/i → external
│ /1/i → change
│ │
│ P2TR(BIP86) from pubkey_i
│ (optionally: CHECK with HSM)
└───────────────────────┘
We want to do part of the derivation inside hsmd and then send this base "pubkey" over the wire so our wallet can do the remaining derivation based on the address type and index. This lays the foundation for the base key wire message.
hsmd: plumb length-aware secret into hsmd_init; keep 32B mirror
BIP86 (from BIP39) wants the full 64-byte BIP32 seed. This commit plumbs a variable-length (32/64B) secret into hsmd and uses the accessors from the previous commit. We keep the old 32B hsm_secret mirror and, for now, only use the first 32 bytes so legacy paths keep working.
Spoiler: HKDFs will keep using the 32B seed; only wallet address derivation
will switch to the full 64B in a follow-up.
Update the exposesecret plugin to work with the new unified HSM secret
format that supports BIP39 mnemonics.
Changelog-Added - exposesecret now has a mnemonic field
Changelog-Changed: hsmd: New nodes will now be created with a BIP-39 12-word phrase as their root secret.
Changelog-Deprecated: config: `encrypted-hsm` to require a passphrase (use `hsm-passphrase`).
Changelog-Added: config: `hsm-passphrase` indicates we should use a manual passphrase with the hsm secret.
Basically, `devtools/reduce-includes.sh */*.c`.
Build time from make clean (RUST=0) (includes building external libs):
Before:
real 0m38.944000-40.416000(40.1131+/-0.4)s
user 3m6.790000-17.159000(15.0571+/-2.8)s
sys 0m35.304000-37.336000(36.8942+/-0.57)s
After:
real 0m37.872000-39.974000(39.5466+/-0.59)s
user 3m1.211000-14.968000(12.4556+/-3.9)s
sys 0m35.008000-36.830000(36.4143+/-0.5)s
Build time after touch config.vars (RUST=0):
Before:
real 0m19.831000-21.862000(21.5528+/-0.58)s
user 2m15.361000-30.731000(28.4798+/-4.4)s
sys 0m21.056000-22.339000(22.0346+/-0.35)s
After:
real 0m18.384000-21.307000(20.8605+/-0.92)s
user 2m5.585000-26.843000(23.6017+/-6.7)s
sys 0m19.650000-22.003000(21.4943+/-0.69)s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Each header should only include the other headers it needs to compile;
`devtools/reduce-includes.sh */*.h` does this. The C files then need
additional includes if they don't compile.
And remove the entirely useless wire/onion_wire.h, which only serves to include wire/onion_wiregen.h.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means we don't have to manually choose what to link against,
which is much of the complexity of our Makefiles: the compiler will
automatically use any object files it needs to link.
We already do this for ccan as libccan.a, now we have libcommon.a.
We don't link against it for *everything*, as some tests require their own
versions.
Notes:
1. I get rid of the weird plugins/test/Makefile2 (accidental commit?)
2. Many tests change due to update-mocks.
3. In some places I added the missing dependency on the Makefile itself, though most are in the next
patch.
Before:
Total program size: 221366528
Total tests size: 364243856
After:
Total program size: 190733656
Total tests size: 337880888
Build time from make clean (RUST=0) (includes building external libs):
Before:
real 0m38.227000-44.245000(41.8222+/-1.6)s
user 3m2.105000-33.696000(23.1442+/-8.4)s
sys 0m35.054000-42.269000(39.7231+/-2)s
After:
real 0m38.944000-40.416000(40.1131+/-0.4)s
user 3m6.790000-17.159000(15.0571+/-2.8)s
sys 0m35.304000-37.336000(36.8942+/-0.57)s
Build time after touch config.vars (RUST=0):
Before:
real 0m18.928000-22.776000(21.5084+/-1.1)s
user 2m8.613000-36.567000(27.7281+/-7.7)s
sys 0m20.458000-23.436000(22.3963+/-0.77)s
After:
real 0m19.831000-21.862000(21.5528+/-0.58)s
user 2m15.361000-30.731000(28.4798+/-4.4)s
sys 0m21.056000-22.339000(22.0346+/-0.35)s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rusty@rusty-Framework:~/devel/cvs/lightni
By policy, our stub hsmd accepts everything: openingd is supposed to
sort this out (or use VLS for a real HSM implementation!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Config: the node no longer crashes if you set `watchtime-blocks` to 0 (which is fine for testing: don't do this on mainnet!).
Add preapprove_check capabilities:
WIRE_HSMD_PREAPPROVE_INCOICE_CHECK and
WIRE_HSMD_PREAPPROVE_KEYSEND_CHECK to the capabilities array
if dev_no_preapprove_check is not set.
Do not assume those occupy the last two slots in the array.
Changelog-None
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Changelog-Added: HSMD: add new wire API to sign messages with bitcoin wallet keys according to BIP137.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
One in 256 times, we will grind a signature to 70 bytes (or shorter). This breaks
our feerate tests. Unfortunately the grinding is deterministic, so there doesn't
seem to be a way to avoid it. So we add a log message, and then we skip the
feerate test if it happens.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The renaming makes it clear that it's HSM specific.
And it has no pointers, so we can have an array instead of an array of pointers.
I tested this hadn't accidentally changed the wire format by disabling
version checks and using an old hsmd with the altered daemons and
running the test suite.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is such a simple struct that we can actually define it in csv.
This prevents us from accidentally breaking the ABI in future.
I tested this hadn't accidentally changed the wire format by disabling
version checks and using an old hsmd with the altered daemons and
running the test suite.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I'm about to update our utxo type, but Christian spotted that this is
part of the ABI for the hsm. So make that a private "hsm_utxo" type,
to insulate it from changes.
In particular, the HSM versions only contain the fields that the
hsm cares about, and the wire format is consistent (even though that
*did* include some of those fields, they are now dummies).
In the long term, this should be removed from the ABI: once we
no longer have "close_info" utxos, this information should already be
in the PSBT.
I tested this hadn't accidentally changed the wire format by disabling
version checks and using an old hsmd with the altered daemons and
running the test suite.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Enabling the cache makes signing significantly faster for segwit inputs,
particularly taproot which was designed with caching in mind.
Changelog-None
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
It's an internal difference, so doesn't actually break compatibility
(it would if we tried to prove we owned an old invoicerequest, but we
don't have infrastructure for that anyway).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The current interface, if given a tweak, uses a *different secret key*
and tweaks it. This was an early experiment: we will switch to using
a secret tweak for invoice_requests like we do for invoice path ids.
To make sure there's no funny business, *hsmd* hashes to form the
tweak (i.e. no zero tweaks!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We still support *existing* channels. Just not new ones (before they could,
in theory, explicitly ask for one).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: hsmd: the hsmd now supports HSM_VERSION 6
This is actually optional, everything would be ok leaving native hsmd
support at HSM_VERSION 5 instead.
We wire through --dev options into the hsmd, and test preapprove accept and deby
with both old and new protocols.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Apparently VLS actually does something when we preapprove: if caller is just
checking we want to tell it not to do that!
I put in a flag so we can test both old and new APIs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This should make VLS's life easier: they can ignore dev flags they
don't understand, but we will know their capabilites after init and so
know what they didn't understand (if required).
The only flag for now is a flag to force failure for "preapprove" calls.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's a u64, we should pass by copy. This is a big sweeping change,
but mainly mechanical (change one, compile, fix breakage, repeat).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This has the benefit of being shorter, as well as more reliable (you
will get a link error if we can't print it, not a runtime one!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Standardizes the is_xxx script function all take a script length, and changes
their first-level callers to pass it. This has several knock on benefits:
- We remove the repeated tal_count/tal_bytelen calls on the script, in
particular the redundant calls that result when we must check for multiple
types of script - which is almost all cases.
- We remove the dependency on the memory being tal-allocated (It is, in
all cases, but theres no reason we need to require that).
- We remove all cases where we create a copy of the script just to id it.
- We remove all allocations for non-interesting scripts while iterating block
txs in process_getfilteredblock_step1().
- We remove all allocations *including for potentially interesting scripts* in
topo_add_utxos().
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
Increasing the min version of the hsmd due that we
added new code that required the hsmd to sign an announcements.
One of the solution is to increase the min version in this way
a signer like VLS fails directly during the init phase.
Link: https://github.com/ElementsProject/lightning/issues/7074
Changelog-None: hsmd: increase the min version
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Before this it was channeld doing it, which was tied to a particular
channel. Create an API for lightningd to sign for any channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: hsmd: Added hsmd_forget_channel to enable explicit channel deletion. ([#6987])
Motivation: Previously, a signer prematurely forgetting a channel led
to failures in unresolved channel requests. This update introduces
hsmd_forget_channel, allowing nodes to explicitly notify signers when
a channel is irrevocably resolved and can be safely forgotten. This
ensures synchronized channel cleanup between nodes and signers.
This change maintains backward and forward compatibility. Nodes
explicitly check whether a signer has `WIRE_HSMD_FORGET_CHANNEL`
capability before sending the message. Nodes without
`WIRE_HSMD_FORGET_CHANNEL` capability won't send this message. Signers
capable of handling this message but not receiving it will continue to
use conservative pruning methods.
Fixes#6987
In general, a validating signer may be under a different operational
environment than the node, and therefore may have a different
source of on-chain data. The signer may therefore temporarily disagree
on whether a funding or splice transaction is locked (buried).
We would like to ensure agreement between the signer and the
node on how to progress a channel's state.
The following message are added to provide a solution:
- `check_outpoint(outpoint) -> bool` - check if the signer agrees that a funding candidate outpoint is buried
- `lock_outpoint(outpoint)` - change the funding/splice state to locked
Link: https://github.com/ElementsProject/lightning/issues/6722
Suggested-by: @devrandom
Co-Developed-by: Ken Sedgwick <ken@bonsai.com>
Changelog-Added: hsmd protocol: Added hsmd_check_outpoint and hsmd_lock_outpoint
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
The assertion may not make sense if we change the
meaning of `push_value` to be relative, especially since
negative values appear as large positive.
Suggested-by: Ken Sedgwick <ken@bonsai.com>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Originally VLS used hsmd_ready_channel as an early call during channel
setup, but later the BOLT-2 spec changed the name of funding_locked to channel_ready.
This is very confusing because the hsmd_ready_channel is not directly
related to the new channel_ready.
This commit is renaming the hsmd_ready_channel to hsmd_setup_channel.
Link: https://github.com/ElementsProject/lightning/issues/6717
Suggested-by: Ken Sedgwick
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
This makes it easier to use outside simple subds, and now lightningd can
simply dump to log rather than returning JSON.
JSON formatting was a lot of work, and we only did it for lightningd, not for
subdaemons. Easier to use the logs in all cases.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The nomenclature confusion mean that we were ANDING a capability
with a message number (29) which always returned non-zero. We really
do need a new capability which we can hand to channeld to make these
splice txs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I obviously like the word "capabilities" since I reused it to refer
to the HSM's overall features :(
Suggested-by: @ksedgwic
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
