Every distribution we have packages this now.
Changelog-Changed: Build: lowdown is now required (we no longer bundle our own).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Instead of having a separate field to derive the bip86 base key, we return it in the hsmd init reply once we know that the hsm_secret is of mnemonic type
This schema change updates newaddr to remove bip86 which was previously added, since don't want to make unnecessary schema changes this is being removed.
The generated files for the exposesecret schema change are also being added
Simplify wallet address generation by using a unified approach where
the derivation method (BIP86 vs BIP32) is determined by the wallet's
HSM secret type rather than having separate address types.
Add TLV field to hsmd_init_reply_v4 to communicate the HSM secret type
(mnemonic vs legacy) from HSM to lightningd. This allows lightningd to
automatically determine whether to use BIP86 or BIP32 derivation without
needing separate address types.
This simplifies the UTXO type system by removing the separate BIP86
enum value. P2TR addresses will now use unified derivation logic
based on the wallet's HSM secret type rather than having separate
enum values."
This commit fixes an issue where BIP86 addresses were not being
discovered during wallet recovery/rescan operations.
The root cause was that init_txfilter() only populated the transaction
filter with BIP32-derived keys, preventing lightningd from recognizing
BIP86 UTXOs during blockchain scans. Now both BIP32 and BIP86 derived
scripts are included in the filter when BIP86 derivation is enabled.
This ensures that wallets restored from BIP39 mnemonics can properly
discover and display previously funded BIP86 addresses without requiring
manual address generation first.
[ We also move the slightly-lost comment about libbacktrace so it is
where we actually include <backtrace.h> --RR ]
This commit is updating hsmtool and exposesecrets to use the new pattern for storing the secret, which is the secret_data and secret_len, to support both 64 byte and 32 byte seeds.
In the case where we receive a taproot utxo we want to be able to tell if it was derived using a BIP32 seed or a BIP86 seed. Considering we will only be supporting BI86 type wallet addresses for mnemonics we can check if the out secret is 64 bytes long and if it is we can use our BIP86 for the withdrawal.
Add the UTXO_P2TR_BIP86 in preparation to add BIP86 wallet functions such as newaddr, listaddr etc. We also add a new index in the database for BIP86 as this is using a completely different derivation path and hsm_secret.
RIP to this commit there's a good chance a lot of this code doesn't even make this into the final PR. Pour one out for the fallen lines of code.
This commit is doing the rest of the derivation. There was a significant overlap between the bip32_pubkey derivation and the bip86_pubkey derivation so that has been refactored in one place.
BIP86 wants the full 64-byte BIP32 seed (from BIP39). This wires up BIP86
support so the HSM derives the hardened base m/86'/0'/0' inside the box,
and exposes helpers:
• derive_bip86_base_key() // m/86'/0'/0'
• bip86_key(index) // m/86'/0'/0'/0/index
Spoiler: derive_bip86_base_key() and bip86_key() now live in libhsmd.c as they will later be used to check the derived wallet address against hsmd's derivation, this is just to sanity check that we haven't had an accidental bit flip while we have generated this address.
Here's some *foreshadowing* for what's to come. Here's what we're aiming for with our derivation flow:
Derivation split (hardened vs unhardened)
========================================
┌───────────────┐
│ HSM │ (secrets live here)
│ │
│ BIP39 → seed (64B)
│ ↓
│ m/86'/0'/0' ← derive hardened base (private)
│ ↓ (neuter)
│ BIP86 base xpub ← public-only + chain code
│ ↓
│ [send once over wire]
└───────────────┘
│
▼
┌───────────────────────┐
│ lightningd / wallet │
│ │
│ local (unhardened) derivations:
│ /0/i → external
│ /1/i → change
│ │
│ P2TR(BIP86) from pubkey_i
│ (optionally: CHECK with HSM)
└───────────────────────┘
We want to do part of the derivation inside hsmd and then send this base "pubkey" over the wire so our wallet can do the remaining derivation based on the address type and index. This lays the foundation for the base key wire message.
hsmd: plumb length-aware secret into hsmd_init; keep 32B mirror
BIP86 (from BIP39) wants the full 64-byte BIP32 seed. This commit plumbs a variable-length (32/64B) secret into hsmd and uses the accessors from the previous commit. We keep the old 32B hsm_secret mirror and, for now, only use the first 32 bytes so legacy paths keep working.
Spoiler: HKDFs will keep using the 32B seed; only wallet address derivation
will switch to the full 64B in a follow-up.
BIP86 derivation requires the full 64-byte seed that comes from the BIP39 mnemonic. The first 32 bytes goes towards to master seed material and the nest 32 bytes go towards the chaincode, so we need the entire 64 bytes for deterministic derivations. I've kept the old secret struct in for now for backwards compatibility and also added some accessors which will eventually die in this branch's git multiverse but that's a spoiler, they're on the ride for the next few commits at least to help us migrate to this length aware API throughout the rest of the code without making a lot of breaking changes.
Update the exposesecret plugin to work with the new unified HSM secret
format that supports BIP39 mnemonics.
Changelog-Added - exposesecret now has a mnemonic field
In several places we were using tal_wally_end(tmpctx), which assumes
that libwally isn't using any of those allocations.
Make an explicit "tal_wally_discard" which asserts that there are no
outstanding libwally allocations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: hsmd: New nodes will now be created with a BIP-39 12-word phrase as their root secret.
Changelog-Deprecated: config: `encrypted-hsm` to require a passphrase (use `hsm-passphrase`).
Changelog-Added: config: `hsm-passphrase` indicates we should use a manual passphrase with the hsm secret.
Changelog-Added: `hsmtool` now supports hsm_secret files using a 12-word mnemonic.
Changelog-Removed: hsmtool support for mnemonics in non-english languages removed.
This is a newer API to replace hsm_encryption.c and hsm_encryption.c, this tidies up the API to be used and also cleans things up to support our new formats. Our hsm_secret formats now include:
- Legacy 32-byte plain format
- Legacy 73-byte encrypted format
- New mnemonic format without passphrase (32 zero bytes + mnemonic)
- New mnemonic format with passphrase (32-byte hash + mnemonic)
This commit includes support to detect the format based on the file size and content structure. The hsm will store mnemonics in the hsm_secret file as:
`passphraseHash`mnemonic`
Incorporate a time: this covers the restart case as well. And make it time_mono(),
which doesn't get overridden when we override normal wall time.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
You can now simply add per-tal-object helpers for memleak, but our older pattern required
calling memleak functions explicitly during memleak handling. Hash tables in particular need
to be dynamically allocated (we override the allocators using htable_set_allocator and assume
this), so it makes sense to have a helper macro that does all three.
This eliminates a huge amount of code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It now simply renames tal names, so it's harmless to do even if we're
not going to do memleak detection.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We watch if they are to do with a channel, or have outputs going to us, but otherwise
we didn't, so we never updated the blockheight in the db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `listtransactions` now correctly updates `blockheight` for txs created by `sendpsbt` which have no change outputs.
I got a NULL deref on `infcopy->remote_funding = *inflight->funding->splice_remote_funding`
at once point in testing, so this should prevent that from happening,
yet still allow us to catch it in CI if it happens again.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This happens if the channel is *not* announcable yet. Then we hit the assertion
in funding_depth_cb that the txid is the same as the current funding.txid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: fixed crash when we splice a channel which hasn't been announced yet.
