They are invalid! This is because our BOLT11_FIELD_BYTE_LIMIT is not the limit,
it's one greater than the limit.
Reported-by: https://github.com/noblepayne
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `invoice` no longer accepts 640-byte descriptions (it would produce malformed invoices).
This is important: if it's tor-only and we don't have a proxy, we will fail
to connect, but it's no indication that the node is unreachable. Same with
IPv6.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In some cases it is helpful to know who offered us the HTLC.
Changelog-Changed: Plugins: The `htlc_accepted` hook now knows the
`peer_id` of the peer that offered us the HTLC.
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
This commit introduces a new field `invoice_msat` to the htlc_accepted
hook. If this field is specified it will replace the amount of the
invoice that belongs to the payment_hash of the HTLC on internal checks.
This is useful in scenarios where we actually expect a smaller amount
than initially specified in an invoice.
Changelog-Changed: Plugins: `htlc_accepted` hook can now override the
expected total amount of the invoice that belongs to the HTLC.
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
Adds `expected_msat_override` to the `invoice_check_payment` check. If
it's set, it will be used to override the invoice amount as the expected
amount of the payment check.
This enables us to charge a different amount for a payment than the
amount stated on the invoice.
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
If we failed after we register (e.g. channeld not available), we don't
mark it failed. We shouldn't register until we've definitely created
the htlc.
Changelog-Fixed: `xpay` would sometimes leave payment parts status `pending` in failure cases (as seen in listpays or listsendpays).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/8629
Instead of having a separate field to derive the bip86 base key, we return it in the hsmd init reply once we know that the hsm_secret is of mnemonic type
Simplify wallet address generation by using a unified approach where
the derivation method (BIP86 vs BIP32) is determined by the wallet's
HSM secret type rather than having separate address types.
Add TLV field to hsmd_init_reply_v4 to communicate the HSM secret type
(mnemonic vs legacy) from HSM to lightningd. This allows lightningd to
automatically determine whether to use BIP86 or BIP32 derivation without
needing separate address types.
This commit fixes an issue where BIP86 addresses were not being
discovered during wallet recovery/rescan operations.
The root cause was that init_txfilter() only populated the transaction
filter with BIP32-derived keys, preventing lightningd from recognizing
BIP86 UTXOs during blockchain scans. Now both BIP32 and BIP86 derived
scripts are included in the filter when BIP86 derivation is enabled.
This ensures that wallets restored from BIP39 mnemonics can properly
discover and display previously funded BIP86 addresses without requiring
manual address generation first.
[ We also move the slightly-lost comment about libbacktrace so it is
where we actually include <backtrace.h> --RR ]
RIP to this commit there's a good chance a lot of this code doesn't even make this into the final PR. Pour one out for the fallen lines of code.
This commit is doing the rest of the derivation. There was a significant overlap between the bip32_pubkey derivation and the bip86_pubkey derivation so that has been refactored in one place.
Changelog-Changed: hsmd: New nodes will now be created with a BIP-39 12-word phrase as their root secret.
Changelog-Deprecated: config: `encrypted-hsm` to require a passphrase (use `hsm-passphrase`).
Changelog-Added: config: `hsm-passphrase` indicates we should use a manual passphrase with the hsm secret.
Incorporate a time: this covers the restart case as well. And make it time_mono(),
which doesn't get overridden when we override normal wall time.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
You can now simply add per-tal-object helpers for memleak, but our older pattern required
calling memleak functions explicitly during memleak handling. Hash tables in particular need
to be dynamically allocated (we override the allocators using htable_set_allocator and assume
this), so it makes sense to have a helper macro that does all three.
This eliminates a huge amount of code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We watch if they are to do with a channel, or have outputs going to us, but otherwise
we didn't, so we never updated the blockheight in the db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `listtransactions` now correctly updates `blockheight` for txs created by `sendpsbt` which have no change outputs.
I got a NULL deref on `infcopy->remote_funding = *inflight->funding->splice_remote_funding`
at once point in testing, so this should prevent that from happening,
yet still allow us to catch it in CI if it happens again.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This happens if the channel is *not* announcable yet. Then we hit the assertion
in funding_depth_cb that the txid is the same as the current funding.txid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: fixed crash when we splice a channel which hasn't been announced yet.
Basically, `devtools/reduce-includes.sh */*.c`.
Build time from make clean (RUST=0) (includes building external libs):
Before:
real 0m38.944000-40.416000(40.1131+/-0.4)s
user 3m6.790000-17.159000(15.0571+/-2.8)s
sys 0m35.304000-37.336000(36.8942+/-0.57)s
After:
real 0m37.872000-39.974000(39.5466+/-0.59)s
user 3m1.211000-14.968000(12.4556+/-3.9)s
sys 0m35.008000-36.830000(36.4143+/-0.5)s
Build time after touch config.vars (RUST=0):
Before:
real 0m19.831000-21.862000(21.5528+/-0.58)s
user 2m15.361000-30.731000(28.4798+/-4.4)s
sys 0m21.056000-22.339000(22.0346+/-0.35)s
After:
real 0m18.384000-21.307000(20.8605+/-0.92)s
user 2m5.585000-26.843000(23.6017+/-6.7)s
sys 0m19.650000-22.003000(21.4943+/-0.69)s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Each header should only include the other headers it needs to compile;
`devtools/reduce-includes.sh */*.h` does this. The C files then need
additional includes if they don't compile.
And remove the entirely useless wire/onion_wire.h, which only serves to include wire/onion_wiregen.h.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means we don't have to manually choose what to link against,
which is much of the complexity of our Makefiles: the compiler will
automatically use any object files it needs to link.
We already do this for ccan as libccan.a, now we have libcommon.a.
We don't link against it for *everything*, as some tests require their own
versions.
Notes:
1. I get rid of the weird plugins/test/Makefile2 (accidental commit?)
2. Many tests change due to update-mocks.
3. In some places I added the missing dependency on the Makefile itself, though most are in the next
patch.
Before:
Total program size: 221366528
Total tests size: 364243856
After:
Total program size: 190733656
Total tests size: 337880888
Build time from make clean (RUST=0) (includes building external libs):
Before:
real 0m38.227000-44.245000(41.8222+/-1.6)s
user 3m2.105000-33.696000(23.1442+/-8.4)s
sys 0m35.054000-42.269000(39.7231+/-2)s
After:
real 0m38.944000-40.416000(40.1131+/-0.4)s
user 3m6.790000-17.159000(15.0571+/-2.8)s
sys 0m35.304000-37.336000(36.8942+/-0.57)s
Build time after touch config.vars (RUST=0):
Before:
real 0m18.928000-22.776000(21.5084+/-1.1)s
user 2m8.613000-36.567000(27.7281+/-7.7)s
sys 0m20.458000-23.436000(22.3963+/-0.77)s
After:
real 0m19.831000-21.862000(21.5528+/-0.58)s
user 2m15.361000-30.731000(28.4798+/-4.4)s
sys 0m21.056000-22.339000(22.0346+/-0.35)s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rusty@rusty-Framework:~/devel/cvs/lightni
This was changing all the time when I tried to make
autogenerate-rpc-examples.py reproducible. Turns out it was being
corrupted (it does suspicious things with pointers); rather than try
to diagnose it, I simply rewrote the code to create it only when we
need it.
```
Valgrind error file: valgrind-errors.34506
==34506== Uninitialised byte(s) found during client check request
==34506== at 0x241732: memcheck_ (mem.h:247)
==34506== by 0x2417BC: towire (towire.c:17)
==34506== by 0x24185C: towire_u16 (towire.c:28)
==34506== by 0x20C8E4: towire_tlv_scb_tlvs_remote_to_self_delay (scb_wiregen.c:213)
==34506== by 0x240E78: towire_tlv (tlvstream.c:342)
==34506== by 0x20C99F: towire_tlv_scb_tlvs (scb_wiregen.c:234)
==34506== by 0x20C298: towire_modern_scb_chan (scb_wiregen.c:89)
==34506== by 0x1A6CF3: json_add_scb (peer_control.c:2488)
==34506== by 0x1A6E0C: json_staticbackup (peer_control.c:2519)
==34506== by 0x177E3F: command_exec (jsonrpc.c:799)
==34506== by 0x1785AE: rpc_command_hook_final (jsonrpc.c:945)
==34506== by 0x1BEC2D: plugin_hook_call_next (plugin_hook.c:199)
==34506== Address 0x1ffeffe736 is on thread 1's stack
==34506== in frame #2, created by towire_u16 (towire.c:26)
==34506==
{
<insert_a_suppression_name_here>
Memcheck:User
fun:memcheck_
fun:towire
fun:towire_u16
fun:towire_tlv_scb_tlvs_remote_to_self_delay
fun:towire_tlv
fun:towire_tlv_scb_tlvs
fun:towire_modern_scb_chan
fun:json_add_scb
fun:json_staticbackup
fun:command_exec
fun:rpc_command_hook_final
fun:plugin_hook_call_next
}
==34506== Uninitialised byte(s) found during client check request
==34506== at 0x241732: memcheck_ (mem.h:247)
==34506== by 0x2417BC: towire (towire.c:17)
==34506== by 0x240EF0: towire_tlv (tlvstream.c:354)
==34506== by 0x20C99F: towire_tlv_scb_tlvs (scb_wiregen.c:234)
==34506== by 0x20C298: towire_modern_scb_chan (scb_wiregen.c:89)
==34506== by 0x1A6CF3: json_add_scb (peer_control.c:2488)
==34506== by 0x1A6E0C: json_staticbackup (peer_control.c:2519)
==34506== by 0x177E3F: command_exec (jsonrpc.c:799)
==34506== by 0x1785AE: rpc_command_hook_final (jsonrpc.c:945)
==34506== by 0x1BEC2D: plugin_hook_call_next (plugin_hook.c:199)
==34506== by 0x1BEBA8: plugin_hook_callback (plugin_hook.c:186)
==34506== by 0x1B771E: plugin_response_handle (plugin.c:705)
==34506== Address 0x7bd1d08 is 40 bytes inside a block of size 42 alloc'd
==34506== at 0x484DCD3: realloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==34506== by 0x3C8614: tal_resize_ (tal.c:755)
==34506== by 0x2417A2: towire (towire.c:14)
==34506== by 0x24185C: towire_u16 (towire.c:28)
==34506== by 0x20C8E4: towire_tlv_scb_tlvs_remote_to_self_delay (scb_wiregen.c:213)
==34506== by 0x240E78: towire_tlv (tlvstream.c:342)
==34506== by 0x20C99F: towire_tlv_scb_tlvs (scb_wiregen.c:234)
==34506== by 0x20C298: towire_modern_scb_chan (scb_wiregen.c:89)
==34506== by 0x1A6CF3: json_add_scb (peer_control.c:2488)
==34506== by 0x1A6E0C: json_staticbackup (peer_control.c:2519)
==34506== by 0x177E3F: command_exec (jsonrpc.c:799)
==34506== by 0x1785AE: rpc_command_hook_final (jsonrpc.c:945)
==34506==
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Using `tal_dup_or_null` does not set the tal context to the included
bytes array. Luckily we already have a function tha deeply duplicates
a `feature_set`: `feature_set_dup`
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
We need to remove the feature bits set via a plugins get_manifest
response when the init response disables the plugin.
Changelog-Fixed Remove feature bits set by a plugin when the plugin
disables itself during init.
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
For old channels, this can take a while, and it stops everything. But
we are only doing this to save space; it's not a *functional* necessity.
A quick and dirty test with 50,000 htlcs shows the htlc deletion took
450msec. I tried adding an index, and changing it to set hstate to
HTLC_STATE_INVALID instead of deleting entries, but it still took about 350ms.
Whereas the "COUNT(*)" only took 1.7msec, so it's worth keeping.
Reported-by: @michael1011
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: lightningd: we defer deletion of old htlcs on channel close, to avoid pausing for a long time (we clean them on startup)
Fixes: https://github.com/ElementsProject/lightning/issues/7962
Deprecated 24.11, disabled 25.05 (they're the default now).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Config: --experimental-offers and --experimental-quiesce (default since v24.11)
Changelog-Removed: Plugins: `onion_message_recv` hook `blinding` field (use `first_path_key` as per modern BOLT 4 naming).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changes:
* Fields renumbered to their draft values + billion.
* offer_recurrence now comes in compulsory or optional (backwards compat) flavors.
* `proportional_amount` is now inside `offer_recurrence_base` not `offer_recurrence_paywindow`.
* New field `invreq_recurrence_cancel`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Draft specification for recurring offers changed: old recurring offers will no longer work.
Offer_absolute_expiry should be used if you want to require starting at the start.
Changelog-EXPERIMENTAL: Protocol: BOLT 12 recurrence `start_any_period` removed, use expiry if you need to restrict when they can start using the offer.
This can happen if we haven't transitioned to channeld yet, but logic is simply to hand
it to lightningd, exactly as channeld does.
```
2025-09-30T03:04:57.8951627Z lightningd-1 2025-09-30T02:59:14.150Z DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-dualopend-chan#1: peer_out WIRE_WARNING
2025-09-30T03:04:57.8952126Z lightningd-1 2025-09-30T02:59:14.150Z **BROKEN** 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-dualopend-chan#1: Unexpected message WIRE_ANNOUNCEMENT_SIGNATURES
2025-09-30T03:04:57.8952521Z lightningd-1 2025-09-30T02:59:14.150Z INFO 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-dualopend-chan#1: Peer connection lost
2025-09-30T03:04:57.8953124Z lightningd-1 2025-09-30T02:59:14.150Z INFO 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: Peer transient failure in DUALOPEND_AWAITING_LOCKIN: dualopend: Owning subdaemon dualopend died (62208)
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
One issue we have in CI is reconnection races: if an incoming
connection arrives while an outgoing one is negotiated, we close the
outgoing one and issue a disconnect, which fails any connect attempts.
By sending a "reconnected" message instead of disconnect/connect we
can avoid disturbing in-progress connection attempts which happens in CI
quite a bit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
