Commit Graph

4673 Commits

Author SHA1 Message Date
Lagrang3
7e5cf41b4e htlc_wire: fix crash when adding an HTLC
In line channeld/channeld_wiregen.c:832 `*added+i` is not a tal object hence
the instruction in common/htlc_wire.c:200 `tal_arr(ctx, struct tlv_field, 0);` crashes CLN.
This is fixed by stating that added_htlc is a a varsize_type.

Logs:

2025-08-16T02:25:28.640Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v25.05-200-g79b959b)V
...
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:95 (call_error) 0x54f6bc
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:169 (check_bounds) 0x54f75a
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:178 (to_tal_hdr) 0x54f782
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:193 (to_tal_hdr_or_null) 0x54f7c7
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:471 (tal_alloc_) 0x54ffe4
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:517 (tal_alloc_arr_) 0x5500c4
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: common/htlc_wire.c:200 (fromwire_len_and_tlvstream) 0x48d63d
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: common/htlc_wire.c:234 (fromwire_added_htlc) 0x48dd23
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: channeld/channeld_wiregen.c:832 (fromwire_channeld_got_commitsig) 0x4c61fa
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:2377 (peer_got_commitsig) 0x4549cb
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/channel_control.c:1552 (channel_msg) 0x4140fe
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/subd.c:560 (sd_msg_read) 0x461513
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:60 (next_plan) 0x544885
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:422 (do_plan) 0x544cea
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:439 (io_ready) 0x544d9d
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:455 (io_loop) 0x54665d
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x42d220
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1487 (main) 0x43280f

gdb inspection:
830             *added = num_added ? tal_arr(ctx, struct added_htlc, num_added) : NULL;
831             for (size_t i = 0; i < num_added; i++)
832                     fromwire_added_htlc(&cursor, &plen, *added + i);
(gdb) p i
$3 = 1

Changelog-None: crash introduced this release.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
[ Added test, removed Changelog --RR ]
2025-08-27 14:14:31 +09:30
Rusty Russell
52d69df10f lightningd: migrate events from bookkeeper at startup.
We take over the --bookkeeper-dir and --bookkeeper-db options, and
then if we can find the bookkeeper db we extract the records to
initialize our chain_moves and channel_moves tables.

Of course, bookkeeper now needs to not register those options.

When bookkeeper gets invoked the first time, it will reconstruct
everything from listchannelmoves and listcoinmoves.  It cannot
preserve manually-added descriptions, so we put those in the datastore
for it ready to go.

Note that the order of onchain_fee changes slightly from the original.
But this is fine.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
254b7ed666 lightningd: new internal-use commands to allow utxo spend / deposit injection.
For the moment, we'll continue to use bookkeeper to monitor the
notifications to insert these (we don't have the internal infrastructure
for that, and actually these commands are probably better than using
notifications).

We hoist param_outpoint() into common code, since there are already
two uses.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
6c626b124b lightningd: add chainmoves and channelmoves to wait command.
Only makes sense to wait on creation, since they neither are deleted
nor updated.

We also enhance the list commands to take the standard index options.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `wait`: new subsystems `chainmoves` and `channelmoves`.
2025-08-19 13:37:50 +09:30
Rusty Russell
7c2a74684e lightningd: add listchainmoves and listchannelmoves commands.
This is where all the previous work pays off: we can access the coinmoves
in the db.

Changelog-Added: JSON-RPC: `listchainmoves` and `listchannelmoves` commands to access the audit log of coin movements.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
233628ad48 lightningd: speed mapping from coinmoves table to channel for listcoinmoves.
Iterating through every peer and channel every time can be very slow
for large nodes, when calling wallet_coinmoves_extract for listcoinmoves.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
0c77cdd2ef lightningd: reorder JSON fields into more coherent order.
This is mainly for the coming list commands, but it's just more
logical to have the common fields at the top.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
f3f26cb560 wallet: record all coin movements into chain_moves or channel_moves tables.
We change notify_chain_mvt to wallet_save_chain_mvt, and
notify_channel_mvt to wallet_save_channel_mvt, which save to the db
and call the notifier themselves.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
9e7be804c2 lightningd: add dev option to save hooks and notifications to/from plugins.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-18 10:01:07 +09:30
Rusty Russell
16819f345d lightningd: make notifications from plugins just like native ones.
Rather than forcing them to wrap their parameters in a "payload"
sub-object, copy in params directly.  We include the "origin" field
one level up, if they care.

The next patch restores compatibility for the one place we currently use
them, which is the pay plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: pyln-client: plugin custom notifications origins and payload (use parameters directly)
2025-08-18 10:01:07 +09:30
Rusty Russell
a686bda4cb lightningd: deprecate null short_channel_id and unknown old_state in channel_state_changed notification
We always prefer to omit fields rather than use 'null' (or unknown!).

Note that before this, the schema was broken, so we have to put a special
exemption in for that case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-18 10:01:07 +09:30
Matt Whitlock
3dd0979721 lightningd: use json_escape_unescape_len for log message strings received from plugins
[ Reduced to just neatening, as 23997b2e1d did
  the unescape handling already -- RR ]
2025-08-15 15:23:57 +09:30
Matt Whitlock
c28f5e70a8 use json_escape_unescape_len()
This avoids making an extra copy of the escaped string.

Note that jsonrpc_command_add() no longer accepts usage strings
containing invalid escape sequences. (Previously, it would quietly
accept such a string without unescaping anything.)

Changelog-None
2025-08-15 15:23:57 +09:30
Rusty Russell
733bdfa67d lightningd: add channel_type to openchannel & openchannel2 hooks.
An obvious omission!

Changelog-Added: Plugins: `openchannel` and `openchannel2` hooks now expose the `channel_type` field for the offered channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-15 11:11:39 +09:30
Rusty Russell
22b452ad19 lightningd: have onchaind inform us when to make a channel penalty_adj.
bookkeeper used to generate these as channel events, now lightningd does.

We also add a "journal" event, which we will need later too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
b0231a59d8 common: put "external" and "wallet" strings, and test functions into common/coin_mvt.h
They're scattered and reproduced in many places: unify them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
841a8bd03a lightningd: extract core of coin_movement notification, for use in list functions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `coin_movement` notification `utxo_txid`, `vout` and `txid` fields (use `utxo` and `spending_txid`).
Changelog-Added: JSON-RPC: `coin_movement` notification `utxo` field.
Changelog-Added: JSON-RPC: `coin_movement` notification `spending_txid` field.
2025-08-14 19:49:09 +09:30
Rusty Russell
d066a5f301 common: make chain_coin_mvt's outpoint member a non-pointer.
It's always set, and in fact we assume it is (journal entries are not
internal to lightningd, so we won't see them in
lightningd/notification.c: that comment is misleading).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
c0221b1b74 coin_mvt: put timestamp into the struct.
This is not particularly relevant now (it's always the current time) but will be
useful when we implement the list commands.

Note that timestamp is set to be "u32" in various schemas.  This will
only become a problem on Sun 07 Feb 2106 06:28:15 UTC.  I apologize to
my grandchildren in advance.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
650f3882dd lightningd: separate coin_movement tags array into primary_tag and extra_tags.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `coin_movement` notification `tags` array (use `primary_tag` and `extra_tags`).
Changelog-Added: JSON-RPC: `coin_movement` notification `primary_tag` and `extra_tags`.
2025-08-14 19:49:09 +09:30
Rusty Russell
8c315c15b6 coin_mvt: use bitmap instead of tal_arr for tags.
We're going to store them in the db this way, so I thought I'd see what it looks like if
we lift that interface all the way through.

We use a struct, so that types are checked strictly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
1833cc2d28 common: mvt_tag_strs() function to turn tags array into strings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
5756b54f38 common: rename enum mvt_tag values.
Prefix MVT_ to them, for clarity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
6fc0b1f3d1 common/coin_mvt: add struct mvt_account_id to separate channels from others.
This means we can keep a pointer to the channel directly, *or* a string.
This avoids gratuitous formatting (on creation) and lookups (later).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
e372c2ec0b common/coin_mvt: use enum rather than true/false for credit/debit.
More readable for me.  Also, change order so we definitely break
compilation on all callers (putting enum before amount).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
bc5e54d153 common: coin movements for payments need group id as well as part id.
The part id is *only* unique within a group.  The payment_hash / partid / groupid tuple is unique.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `coin_movement` notification with `part_id` field now always has `group_id` field.
2025-08-14 19:49:09 +09:30
Rusty Russell
f4ac8fccb6 common/coin_mvt: make more parameters const.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
45678aafc5 lightningd: use channel_coin_mvt / chain_coin_mvt directly for notifications.
Rather than converting to a generic coin_mvt struct, use these directly in
the notification, which is more explicit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
63065aa99c common: handle taken() extra_tlvs in new_existing_htlc properly.
Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 18:57:05 +09:30
Peter Neuroth
2264b93c6c lightningd: Add extra_tlvs to htlc_accepted_hook
Add serializing and deserializing of the extra tlvs to to the
htlc_accepted_hook to allow plugin users to replace the tlv stream that
is attached to the update_add_htlc message on forwards.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Peter Neuroth
ef52de88aa channeld: Add extra_tlvs to incomming wire msg
This appends the extra_tlvs to the internal channeld_offer_htlc wire
msg. We also recombine the extra_tlvs with the blinded path key for
forwarding htlcs.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Rusty Russell
5f5440383d lightningd: fix race with crossover pings.
We cannot use subd_req() here: replies will come out of order, and the
we should not simply assign the reponses in FIFO order.

Changelog-Fixed: lightningd: don't get confused with parallel ping commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 17:35:39 +09:30
Rusty Russell
d1f56eb668 lightningd: use the hash table to lookup scids.
This replaces the old "iterate through each peer, then each peer's channel" suboptimality.

A bit of care required that we don't expose scids if we're forwarding,
but that was already carefully handled.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 14:04:11 +09:30
Rusty Russell
5e263bac78 lightningd: require local_alias in new_channel().
We allowed NULL for stub channels, but just don't put the stub scid
into the hash tables.  This cleans up all the callers to make it
clear this is a non-optional parameter.

We opencode channel_set_random_local_alias, since there's only one caller now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 14:04:11 +09:30
Rusty Russell
639452ac77 bitcoin: have random_scid() function.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 14:04:11 +09:30
Rusty Russell
ab80345754 lightningd: maintain a hash table of short_channel_id, for faster lookup.
This contains real scids, as well as aliases, and old scids.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 14:04:11 +09:30
Rusty Russell
d70fc13b54 lightningd: consider old scids when looking up channels (for routing).
Changelog-Fixed: Protocol: we now allow routing through old short-channel-ids once a splice is done (previously we would refuse, leading to a 6 block gap in service).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 14:04:11 +09:30
Rusty Russell
4abd0af3ec lightningd: save previous short_channel_ids during splice, and keep in db.
There can be any number of these, and it will be useful to allow
routing by older scids (when other nodes haven't seen our gossip, or
even before we *can* announce the new post-splice channel).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 14:04:11 +09:30
Dusty Daemon
a8ff8ce717 splice: Add field to DB to track if we’ve sent sigs
We used to use a check on the active psbt to see if our splice signature was in it — but now we need to generate the signature early.

So we have to add a field tracking if we’ve sent it and add it to the database, wire protocols, and inflight objects.
2025-08-14 04:12:17 +09:30
Rusty Russell
506fa914e0 lightningd: always tell openingd/dualopend what channel type we want.
Prior to it being compulsory, these daemons would need a default value.  Now it's
always required, it's clearer if it's always told.

There's no "default_channel_type" now everyone has to specify channel_type either,
so rename it to "desired_channel_type" and put it in lightningd specifically.

Note that the channel_type can have options added: either option_scid_alias or option_zeroconf.

This results in a slight behavior change: we will get type zeroconf even if we didn't ask for it, if they gave it to us.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: fundchannel / fundchannel_start returned `channel_type` will include option_zeroconf if it was implied by a 0 minimum_depth, even if we didn't explicitly ask for a zero conf channel.
2025-08-13 14:20:28 +09:30
Rusty Russell
4e94a6cb2b lightningd: make option_channel_type compulsory.
As per BOLT recommendation https://github.com/lightning/bolts/pull/1232, this means
we will insist on this being available.

For CLN, we added this in 0.12.0 (2022-08-23), though there were fixes as late as 24.02. Either way that's well outside our support window.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Closes: https://github.com/ElementsProject/lightning/issues/8152
Changelog-Changed: Protocol: We now insist that peers support `option_channel_type` (in CLN since 0.12.0 in late 2022, similar for other implementations).
2025-08-13 14:20:28 +09:30
Dusty Daemon
39451c005a channel: Fix channel state max value 2025-08-07 16:13:20 +09:30
Rusty Russell
c0ff1b5768 lightningd: update comments about known preimages for missing HTLC outputs.
Doesn't change anything for us, since we will already fulfull the incoming
HTLCs if we can, but good to note.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-07-07 11:18:37 +09:30
Rusty Russell
d86b88d409 channeld: remove experimental-upgrade-protocol.
It was already disabled by Dusty due to a number conflict with splicing, and
the proposal probably needs updating to use quiescence now that is merged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: The non-functional `experimental-upgrade-protocol` config option.
2025-07-07 11:18:37 +09:30
Rusty Russell
46fb007ea0 lightningd: remove experimental-anchors / experimental-onion-messages.
Changelog-Removed: Config: `experimental-anchors` and `experimental-onion-messages` (deprecated 24.02 / 24.08, disabled v25.05).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-06-25 09:06:31 +09:30
Rusty Russell
4fc9856cf2 lightningd: remove clnrest override options.
Changelog-Removed: Config: autodetection for rest-port/rest-protocol/rest-host/rest-certs options to clnrest-* (deprecated v23.11, disabled v25.02).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-06-25 09:06:31 +09:30
Rusty Russell
b3c1728abc lightningd: remove --max-locktime-blocks.
Changelog-Removed: Config: `max-locktime-blocks` (deprecated v24.05, disabled v25.02).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-06-25 09:06:31 +09:30
Rusty Russell
884778635c deprecations: update v26.05 to v26.06.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Future release schedule moved one month: v25.05 is now v25.06, and all deprecations incremented accordingly.
2025-06-24 09:57:24 +09:30
Rusty Russell
094ac3518f deprecations: update v26.02 to v26.03.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-06-24 09:57:24 +09:30
Rusty Russell
09e03e98ef deprecations: update v25.11 to v25.12.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-06-24 09:57:24 +09:30