Commit Graph

1933 Commits

Author SHA1 Message Date
Rusty Russell
5a52c6bd63 pyln-testing: catch special CI string so we can have non-BROKEN CI warnings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-09-30 11:37:31 +09:30
Chandra Pratap
f3ddc076a7 common/test: Add a test to trigger the bug
Add a test in `common/test/run-wireaddr.c` that reproduces the
out-of-bounds error when the fix is not applied.
2025-09-22 10:23:13 +09:30
Chandra Pratap
9fe88b430e common/wireaddr: Fix an out-of-bounds bug in the address parser
Changelog-Fixed: In `struct wireaddr`, the `addr` buffer is defined
with a length of DNS_ADDRLEN (255). When parsing a valid DNS name
that is exactly 255 bytes long, the subsequent attempt to append a
`NULL` terminator overruns the buffer and triggers an out-of-bounds
error under UBSan.

Fix this by removing the line that appends `NULL`. This change is
safe because the preceding call to:

`memset(&addr->addr, 0, sizeof(addr->addr))`

already zeroes the entire buffer.
2025-09-22 10:23:13 +09:30
Matt Whitlock
71ddba283f common/json_parse_simple: drop redundant and wrong json_str_to_u64()
The json_str_to_u64() function contains incorrect logic. It chops one character
off of the beginning and end of the JSMN token and then parses the remainder as
a u64, but JSMN_STRING tokens already do not include the enclosing quotation
marks, so json_str_to_u64() would actually parse the JSON string "1234" into
the integer 23. Oops! Also note that it would simply fail on all input strings
shorter than two characters since tok->end would wind up *before* tok->start.

Just drop the function entirely. It was only used in one place, and that place
explicitly doesn't care whether its input is a JSON number or a numeric string,
and it was already calling json_to_u64() as an alternative, and that function
already accepts both JSON strings and JSON numbers as input, so the call to
json_str_to_u64() would have been entirely redundant if it had been correct.

Changelog-Fixed: The `keysend` command no longer corrupts the type numbers of extra TLVs when they are specified as numeric strings longer than 2 digits.
2025-09-22 10:16:51 +09:30
Chandra Pratap
43516ce047 common/amount: prevent scaling with invalid factors in amount scale functions
Changelog-Fixed: Add checks in `amount_msat_scale` and
`amount_sat_scale` to return false when the scaling factor
is -NaN or negative.

This is important as the scaling factor may come from external
sources like wire, which can cause runtime errors if not handled
properly.
2025-09-15 11:29:49 +09:30
Erick Cestari
49c2f290f2 common/bolt11: Fix BOLT11 hash calculation for unknown fallback address versions
Changelog-Fixed: Fixed hash calculation inconsistency when processing
invoices with unknown fallback address versions.
2025-09-04 14:10:58 +09:30
Chandra Pratap
c0673aeac4 common: fix dangling memory allocation in daemon_conn_new_()
Changelog-Fixed: Use the correct context in `daemon_conn_new_()`
by allocating `struct daemon_conn` with `ctx` instead of `NULL`.

This ensures proper ownership and cleanup of `daemon_conn` objects,
avoiding memory leaks.
2025-09-04 13:29:29 +09:30
Rusty Russell
22a5e9e7e6 common: reintroduce "ignored" primary tag.
Prior to 23.05, we used this tag to mark onchain to-self inputs we didn't
wait for (because they were too small).  This fixes migration if that happened
(and we are debating whether we should re-introduce this!).

```
lightningd: FATAL SIGNAL 6 (version v25.09rc2)                      
0x100c8683 send_backtrace                                           
        common/daemon.c:33                                          
0x100c876f crashdump                                                
        common/daemon.c:78                                          
0x7fffb2080493 ???                                                  
        ???:0                                                       
0x7fffb1ab0cac ???                                                  
        __pthread_kill_implementation+0x1bc:0
0x7fffb1a48a5b ???                                                  
        __GI_raise+0x2b:0                                           
0x7fffb1a2a3db ???                                                  
        __GI_abort+0x153:0                                          
0x100935b7 migrate_from_account_db
        wallet/account_migration.c:424
0x10093ff7 db_migrate                                               
        wallet/db.c:1139                                            
0x10096763 db_setup                                                 
        wallet/db.c:1185                                            
0x100a1bcb wallet_new                                               
        wallet/wallet.c:223                                         
0x1004485f main                                                     
        lightningd/lightningd.c:1311
0x7fffb1a2aba3 ???                                                  
        __libc_start_call_main+0x93:0
0x7fffb1a2adeb ???                                                  
        __libc_start_main_alias_1+0x1ab:0
0xffffffffffffffff ???                                              
        ???:0                                                       
lightningd: Died with signal 6                                  
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/8484
2025-08-28 13:41:44 +09:30
Lagrang3
7e5cf41b4e htlc_wire: fix crash when adding an HTLC
In line channeld/channeld_wiregen.c:832 `*added+i` is not a tal object hence
the instruction in common/htlc_wire.c:200 `tal_arr(ctx, struct tlv_field, 0);` crashes CLN.
This is fixed by stating that added_htlc is a a varsize_type.

Logs:

2025-08-16T02:25:28.640Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v25.05-200-g79b959b)V
...
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:95 (call_error) 0x54f6bc
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:169 (check_bounds) 0x54f75a
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:178 (to_tal_hdr) 0x54f782
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:193 (to_tal_hdr_or_null) 0x54f7c7
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:471 (tal_alloc_) 0x54ffe4
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:517 (tal_alloc_arr_) 0x5500c4
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: common/htlc_wire.c:200 (fromwire_len_and_tlvstream) 0x48d63d
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: common/htlc_wire.c:234 (fromwire_added_htlc) 0x48dd23
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: channeld/channeld_wiregen.c:832 (fromwire_channeld_got_commitsig) 0x4c61fa
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:2377 (peer_got_commitsig) 0x4549cb
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/channel_control.c:1552 (channel_msg) 0x4140fe
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/subd.c:560 (sd_msg_read) 0x461513
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:60 (next_plan) 0x544885
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:422 (do_plan) 0x544cea
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:439 (io_ready) 0x544d9d
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:455 (io_loop) 0x54665d
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x42d220
2025-08-16T02:25:28.640Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1487 (main) 0x43280f

gdb inspection:
830             *added = num_added ? tal_arr(ctx, struct added_htlc, num_added) : NULL;
831             for (size_t i = 0; i < num_added; i++)
832                     fromwire_added_htlc(&cursor, &plen, *added + i);
(gdb) p i
$3 = 1

Changelog-None: crash introduced this release.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
[ Added test, removed Changelog --RR ]
2025-08-27 14:14:31 +09:30
Rusty Russell
ebc9a8b28e lightningd: fix name of chainmoves journal entry.
```
lightningd: FATAL SIGNAL 6 (version v25.09rc1-1-ga00ed81)
0x5c9e848ca050 send_backtrace
	common/daemon.c:33
0x5c9e848ca249 crashdump
	common/daemon.c:78
0x7f451664532f ???
	./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x7f451669eb2c __pthread_kill_implementation
	./nptl/pthread_kill.c:44
0x7f451669eb2c __pthread_kill_internal
	./nptl/pthread_kill.c:78
0x7f451669eb2c __GI___pthread_kill
	./nptl/pthread_kill.c:89
0x7f451664527d __GI_raise
	../sysdeps/posix/raise.c:26
0x7f45166288fe __GI_abort
	./stdlib/abort.c:79
0x5c9e84893ac3 migrate_from_account_db
	wallet/account_migration.c:500
0x5c9e848943f7 db_migrate
	wallet/db.c:1139
...
#5  0x0000555555615ac4 in migrate_from_account_db (ld=0x555555999238, db=0x55555599b158) at wallet/account_migration.c:500
500				abort();
(gdb) p ev->tag
$1 = 0x555555a4fbb8 "journal_entry"
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-21 13:21:17 +09:30
Rusty Russell
194febe873 wallet: generate fixup chainmoves and channelmoves when first starting.
If we don't have an accountdb from bookkeeper:

1. Generate a deposit chain event for every confirmed UTXO.
2. Generate an open chain event for every open, confirmed channel.
3. Generate a push/lease event if necessary.
4. Generate a fixup "journal" entry if balance is different from initial.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
52d69df10f lightningd: migrate events from bookkeeper at startup.
We take over the --bookkeeper-dir and --bookkeeper-db options, and
then if we can find the bookkeeper db we extract the records to
initialize our chain_moves and channel_moves tables.

Of course, bookkeeper now needs to not register those options.

When bookkeeper gets invoked the first time, it will reconstruct
everything from listchannelmoves and listcoinmoves.  It cannot
preserve manually-added descriptions, so we put those in the datastore
for it ready to go.

Note that the order of onchain_fee changes slightly from the original.
But this is fine.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
78f88c1c0c common: use "foreign" in extra_tags to indicate a chain movement is injected.
This allows the bookkeeper plugin to know it's not actually a channel account.

Remove the "ignored" tag from the schema too: we removed it previously.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
254b7ed666 lightningd: new internal-use commands to allow utxo spend / deposit injection.
For the moment, we'll continue to use bookkeeper to monitor the
notifications to insert these (we don't have the internal infrastructure
for that, and actually these commands are probably better than using
notifications).

We hoist param_outpoint() into common code, since there are already
two uses.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
74abe33fe1 common: fix values of enum mvt_tag since they're now embedded in the db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
103a49ef51 xpay: support paying a (simple) bolt12 offer directly.
fetchinvoice is still good for detailed diagnostics and handling
recurring invoices and alternate currencies, but this covers the
"throw some sats" case well.

Changelog-Added: JSON-RPC: `xpay` can now pay a simple offer directly, rather than requiring fetchinvoice first.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-18 14:57:46 +09:30
Matt Whitlock
41d31dcd19 avoid UB when calling ctype functions
The character classification functions in <ctype.h> are designed to
classify characters returned by <stdio.h> getchar() and friends, which
return characters as signed integers in the range 0 to 255 or EOF. The
behavior of the ctype functions is undefined if they are passed a value
outside of that range, which may happen if they are passed a char-typed
value and the system's char type is signed.

<ccan/str/str.h> defines some inline utility functions that perform the
necessary cast to coerce a char-typed argument into the allowed value
range. Call these wrappers instead of the bare ctype functions when
classifying char-typed characters.

Changelog-None
2025-08-18 14:03:20 +09:30
Matt Whitlock
c28f5e70a8 use json_escape_unescape_len()
This avoids making an extra copy of the escaped string.

Note that jsonrpc_command_add() no longer accepts usage strings
containing invalid escape sequences. (Previously, it would quietly
accept such a string without unescaping anything.)

Changelog-None
2025-08-15 15:23:57 +09:30
Matt Whitlock
ea2feccbf8 common: set errno=0 before calling strto{l,ul,ull}
The strto{l,ul,ull} functions do not set errno upon a successful return, so a
successful return from a maximally valued input could be misinterpreted as an
overflow error if errno happened already to be set to ERANGE before the call.
To guard against this edge case, always set errno to zero before calling these
functions if checking errno afterward.

Changelog-None
2025-08-15 10:15:16 +09:30
Rusty Russell
22b452ad19 lightningd: have onchaind inform us when to make a channel penalty_adj.
bookkeeper used to generate these as channel events, now lightningd does.

We also add a "journal" event, which we will need later too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
b0231a59d8 common: put "external" and "wallet" strings, and test functions into common/coin_mvt.h
They're scattered and reproduced in many places: unify them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
841a8bd03a lightningd: extract core of coin_movement notification, for use in list functions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `coin_movement` notification `utxo_txid`, `vout` and `txid` fields (use `utxo` and `spending_txid`).
Changelog-Added: JSON-RPC: `coin_movement` notification `utxo` field.
Changelog-Added: JSON-RPC: `coin_movement` notification `spending_txid` field.
2025-08-14 19:49:09 +09:30
Rusty Russell
d066a5f301 common: make chain_coin_mvt's outpoint member a non-pointer.
It's always set, and in fact we assume it is (journal entries are not
internal to lightningd, so we won't see them in
lightningd/notification.c: that comment is misleading).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
c0221b1b74 coin_mvt: put timestamp into the struct.
This is not particularly relevant now (it's always the current time) but will be
useful when we implement the list commands.

Note that timestamp is set to be "u32" in various schemas.  This will
only become a problem on Sun 07 Feb 2106 06:28:15 UTC.  I apologize to
my grandchildren in advance.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
9b2c87f052 common: implement primary_mvt_tag() to extract primary tag.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
8c315c15b6 coin_mvt: use bitmap instead of tal_arr for tags.
We're going to store them in the db this way, so I thought I'd see what it looks like if
we lift that interface all the way through.

We use a struct, so that types are checked strictly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
bafda5ce02 common: mvt_tag parsing routine.
Rather than open-coding in json_parse.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
1833cc2d28 common: mvt_tag_strs() function to turn tags array into strings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
0d40d43c01 common: enforce the use of a "primary" tag in coin_mvt tags.
Undocumented, but the first tag in the coin_movement notification is
considered the primary tag, and the others are optional.  The
bookkeeper plugin relies on this!

Enforce that this is true, and in the process document in the code which
is the primary tag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
5756b54f38 common: rename enum mvt_tag values.
Prefix MVT_ to them, for clarity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
1d3237f801 common/coin_mvt: make more fields const, reorder fields.
Make the common fields the first ones, and make part_and_group and
payment_hash const pointers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
6fc0b1f3d1 common/coin_mvt: add struct mvt_account_id to separate channels from others.
This means we can keep a pointer to the channel directly, *or* a string.
This avoids gratuitous formatting (on creation) and lookups (later).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
e372c2ec0b common/coin_mvt: use enum rather than true/false for credit/debit.
More readable for me.  Also, change order so we definitely break
compilation on all callers (putting enum before amount).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
bc5e54d153 common: coin movements for payments need group id as well as part id.
The part id is *only* unique within a group.  The payment_hash / partid / groupid tuple is unique.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `coin_movement` notification with `part_id` field now always has `group_id` field.
2025-08-14 19:49:09 +09:30
Rusty Russell
f4ac8fccb6 common/coin_mvt: make more parameters const.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
e32b5141ef common: remove struct coin_mvt altogether.
Now we only ever use `struct chain_coin_mvt` or `struct channel_coin_mvt`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
0534c524b5 common: remove "ignored" tag.
We don't actually set it any more.  The bookkeeper db does a migration
for old anchors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
6fbc5d02ca common/htlc_wire: add towire/fromwire helpers for wrapped tlv streams.
And make sure we check the length properly in fromwire!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 18:57:05 +09:30
Rusty Russell
63065aa99c common: handle taken() extra_tlvs in new_existing_htlc properly.
Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 18:57:05 +09:30
Peter Neuroth
631a1d950f channeld: Add extra_tlvs to wire htlcs
This appends the extra_tlvs to the internal wire htlcs "added" and
"existing" for the extra tlvs to be handed to lightningd.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Matt Whitlock
9112c1d518 common/json_parse_simple: make convenience functions inline
json_tok_streq(…) and json_get_member(…) are convenience wrappers for
json_tok_strneq(…) and json_get_membern(…) respectively. Unfortunately, using
them incurs a performance penalty in the common case where they are called with
a string literal argument because the compiler is unable to substitute a
compile-time constant in place of the buried call to strlen(…).

For example,

	json_get_member(buf, tok, "example");

…will have worse performance than…

	json_get_membern(buf, tok, "example", strlen("example"));

…because the former is forced to scan over "example" at run-time to count its
length whereas the latter is able to elide the strlen(…) call at compile time.

Hoist these convenience functions up into common/json_parse_simple.h and mark
them as inline so that the compiler can elide the strlen(…) call in the common
case of calling these functions with a string literal argument.

Changelog-None
2025-08-14 17:53:39 +09:30
Rusty Russell
80309dfbe6 common: add json_add_timerel helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 16:52:44 +09:30
Dusty Daemon
07f4bc39b1 splice: Add start_batch and an internal wire type
We add `start_batch` to match t-bast’s splicing spec and we add a new internal wire type `WIRE_PROTOCOL_BATCH_ELEMENT` using the type number 0

Changelog-Added: support for `start_batch`
2025-08-14 16:40:04 +09:30
Dusty Daemon
a5a71f3b22 psbt: don’t accumlate 0 lenth widtness data 2025-08-14 04:12:17 +09:30
Dusty Daemon
2e660d9f06 splice: Fix cross-channel splices
When doing a cross channel splice, inputs move from having no SIGHASH to having SIGHASH_ALL assigned.

This causes psbt_get_changeset to flag the input as having changed, as the sighash value is compared.

This causes the second channel splice to `tx_remove_input` the input as it doesn’t match anymore, breaking the splice.

We fix this by removing the sighash value from input comparisions.
2025-08-14 04:12:17 +09:30
Dusty Daemon
286124e3d9 interactive_tx: Add verbose logging
Adding verbose logging option to introspect into what interactive tx is doing with inputs.
2025-08-14 04:12:17 +09:30
Rusty Russell
506fa914e0 lightningd: always tell openingd/dualopend what channel type we want.
Prior to it being compulsory, these daemons would need a default value.  Now it's
always required, it's clearer if it's always told.

There's no "default_channel_type" now everyone has to specify channel_type either,
so rename it to "desired_channel_type" and put it in lightningd specifically.

Note that the channel_type can have options added: either option_scid_alias or option_zeroconf.

This results in a slight behavior change: we will get type zeroconf even if we didn't ask for it, if they gave it to us.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: fundchannel / fundchannel_start returned `channel_type` will include option_zeroconf if it was implied by a 0 minimum_depth, even if we didn't explicitly ask for a zero conf channel.
2025-08-13 14:20:28 +09:30
Dusty Daemon
9215653c30 channel: Add test for channel state max
Adds a test so CI will fail if CHANNEL_STATE_MAX isn’t updated when new channel states are added.

Changelog-None
2025-08-07 16:13:20 +09:30
Lagrang3
fae176f4cc common/amount: add ceil division operation on msat
Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-07-19 10:09:17 +09:30
Chandra Pratap
e721b0a89a fuzz-tests: Enhance b64_encode() validation with roundtrip decoding
Changelog-None: Currently, fuzz testing for b64_encode() merely
encodes input and frees the result, providing no real verification
of its behavior.

Introduce a new b64_decode() function (modeled after b32_decode())
and update the fuzz test to perform a roundtrip—encoding followed
by decoding—to ensure that b64_encode() correctly preserves the
original data.
2025-07-08 12:54:56 +09:30