Commit Graph

16600 Commits

Author SHA1 Message Date
Matt Whitlock
ea2feccbf8 common: set errno=0 before calling strto{l,ul,ull}
The strto{l,ul,ull} functions do not set errno upon a successful return, so a
successful return from a maximally valued input could be misinterpreted as an
overflow error if errno happened already to be set to ERANGE before the call.
To guard against this edge case, always set errno to zero before calling these
functions if checking errno afterward.

Changelog-None
2025-08-15 10:15:16 +09:30
Rusty Russell
22b452ad19 lightningd: have onchaind inform us when to make a channel penalty_adj.
bookkeeper used to generate these as channel events, now lightningd does.

We also add a "journal" event, which we will need later too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
b1fa2ef30b bkpr: remove different currency support.
We still output the fields, they're just always the currency of the node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `bookkeeper` now explicitly assumes every transaction is in the same currency as the node (true unless you added manually)
2025-08-14 19:49:09 +09:30
Rusty Russell
47c2735e17 bkpr: remove currency support from balances.
We're going to get rid of this concept, but the main change is that the
account_get_balance API can be drastically simplified:

account_get_credit_debit() accesses the raw fields, never fails, but
returns the a flag which tells us if the account doesn't actually have
any events.

The one place we care about the balance, calculate by hand.  Then
account_get_balance() (and struct account_balance) can simply be
moved to th test.

Subtly, without the "GROUP BY" clause, you always get one row, even if
there are no rows (but the SUM are null).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
0c9dc4cf07 bkpr: remove unused "account_exits" parameter to account_get_balance().
Only used in tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
b0231a59d8 common: put "external" and "wallet" strings, and test functions into common/coin_mvt.h
They're scattered and reproduced in many places: unify them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
841a8bd03a lightningd: extract core of coin_movement notification, for use in list functions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `coin_movement` notification `utxo_txid`, `vout` and `txid` fields (use `utxo` and `spending_txid`).
Changelog-Added: JSON-RPC: `coin_movement` notification `utxo` field.
Changelog-Added: JSON-RPC: `coin_movement` notification `spending_txid` field.
2025-08-14 19:49:09 +09:30
Rusty Russell
d066a5f301 common: make chain_coin_mvt's outpoint member a non-pointer.
It's always set, and in fact we assume it is (journal entries are not
internal to lightningd, so we won't see them in
lightningd/notification.c: that comment is misleading).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
c0221b1b74 coin_mvt: put timestamp into the struct.
This is not particularly relevant now (it's always the current time) but will be
useful when we implement the list commands.

Note that timestamp is set to be "u32" in various schemas.  This will
only become a problem on Sun 07 Feb 2106 06:28:15 UTC.  I apologize to
my grandchildren in advance.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
650f3882dd lightningd: separate coin_movement tags array into primary_tag and extra_tags.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `coin_movement` notification `tags` array (use `primary_tag` and `extra_tags`).
Changelog-Added: JSON-RPC: `coin_movement` notification `primary_tag` and `extra_tags`.
2025-08-14 19:49:09 +09:30
Rusty Russell
9b2c87f052 common: implement primary_mvt_tag() to extract primary tag.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
8c315c15b6 coin_mvt: use bitmap instead of tal_arr for tags.
We're going to store them in the db this way, so I thought I'd see what it looks like if
we lift that interface all the way through.

We use a struct, so that types are checked strictly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
bafda5ce02 common: mvt_tag parsing routine.
Rather than open-coding in json_parse.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
1833cc2d28 common: mvt_tag_strs() function to turn tags array into strings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
0d40d43c01 common: enforce the use of a "primary" tag in coin_mvt tags.
Undocumented, but the first tag in the coin_movement notification is
considered the primary tag, and the others are optional.  The
bookkeeper plugin relies on this!

Enforce that this is true, and in the process document in the code which
is the primary tag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
9c6e7b8d42 pytest: don't assume that tags will always be in the same order.
This isn't a robust assumption, so sort them before comparison.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
5756b54f38 common: rename enum mvt_tag values.
Prefix MVT_ to them, for clarity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
1d3237f801 common/coin_mvt: make more fields const, reorder fields.
Make the common fields the first ones, and make part_and_group and
payment_hash const pointers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
6fc0b1f3d1 common/coin_mvt: add struct mvt_account_id to separate channels from others.
This means we can keep a pointer to the channel directly, *or* a string.
This avoids gratuitous formatting (on creation) and lookups (later).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
e372c2ec0b common/coin_mvt: use enum rather than true/false for credit/debit.
More readable for me.  Also, change order so we definitely break
compilation on all callers (putting enum before amount).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
bc5e54d153 common: coin movements for payments need group id as well as part id.
The part id is *only* unique within a group.  The payment_hash / partid / groupid tuple is unique.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `coin_movement` notification with `part_id` field now always has `group_id` field.
2025-08-14 19:49:09 +09:30
Rusty Russell
f4ac8fccb6 common/coin_mvt: make more parameters const.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
e32b5141ef common: remove struct coin_mvt altogether.
Now we only ever use `struct chain_coin_mvt` or `struct channel_coin_mvt`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
45678aafc5 lightningd: use channel_coin_mvt / chain_coin_mvt directly for notifications.
Rather than converting to a generic coin_mvt struct, use these directly in
the notification, which is more explicit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
b8b1d45ddc db: hand amount_msat / amount_sat by copy, not pointer.
This is how we handle amount_msat and amount_sat everywhere these days, and this
wasn't updated.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
0534c524b5 common: remove "ignored" tag.
We don't actually set it any more.  The bookkeeper db does a migration
for old anchors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
aee7ebbeef onchaind: don't consider our anchors to be payments into the wallet.
This makes our final balance not match our wallet:
1. We only spend the anchor when we need to boost the commitment tx,
   which we don't always do (sometimes the peer does, sometimes it's
   not worth it).
2. We don't put the UTXO in our wallet, because we don't consider it
   "ours": anyone can spend it after 16 blocks.

We used to use the tag "ignored" for this, but that's overly complex
IMHO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
a25c1d45ad libplugin: make jsonrpc_set_datastore_binary() take an explicit length.
This means it doesn't have to be a tal ptr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
ed77edf4c6 db: better diagnostics when a query fails.
Print the error!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
9b4a7085c2 pytest: save pre-movement dbs, and accounting dbs.
We can use these to test migrations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 19:49:09 +09:30
Rusty Russell
6fbc5d02ca common/htlc_wire: add towire/fromwire helpers for wrapped tlv streams.
And make sure we check the length properly in fromwire!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 18:57:05 +09:30
Rusty Russell
63065aa99c common: handle taken() extra_tlvs in new_existing_htlc properly.
Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 18:57:05 +09:30
Peter Neuroth
5c1fd782ec tools: Remove lockfiles from spell-checking
The rare case happened where a lockfile sha-sum contained a "Ctlv" which
spell-check complained about. Stupid lockfiles that don't know it is
actually "cltv"!

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Peter Neuroth
5e620cabce tools: Filter "highlight" case insensitive
There was a problem with a ‘highlight’ that was misunderstood as a
spelling mistake in lib-wally. Since ‘hightlight’ is already filtered
out, we simply instruct grep to ignore upper/lower case when filtering.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Peter Neuroth
40c01ea21b docs: Add extra_tlvs to the htlc_accepted_hook doc
Changelog-Added: The `htlc_accepted_hook` now gets the TLV-stream
attached to the HTLC passed through as `extra_tlvs` and can replace it.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Peter Neuroth
dbbb113123 tests: Add tests for extra_tlvs in hook
Adds some testcases for custom tlvs, set by a htlc_accepted_hook. We
check that the custom tlvs replace the update_add_htlc_tlvs and get
forwarded to the peer. We also check that a malformed tlv will result in
a **BROKEN** behaviour.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Peter Neuroth
2264b93c6c lightningd: Add extra_tlvs to htlc_accepted_hook
Add serializing and deserializing of the extra tlvs to to the
htlc_accepted_hook to allow plugin users to replace the tlv stream that
is attached to the update_add_htlc message on forwards.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Peter Neuroth
ef52de88aa channeld: Add extra_tlvs to incomming wire msg
This appends the extra_tlvs to the internal channeld_offer_htlc wire
msg. We also recombine the extra_tlvs with the blinded path key for
forwarding htlcs.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Peter Neuroth
631a1d950f channeld: Add extra_tlvs to wire htlcs
This appends the extra_tlvs to the internal wire htlcs "added" and
"existing" for the extra tlvs to be handed to lightningd.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
Peter Neuroth
f4cc633c6f channeld: add extra_tlvs from update_add_htlc msg
We currently only consider known tlv types in the internal
representation of a htlc. This commit adds the remaining unknown tlv
fields to the htlc as well. This is in prepareation to forward these to
the htlc_accepted_hook.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
daywalker90
e3e41163e2 cln-bip353: add plugin that fetches payment instructions from human readable addresses
Changelog-Add: new plugin cln-bip353 that can fetch BIP-353 payment instructions from human readable addresses
2025-08-14 18:41:23 +09:30
Matt Whitlock
9112c1d518 common/json_parse_simple: make convenience functions inline
json_tok_streq(…) and json_get_member(…) are convenience wrappers for
json_tok_strneq(…) and json_get_membern(…) respectively. Unfortunately, using
them incurs a performance penalty in the common case where they are called with
a string literal argument because the compiler is unable to substitute a
compile-time constant in place of the buried call to strlen(…).

For example,

	json_get_member(buf, tok, "example");

…will have worse performance than…

	json_get_membern(buf, tok, "example", strlen("example"));

…because the former is forced to scan over "example" at run-time to count its
length whereas the latter is able to elide the strlen(…) call at compile time.

Hoist these convenience functions up into common/json_parse_simple.h and mark
them as inline so that the compiler can elide the strlen(…) call in the common
case of calling these functions with a string literal argument.

Changelog-None
2025-08-14 17:53:39 +09:30
Rusty Russell
5f5440383d lightningd: fix race with crossover pings.
We cannot use subd_req() here: replies will come out of order, and the
we should not simply assign the reponses in FIFO order.

Changelog-Fixed: lightningd: don't get confused with parallel ping commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 17:35:39 +09:30
Rusty Russell
a0fd72eb5e connectd: warn if we ignore peer incoming for longer than 5 seconds.
One reason why ping processing could be slow is that, once we receive
a message from the peer to send to a subdaemon, we don't listen for
others until we've drained that subdaemon queue entirely.

This can happens for reestablish: slow machines can take a while to
set that subdaemon up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 17:35:39 +09:30
Rusty Russell
0938d544ed gossipwith: add flag to insist that we receive all messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 17:35:39 +09:30
Rusty Russell
c88ec27de3 pytest: test xpay notifications.
The custom_notifications handler produces really ugly results, and I
was lazy, but it works!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 16:52:44 +09:30
Rusty Russell
e24bd9685a xpay: add pay_part_start and pay_part_end notifications.
Requested-by: Michael at Boltz
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `xpay` now publishes `pay_part_start` and `pay_part_end` notifications on every payment send attempt.
2025-08-14 16:52:44 +09:30
Rusty Russell
4c23cf296c libplugin: correctly mark that plugin_notification_end STEALS the stream.
And also slightly generalize: plugin_notification_start() can take any
tal ptr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 16:52:44 +09:30
Rusty Russell
80309dfbe6 common: add json_add_timerel helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 16:52:44 +09:30
Dusty Daemon
052f36cf2e connectd: Implement sending of start_batch
Implement the sending of `start_batch` and `protocol_batch_element` from `channeld` to `connectd`.

Each real peer wire message is prefixed with `protocol_batch_element` so connectd can know the size of the message that were batched together.

`connectd` intercepts `protocol_batch_element` messages and eats them (doesn’t forward them to peer) to get individual messages out of the batch.

It needs this to be able to encrypt them individiaully. Afterwards it recombines the now encrypted messages into a single message to send over the wire to the peer.

`channeld` remains responsible for making `start_batch` the first message of the message bundle.
2025-08-14 16:40:04 +09:30