json_tok_streq(…) and json_get_member(…) are convenience wrappers for
json_tok_strneq(…) and json_get_membern(…) respectively. Unfortunately, using
them incurs a performance penalty in the common case where they are called with
a string literal argument because the compiler is unable to substitute a
compile-time constant in place of the buried call to strlen(…).
For example,
json_get_member(buf, tok, "example");
…will have worse performance than…
json_get_membern(buf, tok, "example", strlen("example"));
…because the former is forced to scan over "example" at run-time to count its
length whereas the latter is able to elide the strlen(…) call at compile time.
Hoist these convenience functions up into common/json_parse_simple.h and mark
them as inline so that the compiler can elide the strlen(…) call in the common
case of calling these functions with a string literal argument.
Changelog-None
We cannot use subd_req() here: replies will come out of order, and the
we should not simply assign the reponses in FIFO order.
Changelog-Fixed: lightningd: don't get confused with parallel ping commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
One reason why ping processing could be slow is that, once we receive
a message from the peer to send to a subdaemon, we don't listen for
others until we've drained that subdaemon queue entirely.
This can happens for reestablish: slow machines can take a while to
set that subdaemon up.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Requested-by: Michael at Boltz
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `xpay` now publishes `pay_part_start` and `pay_part_end` notifications on every payment send attempt.
Implement the sending of `start_batch` and `protocol_batch_element` from `channeld` to `connectd`.
Each real peer wire message is prefixed with `protocol_batch_element` so connectd can know the size of the message that were batched together.
`connectd` intercepts `protocol_batch_element` messages and eats them (doesn’t forward them to peer) to get individual messages out of the batch.
It needs this to be able to encrypt them individiaully. Afterwards it recombines the now encrypted messages into a single message to send over the wire to the peer.
`channeld` remains responsible for making `start_batch` the first message of the message bundle.
The new spec sends `batch_size` in `start_batch` and removes it from `commitment_signed` so we need to stop processing it in `commitment_signed`.
Since the tlv is now reduced to one element and that automagically turns it into a direct use TLV so we have to update the code everywhere it is referenced.
We add `start_batch` to match t-bast’s splicing spec and we add a new internal wire type `WIRE_PROTOCOL_BATCH_ELEMENT` using the type number 0
Changelog-Added: support for `start_batch`
This replaces the old "iterate through each peer, then each peer's channel" suboptimality.
A bit of care required that we don't expose scids if we're forwarding,
but that was already carefully handled.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We allowed NULL for stub channels, but just don't put the stub scid
into the hash tables. This cleans up all the callers to make it
clear this is a non-optional parameter.
We opencode channel_set_random_local_alias, since there's only one caller now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We have a migration which ensures this, but then I discovered that did
*not* address channels without an SCID yet. So fixed the migration, and
simpligied the code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: we now allow routing through old short-channel-ids once a splice is done (previously we would refuse, leading to a 6 block gap in service).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There can be any number of these, and it will be useful to allow
routing by older scids (when other nodes haven't seen our gossip, or
even before we *can* announce the new post-splice channel).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When we had to use the number to the db_bind call, these annotations made
sense, but since 0bcff1e76d (for v23.08) we
removed that.
So remove all the counters, which are simple overhead if we want to
change something.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since the signature data may vary, we must copy the new psbt into splicing->current_psbt.
This never occured during normal operation, but when doing a cross splice there may be vital signature(s) in the psbt that came from another splice that get dropped without this step.
When doing a cross channel splice, inputs move from having no SIGHASH to having SIGHASH_ALL assigned.
This causes psbt_get_changeset to flag the input as having changed, as the sighash value is compared.
This causes the second channel splice to `tx_remove_input` the input as it doesn’t match anymore, breaking the splice.
We fix this by removing the sighash value from input comparisions.
When doing a multi channel splice, we need to break the deadlock by signing the shared output early (even though it is not sent to the peer until later).
Previous behavior was to fail on abort when we have signatures in the inflight — change this behavior to fail on abort if we have sent our peer our signatures.
We used to use a check on the active psbt to see if our splice signature was in it — but now we need to generate the signature early.
So we have to add a field tracking if we’ve sent it and add it to the database, wire protocols, and inflight objects.
Changelog-Added: Plugins: `sql` also supports functions `json_object(key1, value1, ...)` to construct JSON objects and `json_group_array(value)` to aggregate rows into JSON array.
Security Considerations
- No new SQL injection risks: Functions only process explicitly provided column values (no arbitrary string parsing).
- Explicit column requirements: Wildcards (*) are not supported, all fields must be named (e.g., json_object('peer_id', id)).
- Permission-bound data access: Functions adhere to the same table/row permissions as the underlying query.
Performance Impact
- Optimized native execution: Leverages SQLite’s built-in JSON1 extension (when available) for efficiency.
- Moderate CPU overhead: Complex nesting may impact performance on large datasets but still faster than application-layer JSON conversion.
This was needed for v23.05 which would set opt_scid_alias even if we didn't.
Now everyone handles it properly, we can simply set it unconditionally.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: backwards compatibility allowances for CLN before 23.08 which didn't handle option_scid_alias properly.
Prior to it being compulsory, these daemons would need a default value. Now it's
always required, it's clearer if it's always told.
There's no "default_channel_type" now everyone has to specify channel_type either,
so rename it to "desired_channel_type" and put it in lightningd specifically.
Note that the channel_type can have options added: either option_scid_alias or option_zeroconf.
This results in a slight behavior change: we will get type zeroconf even if we didn't ask for it, if they gave it to us.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: fundchannel / fundchannel_start returned `channel_type` will include option_zeroconf if it was implied by a 0 minimum_depth, even if we didn't explicitly ask for a zero conf channel.
As per BOLT recommendation https://github.com/lightning/bolts/pull/1232, this means
we will insist on this being available.
For CLN, we added this in 0.12.0 (2022-08-23), though there were fixes as late as 24.02. Either way that's well outside our support window.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Closes: https://github.com/ElementsProject/lightning/issues/8152
Changelog-Changed: Protocol: We now insist that peers support `option_channel_type` (in CLN since 0.12.0 in late 2022, similar for other implementations).
