We don't explicitly save the return code in db, so we need to reconstruct it.
We didn't cover the "peer told us our onion was bad" corner case. But it's hardly
worth a changelog message, since users will never see this.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This showed up as a flake, where we "got lucky" and the sendpay resolved before waitsendpay was called. Instead, make this race explicit, so we can test it.
```
# FIXME: #define PAY_UNPARSEABLE_ONION 202
PAY_UNPARSEABLE_ONION = 202
> assert err.value.error['code'] == PAY_UNPARSEABLE_ONION
E assert 204 == 202
tests/test_misc.py:2152: AssertionError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means that we won't complain to peers which gossip about our
channels, but it does mean that our channel graph (like other nodes on
the network) will show two channels, not one, for the duration.
For this reason, we need askrene to omit local dying channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When installed, the name is `lightning-hsmtool`. We actually copy
`tools/hsmtool` to `tools/lightning-hsmtool` but that's a silly step
which we should get rid of.
So:
1. Make sure our documentation always refers to it as lightning-hsmtool.
2. Make sure our tests invoke it as `lightning-hsmtool`.
3. Rename the C file.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: "filters" can be specified on the `custommsg` hook to limit what message types the hook will be called for.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When running the integration test suite in a deeply nested directory
tree, the path name of the Unix domain socket might be longer than can
fit in a struct sockaddr_un. On Linux, we can use the /proc/self/cwd
trick to shorten the path name.
Changelog-Fixed: Integration tests no longer fail when run in a deeply nested directory on Linux.
```
lightningd-1 2025-10-27T11:26:04.285Z **BROKEN** plugin-bcli: bitcoin-cli exec failed: Argument list too long
```
Use -stdin to bitcoin-cli: we can then handle arguments of arbitrary length.
Fixes: https://github.com/ElementsProject/lightning/issues/8634
Changelog-Fixed: plugins: `bcli` would fail with "Argument list too long" when sending a giant tx.
This schema change updates newaddr to remove bip86 which was previously added, since don't want to make unnecessary schema changes this is being removed.
The generated files for the exposesecret schema change are also being added
Incorporate a time: this covers the restart case as well. And make it time_mono(),
which doesn't get overridden when we override normal wall time.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a trick from bcli: we ask bitcoind for the block, and it hands
us a 2MB hex blob (which we read in multiple parts). Our parser wades
through it all, but a quick search for '}' makes it much faster.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
For old channels, this can take a while, and it stops everything. But
we are only doing this to save space; it's not a *functional* necessity.
A quick and dirty test with 50,000 htlcs shows the htlc deletion took
450msec. I tried adding an index, and changing it to set hstate to
HTLC_STATE_INVALID instead of deleting entries, but it still took about 350ms.
Whereas the "COUNT(*)" only took 1.7msec, so it's worth keeping.
Reported-by: @michael1011
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: lightningd: we defer deletion of old htlcs on channel close, to avoid pausing for a long time (we clean them on startup)
Fixes: https://github.com/ElementsProject/lightning/issues/7962
Note that the test where we remove the database causes the bookkeeper
plugin to assert, since we have removed part (but not all!) of its data
by removing the datastore.
Once the transition to the datastore is complete, this can be restored.
Note that we destroy the request before receiving a response, which causes
a message in the trace span which was confusing our test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that we need a workaround for deprecated APIs where "channel_state_changed" output "null" which violated the schema.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
As per BOLT recommendation https://github.com/lightning/bolts/pull/1232, this means
we will insist on this being available.
For CLN, we added this in 0.12.0 (2022-08-23), though there were fixes as late as 24.02. Either way that's well outside our support window.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Closes: https://github.com/ElementsProject/lightning/issues/8152
Changelog-Changed: Protocol: We now insist that peers support `option_channel_type` (in CLN since 0.12.0 in late 2022, similar for other implementations).
By policy, our stub hsmd accepts everything: openingd is supposed to
sort this out (or use VLS for a real HSM implementation!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Config: the node no longer crashes if you set `watchtime-blocks` to 0 (which is fine for testing: don't do this on mainnet!).
Now we've make it only on existing channels, and not have to call
listdatastore every time, that means we can safely turn it on by
default.
Changelog-Added: Protocol: we now offer peer storage to any peers who create a channel.
Changelog-Deprecated: Config: `--experimental-peer-storage` (it's now the default).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
signmessagewithkey: allows to sign a message with a key associated with
one bitcoin address in our wallet.
Changelog-Added: add a new rpc command signmessagewithkey to sign input messages with keys from our wallet.
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Make sure we are sending bogus channel reestablish after recovering from
emergency.recover file and peer storage backup.
Key Changes:
- Add wait_for_log() with appropriate debug statements
We need to make sure the close tx is generated where we expect, so the following code
doesn't wait foreverL
```
2025-05-08T04:31:31.1071194Z > assert l2.rpc.wait('htlcs', 'deleted', 3)['deleted'] == 5
2025-05-08T04:31:31.1071518Z
2025-05-08T04:31:31.1071645Z tests/test_misc.py:3398:
...
2025-05-08T04:31:31.1079841Z E Failed: Timeout >1800.0s
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need one byte for the number of witness elements. Some callers added it themselves,
but it's always needed. So document and fix the callers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We previously treated it as a P2WPKH, which is wrong.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: wallet: fees are much closer to target feerate when doing txprepare/fundchannel.
To actually evaluate spend cost, we need to know whether it's taproot or not.
Using an enum (rather than making callers examine the script) means we can
ensure all cases are handled.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were supposed to put the sqlite db in a different directory, but
the test was wrong! So occasionally we would crash with:
```
Failed to commit DB transaction: Failed to commit a transaction: disk I/O error
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It makes the schema simpler, and indeed, expressable by GRPC.
Changelog-Added: JSON-RPC: `wait` now has separate `invoices`, `forwards` and `sendpays` objects for each subsystem.
Changelog-Deprecated: JSON-RPC: `wait` reply `details` object: use subsytem specific object instead.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Testing parenting handling revealed several issues:
1. By calling "trace_span_start" when CLN_TRACEPARENT is set produces a bogus
entry, for which the span_id is overwritten so we never end it.
2. We don't need to close the remote parent when we close the first child: in
fact, this causes the remaining traces to be detached from the parent!
3. Suspension should return current to the parent, not to NULL.
Now the traces balance as we expect.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We play with directory permissions, but sqlites needs that, and sometimes (due to a timer, perhaps?)
it gets really upset about it:
```
lightningd-1 2025-03-16T23:29:58.506Z INFO lightningd: Server started with public key 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518, alias JUNIORBEAM-f91eb11-modded (color #0266e4) and lightningd f91eb11-modded
lightningd-1 2025-03-16T23:29:58.617Z DEBUG lightningd: Adding block 101: 55af171ade598f8c4f9a494eaaffe6b9f5ea64c7b0f3b273694148adf7ad7385
lightningd-1 2025-03-16T23:29:58.752Z TRACE lightningd: Calling rpc_command hook of plugin cln-xpay
lightningd-1 2025-03-16T23:29:58.764Z TRACE lightningd: Plugin cln-xpay returned from rpc_command hook call
lightningd-1 2025-03-16T23:29:58.784Z TRACE lightningd: Calling rpc_command hook of plugin cln-xpay
lightningd-1 2025-03-16T23:29:58.840Z TRACE lightningd: Plugin cln-xpay returned from rpc_command hook call
lightningd-1 2025-03-16T23:29:58.854Z TRACE lightningd: Calling rpc_command hook of plugin cln-xpay
lightningd-1 2025-03-16T23:29:58.865Z DEBUG gossipd: REPLY WIRE_GOSSIPD_NEW_BLOCKHEIGHT_REPLY with 0 fds
lightningd-1 2025-03-16T23:29:58.901Z **BROKEN** lightningd: Error executing statement: db/exec.c:108: UPDATE vars SET intval = intval + 1 WHERE name = 'data_version' AND intval = ?: attempt to write a readonly database
lightningd-1 2025-03-16T23:29:58.916Z **BROKEN** lightningd: Error executing statement: db/exec.c:108: UPDATE vars SET intval = intval + 1 WHERE name = 'data_version' AND intval = ?: attempt to write a readonly database
lightningd-1 2025-03-16T23:29:58.941Z **BROKEN** lightningd: FATAL SIGNAL 6 (version f91eb11-modded)
{'github_repository': 'ElementsProject/lightning', 'github_sha': 'f91eb118388dc1455c67d16079168fd0267ba248', 'github_ref': 'refs/pull/8112/merge', 'github_ref_name': 'HEAD', 'github_run_id': 13888173501, 'github_head_ref': 'flake-memleak', 'github_run_number': 12573, 'github_base_ref': 'master', 'github_run_attempt': '1', 'testname': 'test_setconfig_access', 'start_time': 1742167762, 'end_time': 1742167800, 'outcome': 'fail'}
----------------------------- Captured stderr call -----------------------------
Error executing statement: db/exec.c:108: UPDATE vars SET intval = intval + 1 WHERE name = 'data_version' AND intval = ?: attempt to write a readonly database
lightningd: FATAL SIGNAL 6 (version f91eb11-modded)
Log dumped in /tmp/lightning-crash.log.20250316232958
Lost connection to the RPC socket.Lost connection to the RPC socket.
=========================== short test summary info ============================
FAILED tests/test_misc.py::test_setconfig_access - AssertionError: Regex pattern did not match.
Regex: 'Cannot write to config file /tmp/ltests-22a5dz1j/test_setconfig_access_1/lightning-1/regtest/config'
Input: "RPC call failed: method: check, payload: {'command_to_check': 'setconfig', 'config': 'min-capacity-sat', 'val': 1000000}, error: Connection to RPC server lost."
ERROR tests/test_misc.py::test_setconfig_access - ValueError:
Node errors:
- lightningd-1: had BROKEN messages
- lightningd-1: Node exited with return code -6
```
So we move the db file for sqlite.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Unfortunately a spec typo means the data fields are missing (PR pending),
so we still patch those in.
The message "your_peer_storage" got renamed to "peer_storage_retrieval",
and the option "want_peer_backup_storage" was removed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: `experimental-peer-storage` now only advertizes feature 43, not 41.
The old blindedpay fields would have an entry per hop, but that was changed
to a single per-path entry. The devtools hadn't caught up.
Similarly, it didn't like descriptionless offer fields in invreqs and invoices.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
