Commit Graph

479 Commits

Author SHA1 Message Date
Rusty Russell
5d5741e681 libplugin: correctly wrap notifications we send in the notification name.
All the core notifications changed over to wrapping the notification
fields in an object with the name of the notification, but notifications
from plugins were missed.

Changelog-Added: Plugins: `channel_hint_update`, `pay_failure` and `pay_success` notifications now have objects of the same name containing the expected fields.
Changelog-Deprecated: Plugins: `channel_hint_update`, `pay_failure` and `pay_success` notification fields outside the same-named object.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-18 10:01:07 +09:30
Rusty Russell
16819f345d lightningd: make notifications from plugins just like native ones.
Rather than forcing them to wrap their parameters in a "payload"
sub-object, copy in params directly.  We include the "origin" field
one level up, if they care.

The next patch restores compatibility for the one place we currently use
them, which is the pay plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: pyln-client: plugin custom notifications origins and payload (use parameters directly)
2025-08-18 10:01:07 +09:30
Rusty Russell
fc5edea999 pyln-client: adapt for modern plugin notifications.
For older lightningd, we copy field into the raw dict, for newer we recreate the old
"payload" member.

We do fix up the custom_notification test which set params to a string instead of a dict:
that's just weird!

We also change the hacky parsing to proper dict extraction.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: pyln-client: plugin notifications parameters now exposed directly, not wrapped in `params` object.
2025-08-18 10:01:07 +09:30
Rusty Russell
a686bda4cb lightningd: deprecate null short_channel_id and unknown old_state in channel_state_changed notification
We always prefer to omit fields rather than use 'null' (or unknown!).

Note that before this, the schema was broken, so we have to put a special
exemption in for that case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-18 10:01:07 +09:30
Rusty Russell
3015bfcf4b pytest: simple test for pay's custom notifications.
Particularly important since we're going to update the format: this makes sure we don't break them!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-18 10:01:07 +09:30
Rusty Russell
733bdfa67d lightningd: add channel_type to openchannel & openchannel2 hooks.
An obvious omission!

Changelog-Added: Plugins: `openchannel` and `openchannel2` hooks now expose the `channel_type` field for the offered channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-15 11:11:39 +09:30
Peter Neuroth
dbbb113123 tests: Add tests for extra_tlvs in hook
Adds some testcases for custom tlvs, set by a htlc_accepted_hook. We
check that the custom tlvs replace the update_add_htlc_tlvs and get
forwarded to the peer. We also check that a malformed tlv will result in
a **BROKEN** behaviour.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2025-08-14 18:57:05 +09:30
ShahanaFarooqui
bab6429da0 tests: add sql json function tests 2025-08-13 15:52:53 +09:30
Rusty Russell
50e4aabe18 commando: remove old commando rune commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: `commando-rune`, `commando-listrunes`, `commando-blacklist` (deprecated v23.08, disabled v25.05).
2025-06-25 09:06:31 +09:30
Rusty Russell
b7e20b327c lightningd: enable peer storage by default,
Now we've make it only on existing channels, and not have to call
listdatastore every time, that means we can safely turn it on by
default.

Changelog-Added: Protocol: we now offer peer storage to any peers who create a channel.
Changelog-Deprecated: Config: `--experimental-peer-storage` (it's now the default).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
dc2c3668e6 chanbackup: use local memory to store peer backups.
By keeping a local hash table, we won't have to look up every time.
We still write to the datastore when it changes, and we need to
initialize it at plugin start.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
adf7663fef chanbackup: only store backups for peers with current/previous channels.
This seems fair.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-16 23:39:35 +09:30
Rusty Russell
bc4efc2d02 lightningd: pass all log lines to warning notifiers.
It would be wrong to omit those prior to the last, and I want to see
what test this was supposedly breaking...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-14 17:12:47 +09:30
Rusty Russell
c81ec4fe76 lightningd: deprecate (undocumented!) "unknown" old_state field in channel_state_changed notification.
Reported-by: daywalker90
Changelog-Deprecated: JSON-RPC: channel_state_changed notification field `old_state` value "unknown" (it will be omitted, instead)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-08 10:49:31 +09:30
Rusty Russell
7ba6263c48 lightningd: add created_index and updated_index to listhtlcs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listhtlcs` has `created_index` and `updated_index` fields.
2025-04-29 09:38:20 +09:30
Rusty Russell
0e7615b1b7 plugins/topology: remove local channels from listchannels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: RPC `listchannels` no longer includes private local channels (deprecated v23.08, disabled by default in v24.11).
2025-03-12 09:26:08 +10:30
Rusty Russell
73fc9b0c2a plugins: all plugins must now support non-numeric JSON RPC id fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: plugins which didn't accept string JSON RPC fields (deprecated v23.08, disabled by default in v24.11).
2025-03-12 09:26:08 +10:30
Rusty Russell
f95b542c1e plugins: no longer accept 0/1 for boolean options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: allowing 0/1 instead of false/true for plugin options (deprecated v23.08, disabled by default in v24.11).
2025-03-12 09:26:08 +10:30
Rusty Russell
371965cec3 Makefile: update CLN_NEXT_VERSION.
We now have to explicitly enable various deprecated commando commands, and now
when deprecations are disabled, we honour missing MPP option in bolt12 invoices.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-03-12 09:26:08 +10:30
Rusty Russell
ba8b5a6636 pytest: don't need allow-deprecated-apis in test_commando/test_commando_stress any more.
In d18f564324 "pytest: stop using
deprecated commando_rune commands." we stopped using deprecated commands in these tests,
but we didn't remove the 'allow-deprecated-apis' flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-03-12 09:26:08 +10:30
Rusty Russell
a6e476a73b pytest: test sql deprecated field (now we have one!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-21 17:03:36 -06:00
Lagrang3
fe4d5036c2 add max_htlc_value_in_flight_msat to listpeerchan
Changelog-Added: JSON-RPC: `listpeerchannels` new output fields `their_max_total_htlc_out_msat` and `our_max_total_htlc_out_msat` as the value of `max_htlc_value_in_flight` (as of BOLT02) set by the local and remote nodes on channel creation.

Changelog-Deprecated: JSON-RPC: `listpeerchannels` value `max_total_htlc_in_msat`: use `our_max_total_htlc_out_msat` instead to follow spec naming convention.
2025-02-21 17:03:36 -06:00
Rusty Russell
2b3b7fde2c pytest: expect complaints when we use deprecated commando-runes commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-13 21:15:45 -06:00
Rusty Russell
d18f564324 pytest: stop using deprecated commando_rune commands.
These are about to start logging warnings, so use modern versions for tests
which aren't explicitly about testing obsolete ones.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-13 21:15:45 -06:00
Alex Myers
41b05e14b7 pytest: update test_plugin_slowinit to match longer timeout 2025-02-11 08:40:29 -06:00
Rusty Russell
3b16637c27 pytest: fix race in test_autoclean
We can actually delete it before counters are updated:

```
        wait_for(lambda: len(l3.rpc.listinvoices()['invoices']) == 2)
>       assert l3.rpc.autoclean_status()['autoclean']['expiredinvoices']['cleaned'] == 3
E       assert 1 == 3

tests/test_plugin.py:3266: AssertionError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 08:40:29 -06:00
Rusty Russell
c4f8716a55 pytest: fix race in test_autoclean.
We can see the log message about cleanup just after the test ends.

```
>       assert l3.rpc.autoclean_status()['autoclean']['expiredinvoices']['cleaned'] == 1
E       assert 0 == 1

tests/test_plugin.py:3232: AssertionError
```
...
```

lightningd-3 2024-11-18T07:52:55.402Z INFO    lightningd: setconfig: autoclean-cycle 10 (updated /tmp/ltests-ao0p8pem/test_autoclean_1/lightning-3/regtest/config:4)
...
lightningd-3 2024-11-18T07:52:59.747Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-connectd: peer_out WIRE_QUERY_CHANNEL_RANGE
lightningd-3 2024-11-18T07:52:59.747Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-gossipd: reply_channel_range 0+109 (of 0+109) 2 scids
lightningd-3 2024-11-18T07:52:59.747Z DEBUG   gossipd: seeker: state = NORMAL No unannounced nodes
{'github_repository': 'ElementsProject/lightning', 'github_sha': '0729de783e95c5208b1706f7d27b23904596bb71', 'github_ref': 'refs/pull/7835/merge', 'github_ref_name': 'HEAD', 'github_run_id': 11887300979, 'github_head_ref': 'guilt/fix-flakes8', 'github_run_number': 11566, 'github_base_ref': 'master', 'github_run_attempt': '1', 'testname': 'test_autoclean', 'start_time': 1731916359, 'end_time': 1731916385, 'outcome': 'fail'}
--------------------------- Captured stdout teardown ---------------------------
lightningd-3 2024-11-18T07:53:05.503Z DEBUG   plugin-autoclean: cleaned 1 from expiredinvoices
lightningd-3 2024-11-18T07:53:05.503Z DEBUG   plugin-autoclean: setting next timer
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-19 17:51:18 +10:30
Michael Schmoock
e79a275209 pytest: fix a test that broke because of docstring usage 2024-11-19 11:50:42 +10:30
Rusty Russell
0cc52bc281 pytest: don't set experimental-offers in tests: it's the default now.
And about to be deprecated.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 10:42:54 +01:00
Rusty Russell
c93153ec37 lightningd: allow builtin plugins to be stopped.
These are automatically marked "important", in the sense that we won't startup
if they are not working, but this wasn't meant to disallow stopping them.

Changelog-Changed: JSON-RPC: built-in plugins can now be stopped using "plugin stop".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
c79a89d557 pytest: adapt tests to avoid deprecated APIs in close (tx and txid).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 16:04:06 +10:30
Rusty Russell
fcebb33180 lightningd: deprecate decodepay.
It only works on BOLT11, and has long been replaced by the more
generic "decode".

Removing it will stop the confusion!

(Note: documentation claims it was introduced in 23.08, but that was
 wrong, as it's been in CLN since the beginning).

[ Fixup from: niftynei <niftynei@gmail.com> ]
Fixes: https://github.com/ElementsProject/lightning/issues/6419
Changelog-Deprecated: JSON-RPC: `decodepay`: use `decode`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:21:23 +10:30
Rusty Russell
4f5865167e pytest: fix spurious error in test_important_plugin
```
**BROKEN** plugin-spenderp: Lost connection to the RPC socket.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
Rusty Russell
ecc3a00839 pytest: don't try exposesecret test under valgrind.
There's a known issue with hsm passwords and valgrind:

```
        write_all(master_fd, (password + '\n').encode("utf-8"))
>       l1.daemon.wait_for_log("Server started with public key")

tests/test_plugin.py:4526: 
...
                        if self.is_in_log(r):
                            print("({} was previously in logs!)".format(r))
>                   raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E                   TimeoutError: Unable to find "[re.compile('Server started with public key')]" in logs.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
Rusty Russell
8293352425 config: add the ability for plugins to specify that config values should be concealed.
And use it for `exposesecret-passphrase`.  This is probably overly
cautious, but it makes me feel a little better that we won't leak it
to someone with read-only access.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-13 08:49:55 +10:30
Rusty Russell
e261301301 exposesecret: new plugin and command to get hsm_secret.
Being able to back up the hsm_secret is critical, but you cannot do
this through a UI, because of course we do not allow such access.
People have lost funds because they didn't back up.

This allows access to the hsm_secret if you use a password set in the
config file.  (If it's not set, the command does not work).  This is a
compromise, of course.

Changelog-Added: `exposesecret` command for encouraging hsm_secret backups.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-13 08:49:55 +10:30
niftynei
a61b7ef347 bkpr: add new json RPC "bkpr-editdescriptionbypaymentid"
This takes an {payment_id} and {description}.
It looks for all chain + channel events that match
that {payment_id} and updates the description for those events.

We return all the updated events. If no events are updated, an empty
list is returned.

Changelog-Added: PLUGINS: bookkeeper has a new RPC `bkpr-editdescriptionbypaymentid` which will update the description for any event with matching payment_id
2024-11-12 09:46:09 +10:30
jackstar12
0b5944708e tests: adjust cln-grpc tests for new default start 2024-11-12 09:43:54 +10:30
Se7enZ
f9e28b9bfa keysend: Add maxfee to keysend for consistency with pay. ([#7227])
Changelog-Added: keysend: Add `maxfee` to keysend for consistency with pay. ([#7227])
2024-10-14 11:58:00 +02:00
ShahanaFarooqui
3d3e86e22c test: CI error fixes
- Removes CI value error for Broken logs
- Fixes CI errors due to deprecated listconfigs 'important-plugins'
- Removed listchannels deprecated local test

Changelog-None.
2024-09-18 16:59:27 +09:30
Rusty Russell
f0cf088489 lightningd: don't crash if plugin dies during initial plugins_init.
Thanks to Michael Schmook for the excellent bug report.

Fixes: https://github.com/ElementsProject/lightning/issues/7671
Changelog-Fixed: lightningd: no longer crash if a plugin dies during lightningd startup.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 10:24:01 -07:00
Rusty Russell
f4f6279b17 pytest: thoroughly test plugin death during startup.
We modify broken.py to be able to fail at different points,
and test that during startup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 10:24:01 -07:00
Rusty Russell
bfe0557f0b topology: fix overzealous deprecation check in listpeerchannels.
It's an output field (which we don't complain about), not an input field!

Fixes: https://github.com/ElementsProject/lightning/issues/7652
Changelog-Fixed: Logging: removed bogus "**BROKEN** plugin-topology: DEPRECATED API USED: listchannels.include_private" message.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-16 16:03:48 +09:30
Rusty Russell
7e2ea3ed04 pytest: add test for bogus deprecation BROKEN log.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-16 16:03:48 +09:30
Rusty Russell
054eeb8aa2 plugins/sql: fix crash when subobject is missing which has subobject inside it.
There's only one place (added in v24.02) where we have nested subobjects, and
if the outer sub-object is not there it crashes.

```
sql: FATAL SIGNAL 11 (version v24.08rc1-2-ge134f2f-modded)
0x5781d447239b send_backtrace
        common/daemon.c:33
0x5781d4472432 crashdump
        common/daemon.c:75
0x787bcd64531f ???
        ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x5781d447782a json_get_membern
        common/json_parse_simple.c:210
0x5781d44778cc json_get_member
        common/json_parse_simple.c:223
0x5781d445d0e2 process_json_obj
        plugins/sql.c:537
0x5781d445d10d process_json_obj
        plugins/sql.c:538
0x5781d445d598 process_json_list
        plugins/sql.c:684
0x5781d445d65a process_json_result
        plugins/sql.c:699
0x5781d445d7cb default_list_done
        plugins/sql.c:722
0x5781d446349d handle_rpc_reply
        plugins/libplugin.c:1016
0x5781d4463640 rpc_read_response_one
        plugins/libplugin.c:1202
0x5781d44636f1 rpc_conn_read_response
        plugins/libplugin.c:1226
0x5781d459e56c next_plan
        ccan/ccan/io/io.c:60
0x5781d459ea3d do_plan
        ccan/ccan/io/io.c:422
0x5781d459eafa io_ready
        ccan/ccan/io/io.c:439
0x5781d45a0469 io_loop
        ccan/ccan/io/poll.c:455
0x5781d4463dcb plugin_main
        plugins/libplugin.c:2230
```

Changelog-Fixed: Plugins: `sql` crash on querying `listpeerchannels` during channel establishment.
Fixes: https://github.com/ElementsProject/lightning/issues/7505
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-16 07:19:52 +09:30
Rusty Russell
e2ec60a369 pytest: handle more expected broken messages.
Once we fix our broken log detection, we find some we were missing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-15 16:38:12 +09:30
Christian Decker
d496126af8 tests: Adjust tests for the lack of unilateral attempt now 2024-07-29 17:08:33 +02:00
Rusty Russell
d7fa88b686 lightningd: clarify bolt12 "payment secret" logic.
We don't use the payment_secret in bolt12, but in onion_decode() we
do put the path_secret (if of correct length) into payment_secret.  Not
realizing this confused me, so document that, and make sure we insist
on it being present for bolt12.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-09 15:09:29 +02:00
Rusty Russell
1eb1db0e24 pyln-client: allow dynamic option setter to throw exceptions
And don't set the value unless it passes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 17:20:06 +09:30
Christian Decker
2737f2e97e pyln: Add and test a callback based setconfig listener
I was looking into using the `threading.Condition` but since we're
already rather heavily using callbacks, this allows us to stay
single-threaded, and not having to completely hook the `setconfig`
function.

Changelog-Added: pyln-client: Added a notification mechanism for config changes
2024-06-20 17:20:06 +09:30