Lisa *told* me about this on review, and I ignored it. Fool: took an extra day to get the account.db which was triggering this so I could see the problem.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If they ran off master, currency can be null:
```
2025-08-21T10:03:04.566Z **BROKEN** lightningd: bookkeper migration: Accessing a null column e.currency/7 in query SELECT e.id, e.account_id, a.name, e.tag, e.credit, e.debit, e.fees, e.currency, e.payment_id, e.part_id, e.timestamp, e.ev_desc, e.rebalance_id FROM channel_events e LEFT OUTER JOIN accounts a ON a.id = e.account_id ORDER BY e.timestamp, e.id;
```
So allow this, but *also* check if it's a different currency and skip. This won't happen: you had to manually inject events in a different currency.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that this can only happen if you ran a master commit before rc1:
```
2025-08-21T10:03:03.255Z **BROKEN** lightningd: bookkeper migration: Accessing a null column e.ignored/15 in query SELECT e.id, e.account_id, a.name, e.origin, e.tag, e.credit, e.debit, e.output_value, e.currency, e.timestamp, e.blockheight, e.utxo_txid, e.outnum, e.spending_txid, e.payment_id, e.ignored, e.stealable, e.ev_desc, e.spliced, a.closed_count, a.peer_id, a.we_opened FROM chain_events e LEFT OUTER JOIN accounts a ON e.account_id = a.id ORDER BY e.timestamp, e.id;
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Prior to 23.05, we used this tag to mark onchain to-self inputs we didn't
wait for (because they were too small). This fixes migration if that happened
(and we are debating whether we should re-introduce this!).
```
lightningd: FATAL SIGNAL 6 (version v25.09rc2)
0x100c8683 send_backtrace
common/daemon.c:33
0x100c876f crashdump
common/daemon.c:78
0x7fffb2080493 ???
???:0
0x7fffb1ab0cac ???
__pthread_kill_implementation+0x1bc:0
0x7fffb1a48a5b ???
__GI_raise+0x2b:0
0x7fffb1a2a3db ???
__GI_abort+0x153:0
0x100935b7 migrate_from_account_db
wallet/account_migration.c:424
0x10093ff7 db_migrate
wallet/db.c:1139
0x10096763 db_setup
wallet/db.c:1185
0x100a1bcb wallet_new
wallet/wallet.c:223
0x1004485f main
lightningd/lightningd.c:1311
0x7fffb1a2aba3 ???
__libc_start_call_main+0x93:0
0x7fffb1a2adeb ???
__libc_start_main_alias_1+0x1ab:0
0xffffffffffffffff ???
???:0
lightningd: Died with signal 6
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/8484
Sorry :(
We chose to update our own few custom plugin notifications by manually creating the deprecated
fields and adding the new ones, rather than having lightningd fix them up. But this didn't apply
to other plugins which might issue their own notifications: in particular, this hit @daywalker90.
Simply documenting this is lazy, but we're close to release and I don't expect anyone else to
be affected.
Reported-by: @daywalker90
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
@nepet noted that Valgrind complained. Nobody really cares though?
TL;DR: if channel isn't enabled, estimate isn't set.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-None: CI only
Recent versions of urllib3 fail certificate verification if certificates
lack the Authority Key Identifier or Key Usages extensions:
```
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Missing Authority Key Identifier (_ssl.c:1032)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: CA cert does not include key usage extension (_ssl.c:1032)
```
Luckily, rcgen offers parameters in its CertificateParams structure to
add these extensions. Let's use them.
Changelog-Fixed: Certificates auto-generated by grpc-plugin, rest-plugin, and wss-proxy-plugin now include the required Authority Key Identifier and Key Usages extensions.
If you have run recent master, upgrade will fail with:
```
Cannot migrate account database version 18
```
The final migration is:
```
/* We used to send anchors to the wallet, but set ignored tag. Now we send
* them to external. */
{SQL("UPDATE chain_events"
" SET account_id = (SELECT id FROM accounts WHERE name = 'external')"
" WHERE account_id = (SELECT id FROM accounts WHERE name = 'wallet')"
" AND ignored = 1"), NULL},
```
Which is harmless (we do that upgrade ourselves on migration, but if it's done already that will have no effect).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes conflict error in Release action when trying to fetch the commit and tag both together. Like
```
fatal: Cannot fetch both da7d3057ed and refs/tags/v25.09rc1 to refs/tags/v25.09rc1
```
Changelog-None.
Fixes current error:
```
ERROR: Invalid requirement: 'Updating dependencies': Expected end or semicolon (after name and no valid version specifier)
```
Fixes error in current reproducible builds:
```
error: failed to parse lock file at: /build/Cargo.lock
Caused by:
lock file version `4` was found, but this version of Cargo does not understand this lock file, perhaps Cargo needs to be updated?
```
It was breaking a lot, due to Cargo dependencies not being published,
no surprise there, we are just publishing them with the changes it is
complaining about.
Commit ebaa25d9e2fd5582b7fe0e3ec482c1627996e4d7 introduced a couple of
breaking changes to the schema and proto files. The bump ensures
backwards compat for users that have indicated `~0.4` as their version
constraint.
Changelog-Changed: rust: New version of `cln-rpc==0.5` and `cln-grpc==0.5`
This requires us to turn "sql" calls into calls to a local db, which
means pulling in a lot of infrastructure. But it's possible.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If we don't have an accountdb from bookkeeper:
1. Generate a deposit chain event for every confirmed UTXO.
2. Generate an open chain event for every open, confirmed channel.
3. Generate a push/lease event if necessary.
4. Generate a fixup "journal" entry if balance is different from initial.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We take over the --bookkeeper-dir and --bookkeeper-db options, and
then if we can find the bookkeeper db we extract the records to
initialize our chain_moves and channel_moves tables.
Of course, bookkeeper now needs to not register those options.
When bookkeeper gets invoked the first time, it will reconstruct
everything from listchannelmoves and listcoinmoves. It cannot
preserve manually-added descriptions, so we put those in the datastore
for it ready to go.
Note that the order of onchain_fee changes slightly from the original.
But this is fine.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And gracefully fail for this case.
There's no such thing for Postgres, but that's because dbs need to be
set up by the admin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There will be no more missing events (and at initialization time, we will do
that as a migration).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
With some help (and hinderance!) from ChatGPT: the field names
differ slightly from our internal db.
The particilar wrinkle is that we have to restrict all queries to
limit them to entries we've seen already. Our code expects this (we
used to only enter it into the db when we processed it), and it would
otherwise be confusing if a sql query returned inconsistent results
because an event occurred while bookkeeper was processing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We're going to be using this instead of our internal db.
I also made json_out_obj() take the str arg, as it didn't and I
expected it to.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is reliable, meaning we should never get replayed events.
We have to reference count to make sure all commands are complete,
before we return. In particular, annotating with descriptions can
involve several calls to list commands. We need to give them the
results *after* this is all complete.
test_bookkeeping_descriptions() relied on log messages from
notifications, which now only happen when a command is called. This
changes the test a bit.
Since we no longer subscribe to the balance_snapshot event, we
need to create the wallet account at initialization, as callers
expect it to exist.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rearrange all the JSON interfaces to call refresh_moves() (async)
before doing anything.
This does nothing for now, but it will be useful once we transition
from notifications to using the list commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
