Commit Graph

370 Commits

Author SHA1 Message Date
Rusty Russell
6dcf4e7bdd wallet: separate migration table into its own source file.
This will make it easier to share with the downgrade tool.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
378293d268 wallet: add optional sql and functions to downgrade db.
This will allow a downgrade tool to attempt to downgrade a node to the
previous version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-28 09:16:53 +10:30
Rusty Russell
ff433a17a8 lightningd: save funding PSBT to database if we're to withhold it.
Normally we don't care, but if we're withholding it, keep it around
so we can sign & broadcast later.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-19 07:23:39 +10:30
Rusty Russell
7ff0239f6f lightningd: db migration to clean up any pending payments where theres no htlc.
Changelog-Fixed: JSON-RPC: `listpays`/`listsendpays` erroneously left `pending` in xpay are cleaned up.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-13 21:19:12 +10:30
Rusty Russell
575c1a71ba lightningd: db infrastructure for network events.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-11-12 13:58:43 +10:30
Rusty Russell
6e5cb299dd global: remove unnecessary includes from C files.
Basically, `devtools/reduce-includes.sh */*.c`.

Build time from make clean (RUST=0) (includes building external libs):

Before:
	real    0m38.944000-40.416000(40.1131+/-0.4)s
	user    3m6.790000-17.159000(15.0571+/-2.8)s
	sys     0m35.304000-37.336000(36.8942+/-0.57)s
After:
	real    0m37.872000-39.974000(39.5466+/-0.59)s
	user    3m1.211000-14.968000(12.4556+/-3.9)s
	sys     0m35.008000-36.830000(36.4143+/-0.5)s

Build time after touch config.vars (RUST=0):

Before:
	real    0m19.831000-21.862000(21.5528+/-0.58)s
	user    2m15.361000-30.731000(28.4798+/-4.4)s
	sys     0m21.056000-22.339000(22.0346+/-0.35)s

After:
	real    0m18.384000-21.307000(20.8605+/-0.92)s
	user    2m5.585000-26.843000(23.6017+/-6.7)s
	sys     0m19.650000-22.003000(21.4943+/-0.69)s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-23 06:44:04 +10:30
Rusty Russell
fb973fea05 wallet: find and remove any duplicates from the bug we just fixed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-10-02 10:21:06 +09:30
Rusty Russell
52d69df10f lightningd: migrate events from bookkeeper at startup.
We take over the --bookkeeper-dir and --bookkeeper-db options, and
then if we can find the bookkeeper db we extract the records to
initialize our chain_moves and channel_moves tables.

Of course, bookkeeper now needs to not register those options.

When bookkeeper gets invoked the first time, it will reconstruct
everything from listchannelmoves and listcoinmoves.  It cannot
preserve manually-added descriptions, so we put those in the datastore
for it ready to go.

Note that the order of onchain_fee changes slightly from the original.
But this is fine.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
1958efdad6 db: support mode where we don't want to create the db.
And gracefully fail for this case.

There's no such thing for Postgres, but that's because dbs need to be
set up by the admin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
6eb079ce3a wallet: routines to append to chain_moves and channel_moves tables.
Note that bookkeeper de-duplicates chain_moves: we need to too!  So we add
an index to make this efficient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
37cde40630 wallet: chain_moves and channel_moves tables.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
bb4dda6f5a wallet: we can assume local_alias field is non-null.
We have a migration which ensures this, but then I discovered that did
*not* address channels without an SCID yet.  So fixed the migration, and
simpligied the code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 14:04:11 +09:30
Rusty Russell
639452ac77 bitcoin: have random_scid() function.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 14:04:11 +09:30
Rusty Russell
4abd0af3ec lightningd: save previous short_channel_ids during splice, and keep in db.
There can be any number of these, and it will be useful to allow
routing by older scids (when other nodes haven't seen our gossip, or
even before we *can* announce the new post-splice channel).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-14 14:04:11 +09:30
Dusty Daemon
a8ff8ce717 splice: Add field to DB to track if we’ve sent sigs
We used to use a check on the active psbt to see if our splice signature was in it — but now we need to generate the signature early.

So we have to add a field tracking if we’ve sent it and add it to the database, wire protocols, and inflight objects.
2025-08-14 04:12:17 +09:30
Rusty Russell
e051ba6bbf sqlite3: consistently comment where we want to upgrade.
Now you can grep for 'sqlite3 version' and see where we would like
to update.

Debian 11 (Bullseye) and Ubuntu 20.04 (Focal) ship with SQLite 3.31.1.
RHEL 9 ships with 3.34.1.  Fedora 38+ uses SQLite 3.40+.

Unfortunately, RHEL8 ships with 3.26.0, and is still on maintenance Support
(security fixes, no new features): runs until May 31, 2029.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-11 11:08:43 +09:30
Rusty Russell
04ba55d88b wallet: fix incorrect channel_htlcs id migration.
We were supposed to initialize `last_htlcs_created_index` but a typo
made us initialize `last_forwards_created_index` instead.  Fix up the
overwritten one, and actually initialize the one we were supposed to.

Fixes: https://github.com/ElementsProject/lightning/issues/8269
Fixes: https://github.com/ElementsProject/lightning/issues/8270
Changelog-None: recently introduced bug
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-17 13:05:04 -05:00
Dusty Daemon
2bd39f270e splice: Add locked field to inflight db
This is needed to remember if a splice was locked and reconnect occurs mid `splice_locked` attempted so it can be resumed in reestablish.
2025-05-13 14:52:15 +09:30
Rusty Russell
c0ddfa23cc wallet: make enum wallet_output_type UPPERCASE.
No code change, just following convention.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-05-06 12:27:53 +09:30
Rusty Russell
186a9383da wallet: add updated_index to channel_htlcs table.
This prepares us for doing listhtlcs pagination.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-04-29 09:38:20 +09:30
Vincenzo Palazzo
7198fec853 db: replace UPDATE FROM syntax for SQLite compat
Introduced the use of UPDATE FROM syntax in SQLite queries,
which is not supported in versions prior to 3.33.0.

This causes issues on systems with older SQLite versions,
 as reported in issue #8231. Rewrite the query in
 migrate_convert_old_channel_keyidx() to use a subquery
 with IN clause instead of UPDATE FROM, ensuring compatibility with
 older SQLite versions.

Changelog-Fixed: db: replace UPDATE FROM syntax for SQLite compat
Fixes 68f3649d6b
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2025-04-23 09:53:15 +09:30
Rusty Russell
7c6270d051 lightningd: rescan for missing p2wkph for closed channels.
This can happen with 24.11 and later.  We scan back to exposed channel
opens, or that release.

The BROKEN log messages cause some tests to fail, so we fix those.

Fixes: https://github.com/ElementsProject/lightning/issues/8169
Changelog-Fixed: wallet: rescan for missing close outputs (can happen if peer doesn't support option_shutdown_anysegwit)
2025-04-02 11:22:54 +10:30
Rusty Russell
68f3649d6b wallet: do change db to watch both p2tr and p2wkph for not-yet-closed channels.
It's a bit more work to watch multiple addresses, but that's a small
price to pay for each channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-04-02 11:22:54 +10:30
Rusty Russell
1cbc96f0d9 lightningd: remember when we started closing channel.
This lets us maintain a reasonable 2 week target for commitment tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-26 14:49:36 +10:30
Rusty Russell
68feb55dbf wallet: save last known address.
If we connected out, remember that address.  We always remember the last
address, but that may be an incoming address.  This is explicitly the last
outgoing address which worked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-25 15:39:13 +10:30
Dusty Daemon
560ca00a2b splice: Add remote_funding to database
Enable storing the remote funding pubkey in DB if the channel peer decides to change it during splicing. It needs to be in DB incase of restarts mid-splice.

Changelog-None
2024-11-21 14:15:36 +10:30
ShahanaFarooqui
bb252983c2 wallet: insert addrtype ADDR_ALL for issued addresses 2024-11-17 20:27:13 +10:30
Rusty Russell
6da97e6461 wallet: save keytype when issuing new address.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 20:27:13 +10:30
Rusty Russell
20a41fd798 lightningd: only store channel funding spend txs into db.
Now we do replay, we don't need the others.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-13 14:44:03 +10:30
Rusty Russell
7d3d763b96 channeld: BOLT update, no longer allow creation of old (experimental-only!) non-zero-fee anchor channels.
These were removed from the spec.

We still support existing ones, though we were the only implementation
which ever did, and only in experimental mode, so we should be able to
upgrade them and avoid a forced close, with a bit of engineering...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30
Rusty Russell
b6d7ee1f11 common: No longer support new channels without option_static_remotekey.
We still support *existing* channels.  Just not new ones (before they could,
in theory, explicitly ask for one).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30
Rusty Russell
9450d46db1 bitcoin/short_channel_id: pass by copy everywhere.
It's a u64, we should pass by copy.  This is a big sweeping change,
but mainly mechanical (change one, compile, fix breakage, repeat).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 13:51:48 +10:30
Rusty Russell
b5bd907245 wallet: ensure all established channels have aliases.
Commit dac8964093 set aliases for channels at
creation time, but neglected to convert channels in the database.  Do that now!

Fixes: #7039
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-08 14:15:57 +01:00
Rusty Russell
0575f8a544 lightningd: add last_stable_connection field to db, channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Alex Myers
c1beb4b745 db: add private channel inbound fields
This will be used for generating route hints rather than private gossip
2023-12-14 09:16:56 +10:30
Dusty Daemon
a6a9e5b1e3 splice: Reestablish when commit or sig sends fail
Adds tests for when the connection fails during
1) splice tx_signature
2) splice commitment_signed

Fleshed out the reestablish flow for these two cases and implemented the fixes to make these reestablish flows work.

Part of this work required changing commit process for splices: Now we send a single commit_part for the splice where previously we sent all commits, and accordingly, we no longer revoke in response.

Changelog-Fixed: Implemented splicing restart logic for tx_signature and commitment_signed. Splice commitments are reworked in a manner incompatible with the last version.
2023-11-20 07:35:22 +01:00
Rusty Russell
e58ae31947 lightningd: hook forwards into the wait system.
This table doesn't have `id`, except as the implicit one in Sqlite3,
so we need to add it for postgres.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell
896cb61c8c db: initialize variable to hold max payments id.
We generalize the invoice routine, though it's (known) buggy, so we have
to copy it.  We rename the invoice routine to a more specific name though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell
ccb8d4b687 sendpays: add to wait subsystem.
Adding an index means:

1. Add the new subsystem, and new updated_index field to the db, and
   create xxx_index_deleted/created/updated APIs.
2. Hook up these functions to the points they need to be called.
3. Add index, start and limit fields to the list command.
4. Add created_index and updated_index into the list command.

This does #1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell
be96d33461 lightningd: routines to deal with peer commitment anchor info in db.
Store enough to know how to CPFP the peer's commitment tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Chris Guida
6b764c0e7c wallet: add invoice_fallbacks table and paid_outpoint to invoices 2023-10-26 19:11:17 +10:30
ShahanaFarooqui
d9e3d3e6d1 runes: bug fix for db failure to upgrade due to runes table alteration #6770
While rebasing the PR for per rune restriction, I unintentionally merged the
`{SQL("ALTER TABLE runes ADD last_used_nsec BIGINT DEFAULT NULL"), NULL}` database
alteration command ahead of {NULL, migrate_runes_idfix} (commit eacf0b502c (diff-1abcdf1b9d822b30079d6450b790274bdfb7c7fa04baa43ad2d9bd449865d4c9R978)).

`migrate_runes_idfix` was the 234th change (deployed with version 23.08.1)
and adding the `last_used_nsec` column should have been the next
(235th, added in current release) change. Due to this incorrect ordering,
nodes updating from version 23.08.1 to the master branch will not add the
`last_used_nsec` column as they should, and instead execute `migrate_runes_idfix`
again, leading to the error in issue #6770.

After the reordering, db_get_runes method also has to be fixed for only
selecting rune NOT last_used_nsec. Because this column was added after
`migrate_runes_idfix` calls it. I am tempted to change the method name from
`db_get_runes` to `db_migrate_runes` for more clarity on its functionality though.

Changelog-None.
2023-10-23 15:53:30 +10:30
ShahanaFarooqui
9561094433 db: removed deprecated runes_uniqueid from vars table
Fixes: #6696

Changelog-Fixed: rune: use runes table `id` instead `runes_uniqueid` from `vars` because it returns incorrect unique id if rune/s migrated from datastore.
2023-10-03 08:56:53 +10:30
Rusty Russell
1ecf31bae7 db: remove #if DEVELOPER in favor of runtime flag inside db struct.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
ShahanaFarooqui
eacf0b502c wallet: add last_used_nsec in runes table
We will use this for better rate limiting
2023-09-21 13:31:34 +09:30
Rusty Russell
b88583e346 db: migrate old runes table to fix up id fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-31 05:33:15 +09:30
Dusty Daemon
4628e3ace8 channeld: Code to implement splicing
Update the lightningd <-> channeld interface with lots of new commands to needed to facilitate spicing.

Implement the channeld splicing protocol leveraging the interactivetx protocol.

Implement lightningd’s channel_control to support channeld in its splicing efforts.

Changelog-Added: Added the features to enable splicing & resizing of active channels.
2023-07-31 21:00:22 +09:30
Dustin Dettmer
6b81243f96 splicing: add DB details for splice HTLCs
Changelog-None
2023-07-31 21:00:22 +09:30
Rusty Russell
97af40c545 pytest: fix the invoice typo migration if you ran and created both.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-31 13:25:22 +09:30
Rusty Russell
435a4623ad wallet: fix migration of existing invoices's id variable.
Rusty can't type.  Rusty can't test.  Bad Rusty.  Bad.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-27 15:54:08 +09:30