davide/palladum-lightning
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity
Files
ea2f7607b8ff08c6312e09caab22e07f1d0853e7
palladum-lightning/wallet
History
Rusty Russell 9f2e6de69c wallet: fix migration speed for postgres. 
Testing a large db shows Postgres slowing down exponentially as
it inserts the channel_events.  Rather than updating the index in the
db every time, do it at the end, for spectacular speedup:

```
lightningd-1 2025-10-08T05:39:44.333Z INFO    lightningd: Creating database
lightningd-1 2025-10-08T05:39:47.581Z DEBUG   lightningd: Transferring 6166 chain_events
lightningd-1 2025-10-08T05:39:48.455Z DEBUG   lightningd: Transferring 1660043 channel_events
lightningd-1 2025-10-08T05:39:54.390Z INFO    lightningd: Inserted 103100/1660043 channel_events
lightningd-1 2025-10-08T05:40:04.390Z INFO    lightningd: Inserted 283280/1660043 channel_events
lightningd-1 2025-10-08T05:40:14.390Z INFO    lightningd: Inserted 464065/1660043 channel_events
lightningd-1 2025-10-08T05:40:24.390Z INFO    lightningd: Inserted 629559/1660043 channel_events
lightningd-1 2025-10-08T05:40:34.390Z INFO    lightningd: Inserted 800659/1660043 channel_events
lightningd-1 2025-10-08T05:40:44.390Z INFO    lightningd: Inserted 975433/1660043 channel_events
lightningd-1 2025-10-08T05:40:54.390Z INFO    lightningd: Inserted 1134719/1660043 channel_events
lightningd-1 2025-10-08T05:41:04.390Z INFO    lightningd: Inserted 1290549/1660043 channel_events
lightningd-1 2025-10-08T05:41:14.390Z INFO    lightningd: Inserted 1443304/1660043 channel_events
lightningd-1 2025-10-08T05:41:24.390Z INFO    lightningd: Inserted 1590013/1660043 channel_events
lightningd-1 2025-10-08T05:41:29.148Z INFO    lightningd: bookkeeper migration complete: migrated 6166 chainmoves, 1660043 channelmoves, 132481 descriptions
```

Now we complete the entire migration in 1 minute 45 seconds.

Thanks to @michael1101 for reporting this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: db: migration from v25.09 on a reasonable size account database could take almost infinite time.
2025-10-13 14:15:32 +10:30
..
test
wallet: unit test the migration dedup code.
2025-10-02 10:21:06 +09:30
.gitignore
reorganize .gitignore entries across subdirs.
2021-05-18 09:43:50 +09:30
account_migration.c
wallet: fix migration speed for postgres.
2025-10-13 14:15:32 +10:30
account_migration.h
lightningd: migrate events from bookkeeper at startup.
2025-08-19 13:37:50 +09:30
db.c
wallet: find and remove any duplicates from the bug we just fixed.
2025-10-02 10:21:06 +09:30
db.h
db: migrate old runes table to fix up id fields.
2023-08-31 05:33:15 +09:30
invoices.c
lightningd: migrate events from bookkeeper at startup.
2025-08-19 13:37:50 +09:30
invoices.h
lightningd: remove delexpiredinvoice
2024-03-25 15:02:35 +10:30
Makefile
wallet: unit test the migration dedup code.
2025-10-02 10:21:06 +09:30
psbt_fixup.c
wallet/psbt_fixup: routine to fix invalid PBSTs which modern libwally won't load.
2023-03-31 09:16:25 +10:30
psbt_fixup.h
wallet/psbt_fixup: routine to fix invalid PBSTs which modern libwally won't load.
2023-03-31 09:16:25 +10:30
reservation.c
common/utxo: use a real type for the UTXO, not a boolean is_p2sh.
2025-05-06 12:27:53 +09:30
txfilter.c
wallet: create htable to store scriptpubkeys for issued indexes.
2025-02-13 15:42:51 -06:00
txfilter.h
wallet: create htable to store scriptpubkeys for issued indexes.
2025-02-13 15:42:51 -06:00
wallet.c
wallet: find and remove any duplicates from the bug we just fixed.
2025-10-02 10:21:06 +09:30
wallet.h
wallet: find and remove any duplicates from the bug we just fixed.
2025-10-02 10:21:06 +09:30
walletrpc.c
wallet: record all coin movements into chain_moves or channel_moves tables.
2025-08-19 13:37:50 +09:30
walletrpc.h
wallet: save keytype when issuing new address.
2024-11-17 20:27:13 +10:30