If we don't have an accountdb from bookkeeper:
1. Generate a deposit chain event for every confirmed UTXO.
2. Generate an open chain event for every open, confirmed channel.
3. Generate a push/lease event if necessary.
4. Generate a fixup "journal" entry if balance is different from initial.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We take over the --bookkeeper-dir and --bookkeeper-db options, and
then if we can find the bookkeeper db we extract the records to
initialize our chain_moves and channel_moves tables.
Of course, bookkeeper now needs to not register those options.
When bookkeeper gets invoked the first time, it will reconstruct
everything from listchannelmoves and listcoinmoves. It cannot
preserve manually-added descriptions, so we put those in the datastore
for it ready to go.
Note that the order of onchain_fee changes slightly from the original.
But this is fine.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is reliable, meaning we should never get replayed events.
We have to reference count to make sure all commands are complete,
before we return. In particular, annotating with descriptions can
involve several calls to list commands. We need to give them the
results *after* this is all complete.
test_bookkeeping_descriptions() relied on log messages from
notifications, which now only happen when a command is called. This
changes the test a bit.
Since we no longer subscribe to the balance_snapshot event, we
need to create the wallet account at initialization, as callers
expect it to exist.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Before bkpr_listaccountevents() gave entries with origin like:
{'account': "nifty's secret stash",
'blockheight': 111,
'credit_msat': 180000000,
'currency': 'bcrt',
'debit_msat': 0,
'origin': 'null',
'outpoint': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:0',
'tag': 'deposit',
'timestamp': 1679955976,
'type': 'chain'},
Changelog-Changed: Plugins: "utxo_deposit" is allows to have missing `transfer_from`, and null is not considered an account name.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Python's assert gives great analysis of what the differences are,
making debugging much easier.
So feed it dicts, not tuples, and simply do an assert.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that these migrations were inserted for v0.12, so only someone
upgrading directly from before that (2022-08-23) would be affected.
This avoids having to fix the migrations as we make changes.
We are going to mangle the db to allow testing, but then the final
step will be to migrate it to the core.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is inspired by a patch from @whitslack, which overlapped with this series.
Most importantly, there was only one call to bitcoin_tx_simple_input_weight(),
and it is better to be explicit with that one.
This also changes our funder calculation to assume our own input is taproot,
which it is likely to be given we've defaulted to taproot for outputs for
change addresses since 23.08.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need one byte for the number of witness elements. Some callers added it themselves,
but it's always needed. So document and fix the callers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It might be nice to let the bookkeeper keep track of external accounts
as well as the internal onchain wallet? To this end, we add some new
custom notifications, which the bookkeeper will ingest and add to its
ledger.
Suggested-By: @chrisguida
Changelog-Added: PLUGINS: `bookkeeper` now listens for two custom events: `utxo_deposit` and `utxo_spend`. This allows for 3rd party plugins to send onchain coin events to the `bookkeeper`. See the new plugins/bkpr/README.md for details on how these work!
We still want to test non-anchor channels, as we still support them, but
we've made it non-experimental. To test non-anchor channels, we
use dev-force-features: -23.
Changelog-Added: Protocol: `option_anchors_zero_fee_htlc_tx` enabled, no longer experimental.
Changelog-Changed: Config: `experimental-anchors` now does nothing (it's enabled by default).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Header from folded patch 'fixup!_options__make_anchors_enabled_by_default,_ignore_experimental-anchors.patch':
fixup! options: make anchors enabled by default, ignore experimental-anchors.
update_count is just the count of the records for a tx. To calculate onchain fees
for an account we must sum all credits vs debits. We don't need to GROUP BY update_count
nor ORDER BY update_count since it is just a running index of updates to this tx.
Remove grouping by update_count which resulted in a crash due to bad arithmetic
caused by fee calculation returned rows not being consolidated.
Remove xfail.
We had a complete copy of the `db_provider` and associated classes in
teh `tests/` directory, causing them to drift apart. Consolidating on
one version makes this more maintainable.
Changelog-None
We stopped doing empty journal logs, so we no longer need to switch
our log severity based on whether or not an account exists.
Should make bookkeeper less chatty and remove noisy logs
Changelog-None
We were putting out a lot of empty journal entries. Let's
stop doing that.
Now the wallet balance stays uninitialized until/unless you have
funds in it.
Fixes#5672
Using single tuples in Python is ugly, so:
1. Rename wait_for_onchaind_tx to wait_for_onchaind_txs.
2. Make it take tuples explicitly.
3. Make wait_for_onchaind_tx a simpler wrapper/unwrapper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
With the next change (which, as a side-effect, speeds up listpeers),
we seem to hit a race in this test. The bookkeeper doesn't get to
process the final payment before the node is shutdown.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
test_bookkeeping_closing_trimmed_htlcs fails to find 'all outputs
resolved' occassionally, seems like it's because the
OUR_DELAYED_TO_WALLET doesn't make it into the mempool before we start
mining blocks?
So here make sure there's something in the mempool before before we
start making new blocks.
