Commit Graph

23 Commits

Author SHA1 Message Date
Rusty Russell
41ac9db7ea bookkeeper: remove database.
Changelog-Changed: Plugins: `bookkeeper` now uses the lightningd database, not "accounts.db".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
45e860ad58 bkpr: helpers to query sql plugin for chainmoves and channelmoves.
We're going to be using this instead of our internal db.

I also made json_out_obj() take the str arg, as it didn't and I
expected it to.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
c5e359c7ee bkpr: add in-mem & datastore storage for external blockheights.
We won't be able to "UPDATE chain_events", so keep a separate record
of these blockheights, and lookup that when the blockheight is 0.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
6dcc580612 bkpr: infrastructure to keep rebalance pairs in mem & datastore.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
e012a82c60 bookkeeper: infrastructure to keep descriptions (annotations) in mem and datastore.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
5dc88c9e84 bkpr: move onchain fees access into bkpr/onchain_fee.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
f72df6da9c bkpr: cache accounts table in memory.
When we don't have our own db, we're going to need to keep this information
in memory (and the datastore).  As a first step, simply cache it in memory
and still write through to the db.

This introduces some changes:
1. The account structures are not temporary, but in the hash table (so don't steal them).
2. test_forward_pad_fees_and_cltv assumed ordering, which was a latent bug.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
40c0d86a5c bkpr: move account db functions from recorder.c into account.c.
This will let us cache them gradually in account.c, and eventually
remove the database table.

We also change find_close_account() to find_close_account_name(),
and maybe_mark_account_onchain() into account_onchain_closeheight()
and account_update_closeheight();

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
41856dc6c0 bkpr: expose struct bkpr to outside bookkeeper.c.
We're going to add more members here, so we will start handing around
the whole thing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-08-19 13:37:50 +09:30
Rusty Russell
7acaccfb36 wallet: add channel_type field to db.
This was always the intent, but now we have to reconstruct from the
disparate fields.

This means `option_anchor_outputs` is now redundant.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 17:26:47 +09:30
Rusty Russell
eac8401f84 Makefile: separate bolt12 wireobjects
Most things don't want them, so don't link it into everything by default.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell
bcabb3825f Makefile: Revert ba7d4a8f6b (make-schema: don't include tools/fromschema.py in SHASUMS)
1. If the tool changes, you need to regenerate since the output may
   change.

2. This didn't just filter that out, ignored all but the first
   dependency, which made bisecting the bookkeeper plugin a nightmare:
   it didn't regenerate the .po file, causing random crashes.

If we want this, try $(filter-out tools/fromschema.py) instead.  But I
don't think we want that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-08 00:36:51 +03:00
Rusty Russell
305a238810 plugins/Makefile: put bitcoin/chainparams.o in PLUGIN_COMMON_OBJS since everyone needs it.
This was originally done as part of the bookkeeper introduction, but it deserves its
own patch (and that one didn't remove the bitcoin/chainparams.o from the individual
requirements lines).

Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-28 12:08:18 +09:30
niftynei
352b419755 bkpr: save invoice description data to the database and display it
It'll be really nice to be able to read description data about an
invoice, if we've got it!
2022-07-28 12:08:18 +09:30
niftynei
ee47715a1b bkpr: command to calculate some APYs/stats on channel routing fees
Computes some stats from the net routed fees etc

```
{
   "channel_apys": [
      {
         "account": "7de277250f8003c35378c1eb20aacf8fa6a69dd4bb22077266329a9ad812cd5d",
         "routed_out": "2000msat",
         "routed_in": "125100101msat",
         "lease_fee_paid": "0msat",
         "lease_fee_earned": "670000msat",
         "pushed_out": "0msat",
         "pushed_in": "0msat",
         "our_start_balance": "100670000msat",
         "channel_start_balance": "1100670000msat",
         "fees_out": "0msat",
         "fees_in": "10100101msat",
         "utilization_out": "0.0002%",
         "utilization_out_initial": "0.0020%",
         "utilization_in": "11.3658%",
         "utilization_in_initial": "12.5100%",
         "apy_out": "0.0000%",
         "apy_out_initial": "0.0000%",
         "apy_in": "3203.3924%",
         "apy_in_initial": "3525.8779%",
         "apy_total": "3203.3924%",
         "apy_total_initial": "3203.3924%",
         "apy_lease": "8.7014%"
      },
      {
         "account": "b71937a02eb7694d946c311297ca2da454057c5c3e97ac67500739cc2afbc0c5",
         "routed_out": "110000000msat",
         "routed_in": "0msat",
         "lease_fee_paid": "670000msat",
         "lease_fee_earned": "0msat",
         "pushed_out": "0msat",
         "pushed_in": "0msat",
         "our_start_balance": "4000000000msat",
         "channel_start_balance": "4100670000msat",
         "fees_out": "10100101msat",
         "fees_in": "0msat",
         "utilization_out": "2.6825%",
         "utilization_out_initial": "2.7500%",
         "utilization_in": "0.0000%",
         "utilization_in_initial": "0.0000%",
         "apy_out": "2579.4892%",
         "apy_out_initial": "2644.4084%",
         "apy_in": "0.0000%",
         "apy_in_initial": "0.0000%",
         "apy_total": "2579.4892%",
         "apy_total_initial": "2579.4892%"
      },
      {
         "account": "net",
         "routed_out": "110002000msat",
         "routed_in": "125100101msat",
         "lease_fee_paid": "670000msat",
         "lease_fee_earned": "670000msat",
         "pushed_out": "0msat",
         "pushed_in": "0msat",
         "our_start_balance": "4100670000msat",
         "channel_start_balance": "5201340000msat",
         "fees_out": "10100101msat",
         "fees_in": "10100101msat",
         "utilization_out": "2.1149%",
         "utilization_out_initial": "2.6825%",
         "utilization_in": "2.4052%",
         "utilization_in_initial": "11.3658%",
         "apy_out": "677.8788%",
         "apy_out_initial": "859.8297%",
         "apy_in": "677.8788%",
         "apy_in_initial": "3203.3924%",
         "apy_total": "1355.7575%",
         "apy_total_initial": "1355.7575%",
         "apy_lease": "0.2122%"
      }
   ]
}
```
2022-07-28 12:08:18 +09:30
niftynei
593f1e7365 bkpr: add a 'listincome' event
list the income/expense events for a node.
2022-07-28 12:08:18 +09:30
niftynei
462fa20c17 bkpr: move json_to functions to respective type files 2022-07-28 12:08:18 +09:30
niftynei
8089f246c1 bkpr: use tags not str for tag originations 2022-07-28 12:08:18 +09:30
niftynei
8039fde5ab bkpr: if we're missing info about an account, add in journal entry
There's two situations where we're missing info.

One is we get a 'channel_closed' event (but there's no 'channel_open')

The other is a balance_snapshot arrives with information about accounts
that doesn't match what's already on disk. (For some of these cases, we
may be missing 'channel_open' events..)

In the easy case (no channel_open missing), we just figure out what the
2022-07-28 12:08:18 +09:30
niftynei
351dc17e46 bkpr: add bookkeeper to PLUGINS list
This makes it start up automatically
2022-07-28 12:08:18 +09:30
niftynei
b08ccfec1e bookkeeper: initial crud (no tests) 2022-07-28 12:08:18 +09:30
niftynei
cd95d91ed5 bkpr-tests: first test of plugin bkpr database
First test of bookkeeper database, for just the migrations.
2022-07-28 12:08:18 +09:30
niftynei
fb951dbbd6 bkpr: first attempt at database code for accounting
A database scheme and first attempt at drivers for the bookkeeper
database.

Also moves bookkeeper plugin into its own subdirectory
2022-07-28 12:08:18 +09:30