bookkeeper used to generate these as channel events, now lightningd does.
We also add a "journal" event, which we will need later too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We still output the fields, they're just always the currency of the node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `bookkeeper` now explicitly assumes every transaction is in the same currency as the node (true unless you added manually)
We're going to get rid of this concept, but the main change is that the
account_get_balance API can be drastically simplified:
account_get_credit_debit() accesses the raw fields, never fails, but
returns the a flag which tells us if the account doesn't actually have
any events.
The one place we care about the balance, calculate by hand. Then
account_get_balance() (and struct account_balance) can simply be
moved to th test.
Subtly, without the "GROUP BY" clause, you always get one row, even if
there are no rows (but the SUM are null).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means we can keep a pointer to the channel directly, *or* a string.
This avoids gratuitous formatting (on creation) and lookups (later).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
json_tok_streq(…) and json_get_member(…) are convenience wrappers for
json_tok_strneq(…) and json_get_membern(…) respectively. Unfortunately, using
them incurs a performance penalty in the common case where they are called with
a string literal argument because the compiler is unable to substitute a
compile-time constant in place of the buried call to strlen(…).
For example,
json_get_member(buf, tok, "example");
…will have worse performance than…
json_get_membern(buf, tok, "example", strlen("example"));
…because the former is forced to scan over "example" at run-time to count its
length whereas the latter is able to elide the strlen(…) call at compile time.
Hoist these convenience functions up into common/json_parse_simple.h and mark
them as inline so that the compiler can elide the strlen(…) call in the common
case of calling these functions with a string literal argument.
Changelog-None
This is much faster to give 64 bits of data, and we don't need
cryptographic randomness.
This brings us back to 413ns per trace.
Before:
real 0m5.819000-6.472000(6.2064+/-0.26)s
user 0m3.779000-4.101000(3.956+/-0.12)s
sys 0m2.040000-2.431000(2.2496+/-0.15)s
After:
real 0m3.981000-4.247000(4.1276+/-0.11)s
user 0m3.979000-4.245000(4.126+/-0.11)s
sys 0m0.000000-0.002000(0.001+/-0.00063)s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: lightingd: trimmed overhead of tracing infrastructure.
Otherwise it appears to be a leak:
==612637== 11,264 bytes in 1 blocks are still reachable in loss record 1 of 1
==612637== at 0x484D953: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==612637== by 0x1301F2: trace_init (trace.c:153)
==612637== by 0x13065D: trace_span_start (trace.c:263)
==612637== by 0x173968: db_open_ (utils.c:367)
==612637== by 0x17AE43: create_test_wallet (run-wallet.c:1313)
==612637== by 0x17C726: test_shachain_crud (run-wallet.c:1548)
==612637== by 0x18300E: main (run-wallet.c:2329)
Changelog-None
If we connected out, remember that address. We always remember the last
address, but that may be an incoming address. This is explicitly the last
outgoing address which worked.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The `send_outreq` function is a good place to suspend and resume
traces, since these are usually the places where we hand off control
back to the `io_loop`. This assumes that we do not continue doing
heavy liftin after we have queued an `outreq` call, but that is most
likely the case anyway. This frees us from having to track suspensions
whenever we call the RPC from a plugin.
I used `amount_msat_eq(x, AMOUNT_MSAT(0))` because I forgot this
function existed. I probably missed it because the name is surprising,
so add "is" in there to make it clear it's a boolean function.
You'll note almost all the places which did use it are Eduardo's and
Lisa's code, so maybe it's just me.
Fix up a few places which I could use it, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We do some fancy accounting for channel closures; since we're tagging
splice txs as closes we need to mark them as splices so we can treat them
as any other 'normal' on chain event.
Currently make a plugin that do reportings of logs on
a services like graphana is not possible. So this commit
include the possibility to write a plugin that do the report
of this analisys.
Changelog-Added: core: notify plugins when a log line is emitted.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
We don't thoroughly handle `check setconfig`: it would be good to
allow this to do further checking!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This has the benefit of being shorter, as well as more reliable (you
will get a link error if we can't print it, not a runtime one!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We stopped doing empty journal logs, so we no longer need to switch
our log severity based on whether or not an account exists.
Should make bookkeeper less chatty and remove noisy logs
Changelog-None
Also requires us to expose memleak when !DEVELOPER, however we only
ever used the memleak tracking when the LIGHTNINGD_DEV_MEMLEAK
environment variable was set, so keep that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids the mess where we override db_fatal for teqsts, and keeps it
generic.
Also allows us to get rid of one #if DEVELOPER, and an ugly global for
bookkeeper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Don't parse the listpeers.channels output ourselves: with two extra fields
we can simply reuse json_to_listpeers_channels().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
we weren't making records for 'missed' accounts that had a zero balance
at snapshot time (if peer opens channel and is unused)
Fixes: #5502
Reported-By: https://github.com/niftynei/cln-logmaid
Track rebalances, and report income events for them.
Previously `listincome` would report:
- invoice event, debit, outgoing channel
- invoice_fee event, debit, outgoing channel
- invoice event, credit, inbound channel
Now reports:
- rebalance_fee, debit, outgoing channel
(same value as invoice_fee above)
Note: only applies on channel events; if a rebalance falls to chain
we'll use the older style of accounting.
Changelog-None
We rely on sqlite3 being present to run unit tests for some bookkeeper
tests; here we effectively disable these tests if not available
Fixes report in #4928
Reported-By: @whitslack
The initial snapshots on an already-running lightningd are expected to
be unbalanced, but this shouldn't cause users to long for the green,
green grass of home.
This controls the Art of Noise.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Keep the accounts as an 'append only' log, instead we move the marker
for the 'channel_open' forward when a 'channel_open' comes out.
We also neatly hide the 'channel_proposed' events in 'inspect' if
there's a 'channel_open' for that same event.
If you call inspect before the 'channel_open' is confirmed, you'll see
the tag as 'channel_proposed', afterwards it shows up as
'channel_open'. However the event log rolls forward -- listaccountevents
will show the correct history of the proposal then open confirming (plus
any routing that happened before the channel confirmed).
If we expect further events for an onchain output (because we can steal
it away from the 'external'/rightful owner), we mark them.
This prevents us from marking a channel as 'onchain-resolved' before
all events that we're interested in have actually hit the chain.
Case that this matters:
Peer publishes a (cheating) unilateral close and a timeout htlc (which
we can steal).
We then steal the timeout htlc.
W/o the stealable flag, we'd have marked the channel as resolved when
the peer published the timeout htlc, which is incorrect as we're still
waiting for the resolution of that timeout htlc (b/c we *can* steal it).
it's nice to know what node your channel was opened with. in theory we
could use listpeers to merge the data after the fact, except that
channels disappear after they've been closed for a bit. it's better to
just save the info.
we print it out in `listbalances`, as that's a great place account level
information
Anchor outputs are ignored by the clightning wallet, but we keep track
of them in the bookkeeper. This causes problems when we do the balance
checks on restart w/ the balance_snapshot -- it results in us printing
out a journal_entry to 'get rid of' the anchors that the clightning node
doesnt know about.
Instead, we mark some outputs as 'ignored' and exclude these from our
account balance sums when we're comparing to the clightning snapshot.
onchain fees are weird at channel close because:
- you may be missing an trimmed htlc (which went to fees)
- the balance from close may have been rounded (msats cant land on
chain)
- the close might have been a past state and you've actually
ended up with more money onchain than you had in the channel. wut
This commit accounts for all of this appropriately, with some tests.
channel_close.debit should equal onchain_fee.credit (for that txid)
plus sum(chain_event.credit [wallet/channel_acct]).
In the penalty case, channel_close.debit becomes channel_close.debit +
penalty_adj.debit, i.e.
channel-close.debit + (penalty_adj.debit) =
onchain_fee.credit
+ sum(chain_event.credit [wallet/channel_acct])
There's two situations where we're missing info.
One is we get a 'channel_closed' event (but there's no 'channel_open')
The other is a balance_snapshot arrives with information about accounts
that doesn't match what's already on disk. (For some of these cases, we
may be missing 'channel_open' events..)
In the easy case (no channel_open missing), we just figure out what the
When we print events out, we need to know the account name. This makes
our lookup a lot easier, since we just pull it out from the database
every time we query for these.
