Add a periodic deposit reconciler, and stop losing subscribe tasks (B-30)

Deposits were credited exclusively by scripthash-change notifications,
with nothing re-verifying a user's balance against the chain if a
subscription was ever silently lost. address_for_new_user's subscribe
was fire-and-forget: the task wasn't retained, so it could be
garbage-collected mid-flight, and any failure (including self.client
turning None between the check and the task running) vanished into
asyncio's default unretrieved-exception handler instead of being
logged anywhere. On an otherwise healthy, long-lived connection there
may be no reconnect for days to re-subscribe everyone, so a user in
that state never saw their deposits.

address_for_new_user now retains the task and logs its exception if
it fails. New app/deposits/reconcile.py adds DepositReconciler, a
periodic sweep (every 5 minutes, gated on the Electrum client being
connected, same shape as tx/reconcile.py) that round-robins over every
user and calls the listener's own refresh_user (renamed from
_refresh_user since it's now called from outside the class) - so the
notification-driven and periodic paths can never behave differently.
Deliberately sweeps every user rather than only ones missing from the
in-memory scripthash map, since that map can't tell "never subscribed"
apart from "subscribed, but the server stopped delivering
notifications for it". Wired into app/main.py's lifespan alongside the
other three background reconcilers.

Suite grows from 176 to 182 tests. BUGS.md moves B-30 to Previously
fixed.
This commit is contained in:
davide committed 2026-07-27 10:35:23 +02:00
1 parent e8fdea0389
commit 63df38d30b
7 files changed
+255 -40

No files matched your search

+7 -27
View File
@@ -1,17 +1,16 @@
# Known bugs
A second full-codebase audit on 2026-07-27 found **25 further issues** (4 critical, 6 high,
7 medium, 8 low), listed below as B-30 … B-49. B-25 through B-29 are fixed (see "Previously
fixed" below) — no Critical-severity finding remains open; the other 20 are High/Medium/Low.
7 medium, 8 low), listed below as B-31 … B-49. B-25 through B-30 are fixed (see "Previously
fixed" below) — no Critical-severity finding remains open; the other 19 are High/Medium/Low.
The 139-test suite was green at the time of the audit, so none of these were caught by existing
coverage — every fix lands with a regression test (the five fixes so far brought the suite from
139 to 176).
coverage — every fix lands with a regression test (the six fixes so far brought the suite from
139 to 182).
The recurring pattern across the open findings is worth stating once: the code is rigorous
about the failure modes that have actually been hit, and silent about the ones that have not.
The payout phase is now fully recoverable; the "drawing" phase (waiting on a block) still has
no equivalent resume-after-restart or stall visibility (B-36), and there's still no periodic
deposit-side reconciler independent of scripthash notifications (B-30).
no equivalent resume-after-restart or stall visibility (B-36).
For limitations that are accepted by design rather than bugs (single-shared-token admin auth,
single-process assumptions, no user-facing history, etc.), see "Known gaps / TODO" in
@@ -21,26 +20,6 @@ single-process assumptions, no user-facing history, etc.), see "Known gaps / TOD
## High
### B-30 — No deposit-side reconciler: one missed subscription means deposits are never credited
`electrum/listener.py:61-67` (`address_for_new_user`) fires `asyncio.create_task(...)` without
retaining the reference and without handling exceptions. If `self.client` becomes `None`
between the check and the task running, the `assert` at `:163` raises inside an orphan task
and the exception is swallowed.
What makes this serious is what happens next: deposits are credited **exclusively** by
scripthash notifications. There is no periodic routine reconciling balances against the chain
(the reconciler only covers outgoing transactions). On a healthy keepalive'd connection there
are no reconnects, so a lost subscription is never recovered and that user **never sees their
deposits**, indefinitely.
**Proposed fix.** Two parts. (a) Make the subscription reliable: retain the task, log its
exceptions, and retry with backoff instead of relying on a reconnect. (b) Add the missing
safety net — a periodic sweep (say every few minutes, similar in shape to
`PendingTransactionReconciler`) that re-runs `_refresh_user` for users whose scripthash is not
in `_scripthash_to_user`, or simply round-robins over all users so a missed notification is
always eventually caught.
### B-31 — Reconnect costs O(users) sequential round-trips and stalls the draw
In `_run_once` the order is: subscribe headers → `_subscribe_all_users()` → *then* start the
@@ -285,9 +264,10 @@ already does.
- **B-27** — an RBF bump reset the reconciler's own abandon clock, so a repeatedly-bumped tx was never abandoned
- **B-28** — a hostile Electrum server (or a MITM) could single-handedly pick the round's winner
- **B-29** — a UTXO absent from one server's `listunspent` was marked spent immediately, irreversibly, on a single unauthenticated reply
- **B-30** — a lost scripthash subscription meant a user's deposits were never credited, with no periodic safety net
See git history for the fix-by-fix breakdown (commits `f13f685`, `50a43ae`, `933760e`, and the
B-28/B-29 fixes). Suite grew from 139 to 176 tests over the five.
B-28/B-29/B-30 fixes). Suite grew from 139 to 182 tests over the six.
A full-codebase audit on 2026-07-26 (commit `d4e0974`) found 24 bugs across every Python
module under `app/`, both static frontends, and the Docker/Caddy deployment — 5 critical,