Add a periodic deposit reconciler, and stop losing subscribe tasks (B-30)

Deposits were credited exclusively by scripthash-change notifications,
with nothing re-verifying a user's balance against the chain if a
subscription was ever silently lost. address_for_new_user's subscribe
was fire-and-forget: the task wasn't retained, so it could be
garbage-collected mid-flight, and any failure (including self.client
turning None between the check and the task running) vanished into
asyncio's default unretrieved-exception handler instead of being
logged anywhere. On an otherwise healthy, long-lived connection there
may be no reconnect for days to re-subscribe everyone, so a user in
that state never saw their deposits.

address_for_new_user now retains the task and logs its exception if
it fails. New app/deposits/reconcile.py adds DepositReconciler, a
periodic sweep (every 5 minutes, gated on the Electrum client being
connected, same shape as tx/reconcile.py) that round-robins over every
user and calls the listener's own refresh_user (renamed from
_refresh_user since it's now called from outside the class) - so the
notification-driven and periodic paths can never behave differently.
Deliberately sweeps every user rather than only ones missing from the
in-memory scripthash map, since that map can't tell "never subscribed"
apart from "subscribed, but the server stopped delivering
notifications for it". Wired into app/main.py's lifespan alongside the
other three background reconcilers.

Suite grows from 176 to 182 tests. BUGS.md moves B-30 to Previously
fixed.
This commit is contained in:
davide committed 2026-07-27 10:35:23 +02:00
1 parent e8fdea0389
commit 63df38d30b
7 files changed
+255 -40

No files matched your search

+31 -5
View File
@@ -63,6 +63,11 @@ class ElectrumListener:
self._endpoints = list(endpoints or [])
self._endpoint_index = 0
self._scripthash_to_user: dict[str, int] = {}
# Retains address_for_new_user's fire-and-forget subscribe task so it
# can't be garbage-collected mid-flight, and so its exception (if any) is
# actually observed instead of only reaching asyncio's default "Task
# exception was never retrieved" handler (B-30).
self._background_tasks: set[asyncio.Task] = set()
self.tip_height: int = 0
self.tip_header_hex: str | None = None
self.client: ElectrumClient | None = None
@@ -77,11 +82,32 @@ class ElectrumListener:
def address_for_new_user(self, user_id: int, address: str) -> None:
"""Called right after a user registers so their deposit address starts
being watched immediately, without waiting for the next reconnect cycle."""
being watched immediately, without waiting for the next reconnect cycle.
Best-effort, not retried on its own: `self.client` can still become None
between the check below and the task actually running (the connection
drops in between), which used to raise an AssertionError inside an
untracked task and vanish silently (B-30). The exception is now logged
instead, and — since a failure here just means this one address stays
unsubscribed until the next reconnect's `_subscribe_all_users` or the
periodic `DepositReconciler` sweep (also B-30) catches it — that's an
acceptable, self-healing outcome rather than something worth its own
retry/backoff loop.
"""
scripthash = address_to_scripthash(address)
self._scripthash_to_user[scripthash] = user_id
if self.client is not None:
asyncio.create_task(self._subscribe_and_refresh(scripthash, user_id))
task = asyncio.create_task(self._subscribe_and_refresh(scripthash, user_id))
self._background_tasks.add(task)
task.add_done_callback(self._background_tasks.discard)
task.add_done_callback(self._log_subscribe_task_failure)
def _log_subscribe_task_failure(self, task: asyncio.Task) -> None:
if task.cancelled():
return
exc = task.exception()
if exc is not None:
logger.warning("could not subscribe a newly-registered user's address: %r", exc)
async def run(self) -> None:
backoff = 1
@@ -179,7 +205,7 @@ class ElectrumListener:
async def _subscribe_and_refresh(self, scripthash: str, user_id: int) -> None:
assert self.client is not None
await self.client.subscribe_scripthash(scripthash)
await self._refresh_user(user_id, scripthash)
await self.refresh_user(user_id, scripthash)
def _apply_header(self, header: dict) -> None:
"""Record a new chain tip, refusing to move backwards.
@@ -329,9 +355,9 @@ class ElectrumListener:
scripthash, _status = await queue.get()
user_id = self._scripthash_to_user.get(scripthash)
if user_id is not None:
await self._refresh_user(user_id, scripthash)
await self.refresh_user(user_id, scripthash)
async def _refresh_user(self, user_id: int, scripthash: str) -> None:
async def refresh_user(self, user_id: int, scripthash: str) -> None:
"""Three phases, so no DB session is held across a network call (B-18),
same shape as _trigger_payout: read what's needed, corroborate any
candidate external spends against other servers (B-29), then persist.