Run SQLite in WAL mode with a busy_timeout (B-39)

create_async_engine had no connect_args and there was no PRAGMA
anywhere in the repo. SQLite's default rollback-journal mode lets a
writer block every reader for the duration of its transaction, and a
second writer arriving while one is already active fails immediately
with "database is locked" rather than waiting at all - realistic given
five concurrent background tasks (scheduler, confirmation poller, RBF
bumper, two reconcilers) plus every HTTP handler share one file, and
nothing previously handled that error.

app/db/base.py now registers a "connect" event on the engine that sets
journal_mode=WAL, synchronous=NORMAL and a 5-second busy_timeout on
every new DBAPI connection - applied only when the dialect is sqlite,
so a future PostgreSQL DATABASE_URL is unaffected. WAL lets readers and
writers proceed without blocking each other, and busy_timeout gives a
second writer a real window to wait instead of failing instantly.

Left out: an explicit application-level retry wrapper for "database is
locked" in the background loops, the other half of the proposed fix -
busy_timeout already gives SQLite itself several seconds to resolve
writer-vs-writer contention before ever raising, and every background
loop already catches and logs an unhandled exception before its next
scheduled tick, which is itself a retry, just not an immediate one.

Suite grows from 211 to 214 tests. BUGS.md moves B-39 to Previously
fixed.
This commit is contained in:
2026-07-27 14:53:22 +02:00
parent 702b37b319
commit 97545ad91f
3 changed files with 103 additions and 19 deletions
+6 -18
View File
@@ -1,11 +1,11 @@
# Known bugs
A second full-codebase audit on 2026-07-27 found **25 further issues** (4 critical, 6 high,
7 medium, 8 low), listed below as B-39 … B-49. B-25 through B-38 are fixed (see "Previously
fixed" below) — no Critical-severity finding remains open; the other 11 are Medium/Low.
7 medium, 8 low), listed below as B-40 … B-49. B-25 through B-39 are fixed (see "Previously
fixed" below) — no Critical-severity finding remains open; the other 10 are Medium/Low.
The 139-test suite was green at the time of the audit, so none of these were caught by existing
coverage — every fix lands with a regression test (the fourteen fixes so far brought the suite
from 139 to 211).
coverage — every fix lands with a regression test (the fifteen fixes so far brought the suite
from 139 to 214).
The recurring pattern across the open findings is worth stating once: the code is rigorous
about the failure modes that have actually been hit, and silent about the ones that have not.
@@ -18,19 +18,6 @@ admin auth, single-process assumptions, no user-facing history, etc.) are docume
## Medium
### B-39 — SQLite with no WAL, no `busy_timeout`, and five concurrent writer tasks
`db/base.py:6` calls `create_async_engine(settings.database_url)` with no `connect_args`, and
there is no `PRAGMA` anywhere in the repo (verified by grep). Without `journal_mode=WAL`
readers block writers, and the concurrent writers are five background tasks plus every HTTP
handler. `database is locked` under load is realistic, and nothing handles it.
**Proposed fix.** Set `journal_mode=WAL`, `synchronous=NORMAL` and a `busy_timeout` of a few
seconds on connect (a `connect` event listener on the engine, applied only for the SQLite
dialect), and retry `OperationalError: database is locked` in the background loops. Longer
term this is an argument for PostgreSQL, which the single-process constraints in CLAUDE.md
also point at.
### B-40 — `bump_fee` holds a DB session open across N network calls
`tx/broadcast.py:80` issues one `get_transaction` **per input** (up to 15s each) and then a
@@ -145,9 +132,10 @@ already does.
- **B-36** — a stalled draw wait had no timeout, no log, and no audit trail, so a frozen round showed nothing in `/admin`
- **B-37** — a withdrawal covered by unconfirmed change answered "insufficient balance" instead of distinguishing it from actually having no funds
- **B-38** — the SSE subscriber cap was global, so one client opening enough connections degraded every other user to polling
- **B-39** — SQLite ran without WAL or a `busy_timeout`, so a writer could block every reader and a second writer failed immediately instead of waiting
See git history for the fix-by-fix breakdown (commits `f13f685`, `50a43ae`, `933760e`, and the
B-28/B-29/B-30/B-31/B-32/B-33/B-34/B-35/B-36/B-37/B-38 fixes). Suite grew from 139 to 211 tests over the fourteen.
B-28/B-29/B-30/B-31/B-32/B-33/B-34/B-35/B-36/B-37/B-38/B-39 fixes). Suite grew from 139 to 214 tests over the fifteen.
A full-codebase audit on 2026-07-26 (commit `d4e0974`) found 24 bugs across every Python
module under `app/`, both static frontends, and the Docker/Caddy deployment — 5 critical,