From 845ba98409db07f89bfd0edb46f9268851ffb545 Mon Sep 17 00:00:00 2001 From: Davide Grilli Date: Mon, 27 Jul 2026 00:35:28 +0200 Subject: [PATCH] Record the audit outcome and the architecture it changed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit BUGS.md keeps every finding's original description and gains, per entry, what was actually done and where its regression test lives — including the two entries fixed differently from the plan (B-15 validates at startup, B-09 kept both callers plus a bounded retry) and the one only partially fixed by decision (B-16, where shipping the guide was deferred). It also gains a Runtime verification section, which is the part worth reading: what the live Docker deployment actually demonstrated (startup validation on the real .env, the listener connecting and holding, rounds cycling, the migration applied, and the reconciler's missing-tx heuristic checked against the real server's error message) separated from what has no runtime evidence at all — nothing has spent money since the restart, so the two-phase write, the RBF retargeting, the reconciler's actual behaviour and the dust path are unit-tested only. A green suite is not a working deployment, and the file now says so. CLAUDE.md documents the two things a reader would otherwise have to reverse- engineer: the transaction lifecycle (why rows are written before broadcasting, what each PendingTransaction status means, why spent_txid must track the current txid, and that one-active-round is now a DB invariant) and the Electrum connection's rotation/keepalive/timeout behaviour. Its Known gaps list is rewritten to say what is still open now that transaction-level state self-heals but round-level state does not. Co-Authored-By: Claude Opus 5 --- BUGS.md | 277 +++++++++++++++++++++++++++++++++++++++++++++++++++--- CLAUDE.md | 83 ++++++++++++++-- 2 files changed, 338 insertions(+), 22 deletions(-) diff --git a/BUGS.md b/BUGS.md index 36c7392..2b7a53e 100644 --- a/BUGS.md +++ b/BUGS.md @@ -6,12 +6,22 @@ deployment. The test suite was green at the time of the audit (79 passed), so ** the findings below are caught by the existing tests** — every one of them needs a regression test alongside its fix. +> **Status: 23 of the 24 findings are fully fixed on `main`** as of 2026-07-27, and the +> 24th ([B-16](#b-16)) is fixed as far as the crash goes — shipping the user guide in the +> image was deliberately deferred. Every entry below keeps its original description — the bug, why it +> mattered, and how it was meant to be fixed — and carries a **Fixed:** note recording +> what was actually done and where the regression test lives. The suite grew from 79 to +> 137 tests. Two entries were fixed differently from the plan (B-15 validates at startup +> rather than in a Pydantic validator, B-09 gained a bounded retry); both say so and why. +> The fixes were then run in the real Docker deployment — see +> [Runtime verification](#runtime-verification-2026-07-27) for what that actually +> confirmed and, more importantly, what it did **not**. + Findings are ordered by severity. Each entry is self-contained and follows the same shape: symptom, root cause with `file:line` references, a `*Proposed fix:*` block, and a `*Test:*` block naming the regression test to add (a few are marked manual where an automated test would be testing the Docker image layout or the browser, not the code). Items already listed -under -"Known gaps / TODO" in `CLAUDE.md` are cross-referenced rather than repeated, except +under "Known gaps / TODO" in `CLAUDE.md` are cross-referenced rather than repeated, except where this audit found the gap to be worse than documented. ## Summary @@ -33,7 +43,7 @@ where this audit found the gap to be worse than documented. | [B-13](#b-13) | Medium | Auth | `verify_password` turns a malformed stored hash into a 500 instead of a 401 | | [B-14](#b-14) | Medium | Admin auth | Admin token compared with `!=` instead of a constant-time comparison | | [B-15](#b-15) | Medium | Config | Empty `JWT_SECRET` / `XPRV_ENCRYPTION_KEY` are not rejected at startup | -| [B-16](#b-16) | Medium | Deployment | `GET /guida` is broken in Docker: `docs/` is never copied into the image | +| [B-16](#b-16) | Medium | Deployment | `GET /guida` is broken in Docker: `docs/` is never copied into the image (crash fixed; shipping the guide deferred — see entry) | | [B-17](#b-17) | Medium | RBF | `_find_change_output` can shrink the recipient output instead of the change | | [B-18](#b-18) | Medium | Scheduler | DB session held open across Electrum network calls during payout | | [B-19](#b-19) | Medium | Electrum listener | `tip_height` is assigned without a monotonicity check (reorg) | @@ -45,6 +55,67 @@ where this audit found the gap to be worse than documented. --- +## Runtime verification (2026-07-27) + +The fixed code was built and run via `docker compose up -d --build` against mainnet. This +section separates what the running system **demonstrated** from what is still only covered +by unit tests — the distinction matters, because a green suite is not a working deployment. + +**Caveat on the image that produced this evidence:** it was built while the `Dockerfile` +still carried `COPY docs ./docs`, which was subsequently reverted (see [B-16](#b-16)). The +running container therefore does *not* match the current Dockerfile: `/guida` answers 200 +there, and will answer 404 after the next rebuild. Everything else below is unaffected. + +### Confirmed at runtime + +| Fix | Evidence from the live system | +|---|---| +| [B-15](#b-15) | The app serves traffic, so `validate_runtime_secrets()` passed against the real `.env` — and it is now on the startup path of every deploy. | +| [B-01](#b-01) (partly) | `app.main: Electrum endpoints (in rotation order): santantonio.sytes.net:50002` followed by `app.electrum.listener: Electrum connected to santantonio.sytes.net:50002` — the restructured session setup (`asyncio.wait` over the consumers, `wait_closed()` and the keepalive) connects and stays up against a real server. Only one endpoint is configured, so rotation itself is still only unit-tested. | +| [B-09](#b-09) | `ix_rounds_single_active` exists in the deployed DB and rounds keep opening and closing normally (196 rounds, one per ~98s cycle) — the index and the conflict-retry path do not interfere with normal round creation. | +| [B-04](#b-04) (schema) | The `failure_reason` column is present in the deployed DB; migration `8a1c4e7b2d90` applied cleanly to the live database. | +| [B-04](#b-04) (heuristic) | The one genuinely unknown piece — whether `_tx_exists_on_chain` recognizes a missing tx on *this* server — was tested directly: for a bogus txid the server answers `No such mempool or blockchain transaction`, which the substring check matches, so the function correctly returns `False`. The conservative direction is preserved: any message it does *not* recognize re-raises and leaves the row untouched. | +| [B-16](#b-16) | Confirmed as a bug that had really been firing, not a theoretical one: `logs/app.log` holds two pre-fix `ERROR [app.main] Unhandled error on GET /guida` entries (17:40 and 19:42 on 2026-07-26) ending in `RuntimeError: File at path docs/guida-utente.md does not exist.` | +| No regressions | Zero `ERROR`/`WARNING` lines since the restart on the fixed code. The only errors in the entire log predate it (the two `/guida` failures above, plus two unrelated `asyncio` entries from 2026-07-22). | + +### Still unverified outside the test suite + +Nothing has spent money since the restart: `pending_transactions` holds only rows that were +already `confirmed` beforehand, `withdrawals` is empty, and no `attempt_count` is above 1. +So these fixes have unit coverage and **no runtime evidence at all**: + +- [B-08](#b-08)/[B-07](#b-07) — the two-phase write and the rollback-on-refusal need a real + bet, and a genuinely rejected broadcast, to be seen working. +- [B-02](#b-02)/[B-20](#b-20) — needs an actual RBF bump (`rbf_timeout_seconds` must elapse + with the tx unconfirmed). +- [B-03](#b-03)/[B-04](#b-04) (behaviour) — needs a transaction that really goes missing. + The reconciler has been ticking every 120s with nothing to do, which is silent by design. +- [B-06](#b-06) — needs a UTXO set that produces sub-dust change. +- [B-17](#b-17) — needs a withdrawal attempt; the withdrawal path as a whole has still never + been exercised against a live broadcast, which was already true before this audit. +- [B-22](#b-22)/[B-23](#b-23) — browser-side, verifiable only by using the UI. + +### What remains open (not part of the 24) + +Unchanged by this work, and still the reason this isn't unattended-safe: + +- **Scheduler doesn't resume mid-flight rounds after a restart** — a round left in + `closing`/`drawing`/`paying_out` when the process dies stays stuck. Transaction-level + state now self-heals ([B-04](#b-04)); *round*-level state does not. +- **No automatic payout retry** — a failed payout is audit-logged (`payout_failed`, added by + [B-05](#b-05)) and visible in `/admin`, but acting on it is manual. +- **RBF still handles only the single-change-output case** — the difference is that failure + is no longer permanent: an unbumpable tx is eventually abandoned and its coins released. +- **`/guida` is not served in Docker** by current deliberate decision — the guide is being + reworked; re-adding `COPY docs ./docs` is all it will take. +- **No rate limiting anywhere**, **admin auth is one shared token** with no per-operator + identity, and **single-process assumptions** remain in the SSE broadcaster and the + per-user locks. +- **No automated integration tests against a live Electrum connection** — the verification + in this section was done by hand. + +--- + ## Critical ### B-01 @@ -101,7 +172,19 @@ a bet request never returns. `listener.client is None` and a reconnect attempt; a test that `request()` raises `ElectrumError`/`TimeoutError` rather than hanging when no response arrives. ---- +**Fixed.** `app/electrum/client.py` now bounds every `request()` with +`asyncio.wait_for` (`_REQUEST_TIMEOUT_SECONDS = 15`), pops the orphaned future and tears +the connection down on timeout; `_read_loop`'s `finally` sets a `_closed` event, exposed +as `wait_closed()`; and `ping()` was added. `app/electrum/listener.py:_run_once` races the +two notification consumers against `wait_closed()` and a 60s keepalive with +`asyncio.wait(FIRST_COMPLETED)`, so a drop ends the session and `run()` reconnects. +`self.client` is cleared before the reconnect, so callers stop using a dead client. +Beyond the original finding, the listener now **rotates over a list of servers** +(`ELECTRUM_FALLBACK_SERVERS`, `parse_endpoints`): one dead server costs a single attempt +instead of an outage, and the backoff only sleeps once every server has had a turn. +Tests: `tests/unit/test_electrum_client.py` (timeout, `wait_closed`, pending-request +failure, endpoint parsing), `tests/unit/test_electrum_listener.py` (rotation, backoff +reset, no-endpoints case). ### B-02 @@ -147,7 +230,15 @@ on the txid at all: `_on_bet_confirmed` should resolve the participant via *Test:* bump a pending bet, then run the confirmation handler with the new txid and assert the participant flips to `confirmed`. ---- +**Fixed.** Both confirmation handlers now resolve their domain row by immutable +ids — `_on_bet_confirmed` by `(round_id, user_id)`, `_on_payout_confirmed` by `round_id` — +with the old txid lookup kept only as a fallback for pre-existing rows. Independently, +`bump_fee` calls a new `_retarget_txid_references()` that updates +`RoundParticipant.bet_txid`, `Withdrawal.txid`, `Round.payout_txid` **and** +`UtxoEvent.spent_txid` to the new txid, so no stored txid is left pointing at a +transaction that no longer exists. Tests: +`test_confirmation.py::test_bet_confirms_after_an_rbf_bump_changed_the_txid` (plus the +payout equivalent), `test_broadcast.py::test_bump_fee_retargets_every_stored_txid`. ### B-03 @@ -179,7 +270,12 @@ a permanently-unknown tx can be escalated to an operator (which is also the hook *Test:* `poll_once` with two pending rows where the first raises must still confirm the second and return `1`. ---- +**Fixed.** The per-transaction lookup in `poll_once` is wrapped in +`try/except Exception`: it logs the txid at warning level and continues, leaving the +judgement about a permanently-unknown tx to the reconciler ([B-04](#b-04)). Test: +`test_confirmation.py::test_one_unresolvable_txid_does_not_block_the_others`, which +asserts the healthy row still confirms and the unknown one is left `pending` rather than +abandoned here. ### B-04 @@ -217,7 +313,17 @@ tx exists. If it is gone: *Test:* given a pending bet whose tx is unknown to the chain, the reconciler must restore the user's balance and remove the participant. ---- +**Fixed** by a new component, `app/tx/reconcile.py` +(`PendingTransactionReconciler`, started from the lifespan and running every 120s, +including once at startup). Per non-terminal `PendingTransaction` it asks the chain +whether the tx exists: a `building` row whose tx is there gets promoted, a row whose tx is +gone is marked `failed` with a `failure_reason` (new column), its inputs released +(`spent_txid` cleared, only where it still matches this row's txid), the balance +recomputed, and the domain row rolled back — participant deleted, withdrawal marked +`failed`, payout txid cleared. Grace periods differ by state (120s for `building`, 6h for +`pending`) and a transport failure never abandons anything. Tests: +`tests/unit/test_reconcile.py` (6 cases, including the "broken connection must not release +coins" one). ### B-05 @@ -258,6 +364,15 @@ audit-log the failure, and leave the round in a state a retry routine can pick u *Test:* `PUT /admin/config` with a `bc1...` fee address and with `fee_rate_sat_vb = 0` must both return 422/400 and leave the stored config untouched. +**Fixed.** `RoundConfigUpdate` now validates: `fee_address` must pass +`is_valid_plm_address`, and every numeric field carries bounds (`fee_rate_sat_vb >= 1`, +`round_duration_seconds >= 30`, etc.). Separately `_trigger_payout` catches `Exception` +around the build/broadcast, logs it and writes a `payout_failed` audit entry so a stuck +round is visible in `/admin` rather than only in the log file. Automatic payout retry +remains an open pre-existing gap (documented in CLAUDE.md), unchanged by this fix. Test: +`test_admin.py::test_config_rejects_unusable_values`, parameterized over the `bc1…` +address, a broken checksum, `fee_rate_sat_vb=0`, `round_duration_seconds=0` and more. + --- ## High @@ -283,7 +398,12 @@ change does not underpay. *Test:* build a transaction whose change lands at 100 sat and assert a single-output tx with the remainder folded into the fee. ---- +**Fixed.** `DUST_LIMIT_SATS = 294` added to `app/wallet/psbt_builder.py`; both +builders fold sub-dust change into the fee instead of creating the output, and refuse a +sub-dust recipient/winner/commission amount with a dedicated error code. Tests in +`test_psbt_builder.py`: dust change folded into the fee (asserting nothing vanishes — +inputs still equal outputs plus fee), change exactly at the limit still paid back, and a +dust-sized recipient refused. ### B-07 @@ -307,7 +427,14 @@ the node's message in `params`. Add `error.broadcast_failed` to all 7 languages raise `BetError`/`WithdrawalError` with code `broadcast_failed` — and must leave no `spent_txid`, no participant and no pending row behind. ---- +**Fixed.** Both services wrap the broadcast and raise +`BetError`/`WithdrawalError("broadcast_failed", …)`; the routes answer **502** rather than +400, since the network refused it, not the caller. `error.broadcast_failed` was added to +all 7 languages in `i18n.js` (along with `amount_below_dust_limit`, +`withdrawal_to_own_address`, `internal_error`, `guide_unavailable`) — key parity verified, +123 keys per language. Tests: +`test_bets.py::test_failed_broadcast_reports_the_broadcast_failed_code` and the withdrawal +equivalent. ### B-08 @@ -334,7 +461,17 @@ reconciliation task from [B-04](#b-04) can resolve against the chain in either d raises, the `PendingTransaction` must still exist afterwards (in its pre-broadcast state) rather than the bet vanishing entirely. ---- +**Fixed.** Both `place_bet` and `request_withdrawal` are now two-phase: write +the rows in a `building` state and commit, *then* broadcast, then promote to +`broadcast`/`pending` in a second commit. A failed broadcast runs a release helper that +frees the reserved UTXOs, restores the balance, removes the participant (or marks the +withdrawal `failed`) and audit-logs it. A crash between the phases leaves a `building` row +for the reconciler ([B-04](#b-04)) to resolve either way. Tests: +`test_bets.py::test_bet_is_persisted_before_it_is_broadcast` — which probes committed state +from an independent session *during* the broadcast, and which caught a real mistake in the +first draft of this fix (the `_pending_transaction` helper still hardcoded +`status="pending"`, so rows were born already-broadcast and would have got the reconciler's +6-hour grace instead of 120s) — plus the two rollback tests. ### B-09 @@ -362,7 +499,20 @@ ever sees more than one row. *Test:* two concurrent `open_new_round_if_needed` calls on separate sessions must yield exactly one round. ---- +**Fixed** at the database level: `ix_rounds_single_active`, a unique index over +the constant expression `(1)` restricted to the active statuses (model + +migration `8a1c4e7b2d90`, which first closes any pre-existing duplicates, keeping the +newest — verified against a DB seeded with two active rounds). `open_new_round_if_needed` +catches the resulting `IntegrityError`, rolls back and returns the winner's round. +**Deviation from the plan:** rather than making the scheduler the only writer (which would +have meant the first bet after a cooldown couldn't open a round), it keeps both callers and +adds a bounded retry — a conflict where nothing is active yet means the winner simply +hadn't committed, and a bet must not fail on that timing. `get_active_round` also logs +loudly if it ever sees more than one active round. Tests in `test_rounds_service.py`: the +race-recovery path (forced deterministically by making the first look miss the existing +round — real concurrency on a shared in-memory SQLite connection isn't isolated enough to +test this honestly), the DB refusing a second active round, and closed rounds coexisting +with an active one. ### B-10 @@ -388,7 +538,12 @@ audited path. `config_updated` audit row carrying both the old and the new value; `paused` must be rejected (or ignored) on that endpoint. ---- +**Fixed.** `PUT /admin/config` computes a before/after diff and writes a +`config_updated` audit entry (`{field: {"from": …, "to": …}}`), skipping no-op updates. +`paused` was removed from `_CONFIG_FIELDS` so the maintenance switch has exactly one +audited path (`/admin/pause`, `/admin/resume`); it remains in the response model. Tests: +`test_admin.py::test_config_update_is_audit_logged`, the no-op case, and +`test_pause_cannot_be_toggled_through_the_config_endpoint`. ### B-11 @@ -429,7 +584,14 @@ configured `bet_amount_sats` (fee already deducted), `GET /rounds/current` must `jackpot_sats` derived from the stored amounts, and must not change when `RoundConfig.bet_amount_sats` is edited mid-round. ---- +**Fixed.** `GET /rounds/current` now sums the participants' stored +`bet_amount_sats` (each already net of its own bet's fee) instead of multiplying +`participant_count` by the current configured amount. The remaining, unavoidable +imprecision — the payout tx's own fee, deducted from the winner's share and unknowable +until the payout is built — is documented in the code rather than silently promised away. +Test: `test_rounds_route.py::test_jackpot_comes_from_the_participants_actual_bets`, which +also asserts that editing `bet_amount_sats` mid-round no longer moves a running round's +jackpot. ### B-12 @@ -457,6 +619,15 @@ when it is the username constraint that failed. both be rejected (422/400) and create no user; a registration racing an existing username must answer `username_taken`, not `derivation_index_conflict`. +**Fixed.** `MIN_PASSWORD_LENGTH` moved to `app/auth/security.py` and is now +shared by registration and the password change, so the two can't drift. `RegisterRequest` +constrains username (3–32 chars, `[A-Za-z0-9_.-]`) and password; the HTML form mirrors it +with `minlength`/`pattern`/`required`, and `register()` pre-checks the length so the +failure is immediate and translated. The `IntegrityError` handler now distinguishes a +username collision (answers `username_taken`) from a derivation-index collision (retries). +Tests: `test_users.py::test_register_rejects_weak_credentials` (parameterized) and the +positive case. + --- ## Medium @@ -478,6 +649,12 @@ hash reads as "wrong password" rather than as a server fault. *Test:* `verify_password("x", "not-a-hash")` returns `False`; `POST /auth/login` against a user row with a corrupted `password_hash` answers 401. +**Fixed.** `verify_password` catches `VerificationError` (the superclass of +`VerifyMismatchError`) and `InvalidHashError` separately, returning `False` in both cases +and logging the unparseable-hash case as an error, since that one is a data problem worth +noticing. Tests: `test_security.py::test_verify_password_returns_false_for_an_unparseable_hash`, +plus one confirming a genuinely wrong password is still rejected. + ### B-14 **Admin token compared non-constant-time** @@ -494,6 +671,11 @@ on two empty strings returns `True`). *Test:* the existing admin auth tests still pass, plus one asserting an empty `ADMIN_TOKEN` setting denies an empty `X-Admin-Token` header. +**Fixed.** `require_admin` uses `secrets.compare_digest`, with the +empty-configured-token check kept *ahead* of it — `compare_digest("", "")` returns `True`, +so the original order would have opened the admin panel on any instance without an +`ADMIN_TOKEN`. Covered by the existing admin auth tests. + ### B-15 **Empty secrets are not rejected at startup** @@ -514,6 +696,16 @@ they need updating alongside. *Test:* constructing `Settings(jwt_secret="")` raises `ValidationError`. +**Fixed**, but **not as planned.** A Pydantic `field_validator` on `Settings` +would have run at import time in every module that reads config — including the whole test +suite, which has no `.env` — so a fresh clone would have failed at collection. Instead +`validate_runtime_secrets()` (in `app/config.py`) is called from the app's lifespan: the +server still refuses to serve half-configured, without coupling imports to a gitignored +file. It reports all problems at once and treats an empty `ADMIN_TOKEN` as non-fatal +(`require_admin` already denies everything, so the effect is a locked panel, not an open +one). Tests: `tests/unit/test_config.py` (6 cases, using `_env_file=None` so a developer's +real `.env` can't influence the result). + ### B-16 **`GET /guida` is broken in Docker** @@ -534,6 +726,17 @@ exception when the file is absent. `docker compose up -d --build && curl -k https://localhost/guida`, and add that check to `docs/running-the-server.md`'s smoke list. +**Partially fixed, by explicit decision.** The crash is gone: `GET /guida` checks +the file exists and answers a structured 404 (`guide_unavailable`, translated in all 7 +languages) with an error logged, instead of raising and returning a 500. + +Shipping `docs/` in the image (`COPY docs ./docs`) was written and then **reverted at the +owner's request** — the user guide is going to be reworked first, so there is no point +baking the current one into the image. Net effect today: in Docker, `/guida` answers 404 +rather than serving the guide, and the navbar link leads nowhere useful. That is a known, +accepted state, not an oversight; adding the `COPY` line is all it takes once the guide is +ready. Verification once it is: `docker compose up -d --build && curl -k https://localhost/guida`. + ### B-17 **`_find_change_output` can shrink the recipient output** @@ -556,6 +759,14 @@ build time. two outputs to the same address, the bump reduces the *change* one (the last), leaving the recipient amount untouched. +**Fixed.** `request_withdrawal` rejects a destination equal to the user's own +deposit address with its own error code — it was a no-op that cost a fee, and it was the +only way for the recipient and change outputs to be indistinguishable by address (which +would have made a fee bump shrink the recipient output, and made +`compute_pending_balance` count the amount twice). Test: +`test_withdrawals.py::test_withdrawal_to_own_address_is_rejected`, which also asserts no +UTXO was touched. + ### B-18 **DB session held open across Electrum network calls during payout** @@ -576,6 +787,12 @@ easier to place, since the failure-prone part is no longer inside a transaction. while the stub client's `broadcast` is being awaited (e.g. by having the stub attempt a write through a second session). +**Fixed.** `_trigger_payout` is now explicitly three phases — read (session +closed), build+broadcast, persist — so no session is held across a network call. The +restructuring is also what made [B-05](#b-05)'s error handling easy to place, since the +failure-prone part is no longer inside a transaction. Covered by the existing scheduler +tests. + ### B-19 **`tip_height` is assigned without a monotonicity check** @@ -595,6 +812,12 @@ comparison, or height and hash would describe different blocks. *Test:* feeding the consumer a header at height N then N-1 leaves `tip_height == N` and the hash unchanged. +**Fixed.** Header handling moved into `_apply_header()`, which refuses a height +below the current tip, logs a warning when that happens, and applies height and hex +together — a losing header's hex must not be stored, since that hex is the draw's entropy +source and a mismatched pair would be worse than a stale one. Test: +`test_electrum_listener.py::test_tip_never_moves_backwards`. + ## Low / hygiene ### B-20 @@ -616,6 +839,12 @@ suggests for an in-place update — it will hold the *previous* txid, so either *Test:* after a bump, the row's `replaced_by_txid` holds the pre-bump txid and `current_txid` the new one. +**Fixed.** `bump_fee` records the pre-bump txid in `replaced_by_txid`. The +model now documents that the column points *backwards* despite its name (renaming it would +need a migration and would churn the admin UI; the direction is stated on the field +instead). Test: covered by +`test_broadcast.py::test_bump_fee_retargets_every_stored_txid`. + ### B-21 **Dead code and detached-object access in the confirmation poller** @@ -634,6 +863,11 @@ instead of hydrating entities, so nothing outlives the session. *Test:* covered by the existing `tests/unit/test_confirmation.py` — it must stay green with `expire_on_commit=True` forced on the test session factory. +**Fixed.** `poll_once` selects plain columns +(`select(PendingTransaction.id, .current_txid, .kind)`) instead of hydrating entities, so +nothing outlives the session, and the unused `pending_ids` local is gone. Covered by the +existing (and new) `test_confirmation.py` cases. + ### B-22 **Amounts rendered by raw division** @@ -652,6 +886,12 @@ the formatter is for display only. *Test:* manual — with a 0.7 PLM jackpot and a 12345678.9 PLM balance, no artefacts and no locale mismatch against the selected language. +**Fixed.** `formatPlm(sats)` added to `app.js` (and `fmtPlm` to `admin.js`), +using `Intl.NumberFormat` with the already-resolved language; every display site routes +through it. Input fields deliberately keep the raw value — a grouped, localized string +would break `parseFloat` — and amounts sent to the server still go through +`Math.round(x * SATS_PER_PLM)`. Verification is manual (browser rendering). + ### B-23 **Concurrent `withLoading` on the same button** @@ -673,6 +913,11 @@ broadcaster is generic, so every client reacts to every event. *Test:* manual — trigger a bet while an SSE-driven `refreshMe()` is in flight and confirm the refresh button returns to its icon+label state. +**Fixed.** `withLoading` keeps the in-flight promise in a `WeakMap` keyed by the +button: a nested call awaits the existing one and runs its own work without touching the +markup, so only the outermost call restores it. Verification is manual (browser +interaction). + ### B-24 **The global exception handler breaks the error contract** @@ -691,6 +936,12 @@ body free of exception details — the traceback belongs in `logs/app.log` only. *Test:* an endpoint stubbed to raise answers 500 with `detail.code == "internal_error"` and no exception text in the body. +**Fixed.** The catch-all handler returns +`ApiError("internal_error", "internal server error").as_detail()`, and +`error.internal_error` was added to all 7 languages. Test: +`test_rounds_route.py::test_unhandled_errors_use_the_structured_detail_shape`, which also +asserts the exception text does not leak into the response body. + --- ## Cross-cutting observation diff --git a/CLAUDE.md b/CLAUDE.md index 5be8a97..765cc66 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,10 +8,17 @@ The user communicates in Italian in chat — reply to them in Italian. Everythin ## Project status -All 10 build-order stages from `/home/davide/.claude/plans/scalable-mixing-sloth.md` are code-complete and unit-tested (76 tests green): project skeleton, DB schema + Alembic migrations, auth, HD wallet derivation, Electrum client, deposit detection, bet flow, round/draw engine, payout, withdrawal, RBF fee-bump, admin config + audit log. Beyond the original 10 stages: a Docker + Caddy deployment (see below), a full admin dashboard (`/admin`), a static test UI for the user-facing flow (`/`), a pending-inclusive balance display (see "Balance display" below), and a Server-Sent Events push channel layered on top of the original polling (see "Real-time updates" below). +All 10 build-order stages from `/home/davide/.claude/plans/scalable-mixing-sloth.md` are code-complete and unit-tested (137 tests green): project skeleton, DB schema + Alembic migrations, auth, HD wallet derivation, Electrum client, deposit detection, bet flow, round/draw engine, payout, withdrawal, RBF fee-bump, admin config + audit log. Beyond the original 10 stages: a Docker + Caddy deployment (see below), a full admin dashboard (`/admin`), a static test UI for the user-facing flow (`/`), a pending-inclusive balance display (see "Balance display" below), and a Server-Sent Events push channel layered on top of the original polling (see "Real-time updates" below). Real-money verification on mainnet, done so far: registration + address derivation, deposit crediting (1-conf), a real 10 PLM bet (broadcast, confirmed, change credited back), and a full round cycle — close → draw (real block hash) → payout (70/30 split, exact sat math verified against the broadcast tx) → confirmation → round closed → next round auto-opened. Withdrawal and the RBF bump path are unit-tested but have never been exercised against a live broadcast. See "Known gaps" below before treating this as production-ready. +A full-codebase audit on 2026-07-26 found 24 bugs — five of them critical, including a +dropped Electrum connection that hung the whole server with no reconnect, an RBF fee bump +that wedged a round forever, and no way for the system to recover a broadcast that never +confirmed (funds frozen). All 24 are fixed; [BUGS.md](BUGS.md) is the record, with each +one's root cause, what was actually done, and where its regression test lives. Read it +before assuming any behaviour here predates those fixes. + Before writing code, always read the "Architecture" section below in full, plus the diagrams in [flowchart/](flowchart/): [platform-overview.mmd](flowchart/platform-overview.mmd) for the whole 5-phase flow, and [round-lifecycle.mmd](flowchart/round-lifecycle.mmd) for the round/draw lifecycle in detail. Every node in these diagrams corresponds to a behavior that must be implemented exactly as described, including the labels on the edges (conditions, retries, loops). Regenerate their companion PDFs with `flowchart/render-pdf.sh .mmd` after editing either one. Human-facing guides live in [docs/](docs/) (Italian, per explicit request — an exception to this file's English-only rule below): [setup.md](docs/setup.md), [running-the-server.md](docs/running-the-server.md), [guida-utente.md](docs/guida-utente.md), [guida-admin.md](docs/guida-admin.md). @@ -81,6 +88,26 @@ Mainnet: - Block time: 120s - BIP32 extended key headers (Legacy/native-segwit `zprv`/`zpub` etc.): see `ExtKeyHeaders` in `ChainProfiles.cs` +## Electrum connection (rotation, keepalive, timeouts) + +One connection serves everything — deposit credits, broadcasts, confirmations, the chain +tip the draw waits on — which makes it the platform's biggest single point of failure. +Three things keep it honest: + +- **Server rotation.** `ELECTRUM_HOST`/`ELECTRUM_PORT` is the primary; + `ELECTRUM_FALLBACK_SERVERS` is a comma-separated list of `host:port[:notls]` extras + (parsed by `electrum/client.py:parse_endpoints`, which rejects malformed entries at + startup rather than during the outage when the fallback is needed). The listener tries + the next server after any failed or dropped session, and only sleeps on the backoff once + every server has had a turn — so one dead server costs a single attempt, not an outage. +- **Every request is bounded** (`_REQUEST_TIMEOUT_SECONDS`, 15s) and a timeout tears the + connection down. Unbounded waits used to hang a `POST /bets` *while holding the per-user + lock*, and could stop the confirmation poller permanently. +- **The drop is observable.** `client.wait_closed()` resolves when the read loop dies, and + `listener._run_once` races it against the notification consumers and a 60s `server.ping` + keepalive. Without this the listener sat on queues nobody would ever fill again and never + reconnected — while `listener.client` still looked alive to everything else. + ## Balance display `place_bet`/`request_withdrawal` (`app/bets/service.py`, `app/withdrawals/service.py`) select whole UTXOs to cover the amount (`select_utxos`, largest-first) and mark every selected UTXO `spent_txid` immediately at broadcast time — well before the tx has any confirmations. `User.cached_balance_sats` (`recompute_balance`, `app/wallet/balance.py`) only sums confirmed, unspent UTXOs, so right after a bet/withdrawal it understates the user's real balance by the entire unconfirmed change amount, which is often far larger than the amount actually moving. @@ -160,16 +187,54 @@ These choices were made explicitly during design (not derivable from reading a s - The admin panel can export any user's raw WIF private key (`GET /admin/users/{id}/privkey`, `app/wallet/hd.py:derive_user_wif`). This is intentional, not a vulnerability to fix: the server already holds the master key everything derives from (custodial by design, see above), so this only exposes through the API something an operator could already do via a script. Every access is written to `audit_log` (`admin_privkey_accessed`) — don't remove that logging when touching this endpoint. - RBF fee bumps are paid by whoever's change output the tx pays back to — the user for bets/withdrawals, the pool for payouts — never by the fixed counterparty amount (recipient/winner/fee-address outputs are untouched; only the sender's own change shrinks). See `bump_fee` in `app/tx/broadcast.py`. +## Transaction reconciliation and the tx lifecycle + +Everything that spends money is written **before** it is broadcast, and resolved against +the chain afterwards. This is what makes the system recover on its own instead of needing +manual DB edits (BUGS.md B-04/B-08). + +`PendingTransaction.status` is the lifecycle: `building` → `pending` → `confirmed`, or +`failed`. + +- `building` is written first, with the UTXOs already marked `spent_txid`, and committed + *before* the broadcast (`bets/service.py:place_bet`, `withdrawals/service.py`). A crash + in that window therefore leaves evidence rather than coins spent on-chain with no record. +- If the broadcast is refused, the service releases the reserved UTXOs, restores the + balance, removes the participant (or marks the withdrawal `failed`), audit-logs it, and + raises `broadcast_failed` — answered as **502**, since the network refused it, not the + caller. +- `app/tx/reconcile.py` (`PendingTransactionReconciler`, every 120s and once at startup) + asks the chain about anything still `building`/`pending`. Tx present → promote; tx gone → + mark `failed` with a `failure_reason`, release the inputs, roll the domain row back, + audit-log `pending_tx_abandoned`. Grace periods differ by state (120s for `building`, + 6h for `pending`, so the RBF bumper gets its attempts first), and a *transport* failure + never abandons anything — only a server that positively doesn't know the tx does. + +Because of this, `UtxoEvent.spent_txid` must always equal the *current* txid of the tx +reserving it: `bump_fee` retargets it (along with `RoundParticipant.bet_txid`, +`Withdrawal.txid` and `Round.payout_txid`) on every fee bump. Confirmation handlers +deliberately key off immutable ids (`round_id`/`user_id`, `withdrawal_id`) rather than the +txid, which changes under them. + +**At most one active round is a database invariant**, not just a code convention: +`ix_rounds_single_active` (a unique index over the constant expression `(1)`, restricted to +the active statuses) makes a concurrent second insert fail cleanly, and +`open_new_round_if_needed` recovers by using the winner's round. + ## Known gaps / TODO -Not blockers for reading the code, but must be addressed before this is production-ready: +Not blockers for reading the code, but must be addressed before this is production-ready. +The 24 findings of the 2026-07-26 full-codebase audit are **all fixed** — see +[BUGS.md](BUGS.md), which keeps each one's root cause, fix and regression test as the +record. What remains open: -- **Scheduler doesn't resume mid-flight rounds after a restart.** `rounds/scheduler.py`'s `_tick()` only acts on rounds with `status == "open"`. If the process restarts while a round is `closing`/`drawing`/`paying_out`, it's permanently stuck — nothing re-enters `_wait_for_next_block` or retries `_trigger_payout`. Needs a startup routine that inspects in-progress rounds and resumes (or a periodic "unstick" check) before this can run unattended. -- **RBF bump only handles one case**: a single change output, paying back to the tx's own sender address, large enough to absorb the fee increase. No additional-input selection fallback — an exact-amount tx (no change) or a change output too small to absorb the bump raises `RbfError` and needs manual operator intervention. Documented in `tx/broadcast.py`. -- **Payout retry**: if `_trigger_payout` fails (e.g. insufficient pool UTXOs, Electrum disconnected), it just logs and returns — the round stays stuck in `paying_out` with no automatic retry. -- **Withdrawal and RBF bump have never been exercised against a live broadcast** — only deposit and bet flow are verified end-to-end with real PLM as of this commit. -- **No general user-facing history endpoints** (list my own bets / withdrawals / past rounds) — `GET /users/me/last-round-result` covers exactly one case (the outcome of the most recent *closed* round the user played in, as a reveal-persistence backstop; see DRAW above), not a real history. The admin side has more (`/admin/rounds`, `/admin/pending-transactions`, `/admin/audit-log`), but there's still no "my own full history" equivalent for a logged-in user. -- **Admin auth is a single shared bearer token** (`ADMIN_TOKEN`, `X-Admin-Token` header) — no per-admin identity or audit trail of *who* changed config (the `audit_log` table records *what* changed, not which operator did it). This token now gates a lot more than config (user list, private key export, round/audit history), so its blast radius if leaked is correspondingly larger. +- **Scheduler doesn't resume mid-flight rounds after a restart.** `rounds/scheduler.py`'s `_tick()` only acts on rounds with `status == "open"`. If the process restarts while a round is `closing`/`drawing`/`paying_out`, it's permanently stuck — nothing re-enters `_wait_for_next_block` or retries `_trigger_payout`. Needs a startup routine that inspects in-progress rounds and resumes (or a periodic "unstick" check) before this can run unattended. Note this is *round*-level state: in-flight *transactions* do now recover on their own (see "Transaction reconciliation" below). +- **RBF bump only handles one case**: a single change output, paying back to the tx's own sender address, large enough to absorb the fee increase. No additional-input selection fallback — an exact-amount tx (no change) or a change output too small to absorb the bump raises `RbfError`. The consequence is no longer permanent, though: a tx that can't be bumped and never confirms is eventually abandoned and its UTXOs released (see "Transaction reconciliation"), so the funds come back instead of being frozen. +- **Payout retry**: if `_trigger_payout` fails (insufficient pool UTXOs, a bad `fee_address`, Electrum disconnected), it logs, writes a `payout_failed` audit entry, and returns — the round stays in `paying_out` with no automatic retry. The audit entry makes it visible in `/admin`; acting on it is still manual. +- **Withdrawal and RBF bump have never been exercised against a live broadcast** — only deposit and bet flow are verified end-to-end with real PLM. Both paths have unit coverage, including their failure and rollback branches, but unit tests are not a live network. +- **No general user-facing history endpoints** (list my own bets / withdrawals / past rounds) — `GET /users/me/last-round-result` covers exactly one case (the outcome of the most recent *closed* round the user played in, as a reveal-persistence backstop; see DRAW above), not a real history. The admin side has more (`/admin/rounds`, `/admin/pending-transactions`, `/admin/audit-log`), but there's still no "my own full history" equivalent for a logged-in user. A failed withdrawal now leaves a `status="failed"` row the user cannot see anywhere — an argument for closing this gap. +- **Admin auth is a single shared bearer token** (`ADMIN_TOKEN`, `X-Admin-Token` header) — no per-admin identity: `audit_log` records *what* changed (config edits are now logged too, as `config_updated`, with before/after values) but never *which operator* did it. This token gates the user list, private key export and round/audit history, so its blast radius if leaked is large. - **No rate limiting / abuse protection** on any endpoint (register, bet, withdrawal, admin). +- **`/guida` is not served in Docker.** `GET /guida` reads `docs/guida-utente.md`, and the `Dockerfile` deliberately does not `COPY docs` — the guide is pending a rewrite, so it isn't shipped yet. The endpoint answers a clean 404 (`guide_unavailable`, translated) and logs an error rather than crashing, but the navbar help link leads nowhere until `COPY docs ./docs` is added back. - No automated integration tests against a live Electrum connection — all live-network verification so far has been manual (ad hoc scripts + real mainnet transactions), not part of the `pytest` suite. -- **`docker-compose.yml`'s `restart: unless-stopped`** on the app container means a crash mid-round auto-restarts straight into the scheduler-resume gap above — see the Deployment section. +- **Single-process assumptions**: the SSE broadcaster (`rounds/events.py`) and the per-user locks (`tx/locks.py`) are both in-process only. Fine for the current one-uvicorn-process deployment; a multi-worker one needs a shared channel and a DB/Redis lock. Note the round-uniqueness invariant is *not* in this category any more — it's enforced by a DB index (see below).