10 Commits
Author SHA1 Message Date
davideandClaude Sonnet 5 6a90136b50 Widen raw_tx_hex and payload_json from String to Text (B-47)
Both held arbitrary-length data (a raw signed transaction hex, an audit
payload) in a bare String, which SQLAlchemy compiles to VARCHAR with no
length. SQLite and PostgreSQL accept that; other backends like MySQL
require a length on VARCHAR and would reject it. Add a migration
(verified upgrade/downgrade/upgrade round-trip, and confirmed with
`alembic check` that it leaves no further diff against the models).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-27 16:24:05 +02:00
davideandClaude Sonnet 5 31bc9a327f Compare admin token as UTF-8 bytes to avoid TypeError on non-ASCII input (B-46)
secrets.compare_digest raises TypeError instead of returning False when a
str argument contains non-ASCII characters, turning a bad admin token into
an unhandled 500 instead of the expected 403. Encode both sides before
comparing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-27 16:17:40 +02:00
davideandClaude Sonnet 5 6045c89ed0 Bound admin list endpoint limits, add status filter to pending-transactions (B-45)
/admin/rounds and /admin/audit-log accepted any limit, including -1 (which
SQLite treats as "no limit"), and /admin/pending-transactions had no limit
at all -- it grows without end. Add Query(default=..., ge=1, le=500) to all
three, plus an optional status filter on pending-transactions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-27 16:10:53 +02:00
davideandClaude Sonnet 5 a574db0d93 Align README and running-the-server.md with the Docker-only policy (B-44)
CLAUDE.md declares the server always runs via Docker (dev and prod
alike) with no supported bare-uvicorn workflow, but README's Quick
start and docs/running-the-server.md's "Locale / venv" section still
documented running uvicorn directly — a leftover from before that
policy was adopted. Rewrite both to a single Docker-only path and
update CLAUDE.md's own note about it.

Verified docker compose run --rm app python scripts/generate_master_key.py
against a real build/run to confirm the Quick start's Docker commands
actually work as documented.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-27 16:04:24 +02:00
davideandClaude Sonnet 5 d60da11603 Add baseline HTTP security headers in Caddy (B-43)
Caddy adds none of these on its own. Add HSTS, X-Content-Type-Options,
X-Frame-Options, Referrer-Policy and a CSP scoped to default-src
'self' plus the one external asset (Google Fonts). script-src/style-src
need 'unsafe-inline' because both SPAs rely on inline onclick handlers
and style="" attributes throughout — removing those is a separate,
larger refactor.

Validated with `caddy validate` and a live container curl check.
Adds a static regression test asserting the header directives stay
present in the Caddyfile, since nothing else in the Python suite
exercises it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-27 15:44:47 +02:00
davideandClaude Sonnet 5 22e3cfb2be Disable Swagger/ReDoc/OpenAPI JSON by default (B-42)
They enumerate the entire API surface, admin endpoints included, to
anyone who requests them. Gate them behind a new ENABLE_API_DOCS
setting (off by default) and update README/docs and BUGS.md/CLAUDE.md
open-bug counts accordingly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-27 15:34:49 +02:00
davide 4124dc08e6 Check confirmation/existence via scripthash history, not verbose replies (B-41)
poll_once and reconcile.py's existence check both called
blockchain.transaction.get(txid, verbose=True). Several Electrum server
implementations and versions reject the verbose flag outright
("verbose transactions are currently unsupported"), which would have
meant no confirmations and no reconciliation ever running against such
a server, read as a plain transport error. reconcile.py additionally
decided whether to abandon a transaction - releasing its funds - by
substring-matching the error text ("missing", "not found", ...), which
only works against ElectrumX's specific wording.

Both now ask blockchain.scripthash.get_history for the address that
owns every input of the transaction (a user's own address for a
bet/withdrawal, the pool address for a payout) and look for the txid in
the result: present with height > 0 means confirmed, present with
height <= 0 means still in the mempool, absent means the server
doesn't know it. get_history is a plain, universally-supported Electrum
method, and "not in the list" replaces the old substring-matching
entirely - no more guessing at error wording to decide whether to
release funds. History is cached per scripthash within one pass, since
every "payout" row shares the same pool address.

New app/tx/pending_address.py factors out own_address_for (the
address derivation was previously duplicated informally inside
tx/broadcast.py's signing context) so confirmation.py and reconcile.py
share one definition instead of two that could compute different
addresses for the same row.

tests/unit/test_confirmation.py and test_reconcile.py needed real User
rows and a master-key bootstrap they didn't have before, since address
derivation is now exercised for real rather than assumed. Suite grows
from 217 to 222 tests. BUGS.md moves B-41 to Previously fixed - no
Medium-severity finding remains open.
2026-07-27 15:27:58 +02:00
davide 08c566d547 Restructure bump_fee into three phases, drop float fee math (B-40)
bump_fee issued one get_transaction per input (up to 15s each) and
then a broadcast, all with the caller's DB session held open - exactly
the pattern already fixed elsewhere for the same reason (B-18's
_trigger_payout, B-31's refresh_user). Also, _prevout_amount computed
a prevout's satoshi value via round(value_coins * 100_000_000) on a
float the server reported, in a codebase that is otherwise strictly
integer-satoshi.

bump_fee now takes a session_factory and a pending_id instead of a
live session and row, with three phases: read what's needed (the
signing key, current fee rate, raw tx) and close the session before
any network call; do the chain reads, signing and broadcast with no
session open; reopen a session only to persist the outcome.
_prevout_amount now asks for the raw (non-verbose) transaction and
reads embit's parsed TransactionOutput.value directly - already an
exact integer, no float conversion involved at all.

A pending_transaction that's no longer "pending" by the time bump_fee
actually runs (it confirmed in the meantime, a normal race) is now a
quiet no-op returning None, rather than being folded into RbfBumper's
error-logging path.

Suite grows from 214 to 217 tests. BUGS.md moves B-40 to Previously
fixed, and trims its own now-stale claim that bump_fee still depended
on verbose=True (B-41) - it no longer does.
2026-07-27 15:14:58 +02:00
davide 97545ad91f Run SQLite in WAL mode with a busy_timeout (B-39)
create_async_engine had no connect_args and there was no PRAGMA
anywhere in the repo. SQLite's default rollback-journal mode lets a
writer block every reader for the duration of its transaction, and a
second writer arriving while one is already active fails immediately
with "database is locked" rather than waiting at all - realistic given
five concurrent background tasks (scheduler, confirmation poller, RBF
bumper, two reconcilers) plus every HTTP handler share one file, and
nothing previously handled that error.

app/db/base.py now registers a "connect" event on the engine that sets
journal_mode=WAL, synchronous=NORMAL and a 5-second busy_timeout on
every new DBAPI connection - applied only when the dialect is sqlite,
so a future PostgreSQL DATABASE_URL is unaffected. WAL lets readers and
writers proceed without blocking each other, and busy_timeout gives a
second writer a real window to wait instead of failing instantly.

Left out: an explicit application-level retry wrapper for "database is
locked" in the background loops, the other half of the proposed fix -
busy_timeout already gives SQLite itself several seconds to resolve
writer-vs-writer contention before ever raising, and every background
loop already catches and logs an unhandled exception before its next
scheduled tick, which is itself a retry, just not an immediate one.

Suite grows from 211 to 214 tests. BUGS.md moves B-39 to Previously
fixed.
2026-07-27 14:53:22 +02:00
davide 702b37b319 Cap SSE subscribers per client IP instead of only globally (B-38)
GET /rounds/stream capped concurrent subscribers with one global
counter (MAX_SUBSCRIBERS=500): anyone opening 500 connections degraded
every other user to polling. The comment called it a defensive cap;
it was actually the vector, since nothing stopped a single source from
exhausting it alone.

RoundEventBroadcaster now also tracks subscribers per client IP,
capped much lower (MAX_SUBSCRIBERS_PER_IP=5). Past that cap, opening
one more stream evicts that same IP's own oldest connection (woken via
a new EVICTED sentinel so the SSE generator closes it promptly) rather
than refusing the new one or letting one abusive IP crowd out unrelated
clients under the old global-only cap. The global cap stays as a
backstop against overall resource exhaustion regardless of source.

Extracted client_ip() (X-Forwarded-For, since Caddy reverse-proxies
every request) out of auth/routes.py into app/api/client_ip.py so the
login/registration throttles (B-33) and this new per-IP cap share one
definition instead of two that could drift apart.

Not implemented: enforcing the connection cap at the Caddy layer
itself, which the proposed fix also suggested - the standard Caddy
image this project uses has no such directive without a third-party
module, and building a custom image felt like a bigger, separate change
than this fix warranted.

Suite grows from 201 to 211 tests. BUGS.md moves B-38 to Previously
fixed.
2026-07-27 14:44:15 +02:00
35 changed files with 1198 additions and 317 deletions

No files matched your search

+5
View File
@@ -30,3 +30,8 @@ ADMIN_TOKEN=
# Every business/round parameter (bet amount, round duration/cooldown, min # Every business/round parameter (bet amount, round duration/cooldown, min
# amount, fee rate, RBF timeout, fee address) is configured live from the # amount, fee rate, RBF timeout, fee address) is configured live from the
# admin panel (/admin) instead of here — see docs/guida-admin.md. # admin panel (/admin) instead of here — see docs/guida-admin.md.
# Swagger/ReDoc/the raw OpenAPI JSON expose the entire API surface — admin
# endpoints included — to anyone who requests them. Off by default; set to
# true only for local development, never in production.
ENABLE_API_DOCS=false
+16 -111
View File
@@ -1,11 +1,11 @@
# Known bugs # Known bugs
A second full-codebase audit on 2026-07-27 found **25 further issues** (4 critical, 6 high, A second full-codebase audit on 2026-07-27 found **25 further issues** (4 critical, 6 high,
7 medium, 8 low), listed below as B-33 … B-49. B-25 through B-37 are fixed (see "Previously 7 medium, 8 low), listed below as B-48 … B-49. B-25 through B-47 are fixed (see "Previously
fixed" below) — no Critical-severity finding remains open; the other 12 are High/Medium/Low. fixed" below) — no Critical-, High- or Medium-severity finding remains open; the remaining 2 are
The 139-test suite was green at the time of the audit, so none of these were caught by existing Low/hygiene. The 139-test suite was green at the time of the audit, so none of these were caught
coverage — every fix lands with a regression test (the thirteen fixes so far brought the suite by existing coverage — every fix lands with a regression test (the twenty-three fixes so far
from 139 to 201). brought the suite from 139 to 248).
The recurring pattern across the open findings is worth stating once: the code is rigorous The recurring pattern across the open findings is worth stating once: the code is rigorous
about the failure modes that have actually been hit, and silent about the ones that have not. about the failure modes that have actually been hit, and silent about the ones that have not.
@@ -16,113 +16,8 @@ admin auth, single-process assumptions, no user-facing history, etc.) are docume
--- ---
## Medium
### B-38 — The 500-subscriber SSE cap is a zero-cost DoS of the realtime feature
`GET /rounds/stream` requires no authentication and each connection takes a slot on a
**global** counter (`rounds/events.py:33-38`). Anyone opening 500 connections degrades every
real user to polling. The comment describes it as a defensive cap; it is in fact the vector,
not the defence.
**Proposed fix.** Cap per client IP (and, once available, per authenticated user) rather than
globally, and evict the oldest idle subscriber instead of refusing new ones. The reverse proxy
is the right place for the connection-count limit — Caddy can enforce it before the request
reaches the app.
### B-39 — SQLite with no WAL, no `busy_timeout`, and five concurrent writer tasks
`db/base.py:6` calls `create_async_engine(settings.database_url)` with no `connect_args`, and
there is no `PRAGMA` anywhere in the repo (verified by grep). Without `journal_mode=WAL`
readers block writers, and the concurrent writers are five background tasks plus every HTTP
handler. `database is locked` under load is realistic, and nothing handles it.
**Proposed fix.** Set `journal_mode=WAL`, `synchronous=NORMAL` and a `busy_timeout` of a few
seconds on connect (a `connect` event listener on the engine, applied only for the SQLite
dialect), and retry `OperationalError: database is locked` in the background loops. Longer
term this is an argument for PostgreSQL, which the single-process constraints in CLAUDE.md
also point at.
### B-40 — `bump_fee` holds a DB session open across N network calls
`tx/broadcast.py:80` issues one `get_transaction` **per input** (up to 15s each) and then a
`broadcast`, all with the session open. This is precisely the pattern B-18 removed from
`_trigger_payout` via its three-phase structure; it survives here.
Side note in the same function: `_prevout_amount` does `round(value_coins * 100_000_000)` on a
float from the server — acceptable at these magnitudes, but it is floating-point money
arithmetic in a codebase that is otherwise strictly integer-satoshi.
**Proposed fix.** Restructure into the same three phases: read what is needed and close the
session, do the chain work, then reopen to persist. For the float: prefer the raw (non-verbose)
transaction and parse the output value as an integer with `embit`, which is what
`reconcile.py:_release_inputs` already does for inputs.
### B-41 — All confirmation logic depends on `verbose=True`, which is not universally supported
`poll_once`, `reconcile._tx_exists_on_chain` and `bump_fee` all call
`blockchain.transaction.get(txid, True)`. Several Electrum server implementations and versions
reject the verbose flag ("verbose transactions are currently unsupported"). Falling back onto
such a server means **no confirmations, no reconciliation, no bumps** — and the code would read
that as a transport error and stay silent.
Related: `reconcile.py:83` decides whether to **abandon a transaction** by substring-matching
the error text (`"missing"`, `"not found"`, `"no such"`, `"unknown"`). It works against
ElectrumX; it is fragile as the basis for a decision that releases funds.
**Proposed fix.** Use `blockchain.transaction.get_merkle` (or the scripthash history) for
confirmation and existence checks — both are portable and give the confirming height directly.
Probe verbose support once at connect time and record it on the client, so an unsupported
server is detected loudly at session start rather than silently mid-operation.
---
## Low / hygiene ## Low / hygiene
### B-42 — `/docs` exposed in production
FastAPI mounts Swagger by default, so the entire API surface — `/admin` included — is publicly
enumerable. The README advertises it.
**Fix:** `docs_url=None, redoc_url=None, openapi_url=None` in production (env-gated), or place
them behind `require_admin`.
### B-43 — No HTTP security headers
The [Caddyfile](Caddyfile) sets no CSP, no `X-Frame-Options`/`frame-ancestors`, and no HSTS
(Caddy does not add it on its own). The JWT lives in `localStorage`, so any XSS exfiltrates
it, and the page is iframeable.
**Fix:** a `header` block in the Caddyfile with `Strict-Transport-Security`,
`X-Content-Type-Options: nosniff`, `Referrer-Policy` and a CSP tight enough for two static
pages with no external assets (`default-src 'self'`).
### B-44 — README and CLAUDE.md contradict each other
The README says to run `uvicorn --reload` directly and
`docker compose run --rm app python scripts/generate_master_key.py`; CLAUDE.md says explicitly
that neither is supported. Whoever opens the repo reads the README first.
**Fix:** align the README's Quick start with the Docker-only workflow documented in
CLAUDE.md and `docs/setup.md`.
### B-45 — Unvalidated and unpaginated admin list endpoints
`limit: int = 50` on `/admin/rounds` and `/admin/audit-log` has no bounds (`-1` means
"everything" on SQLite), and `/admin/pending-transactions` has no limit at all — it grows
without end.
**Fix:** `Query(default=50, ge=1, le=500)` on both, and the same treatment plus a status filter
on the pending-transaction list.
### B-46 — `secrets.compare_digest` on a `str` raises on non-ASCII input
`api/routes/admin.py:27` raises `TypeError` — a 500 instead of a 403 — when the header contains
non-ASCII characters.
**Fix:** compare the UTF-8 encoded bytes of both sides.
### B-47 — Unbounded `String` columns for large text
`raw_tx_hex` (`db/models.py:146`) and `payload_json` (`:178`) should be `Text`. It works on
SQLite and PostgreSQL and breaks elsewhere.
**Fix:** switch both to `Text` in a migration.
### B-48 — No cap on input count in `select_utxos` ### B-48 — No cap on input count in `select_utxos`
A user with hundreds of small UTXOs builds a huge transaction whose fee — deducted from the bet A user with hundreds of small UTXOs builds a huge transaction whose fee — deducted from the bet
@@ -150,15 +45,25 @@ already does.
- **B-29** — a UTXO absent from one server's `listunspent` was marked spent immediately, irreversibly, on a single unauthenticated reply - **B-29** — a UTXO absent from one server's `listunspent` was marked spent immediately, irreversibly, on a single unauthenticated reply
- **B-30** — a lost scripthash subscription meant a user's deposits were never credited, with no periodic safety net - **B-30** — a lost scripthash subscription meant a user's deposits were never credited, with no periodic safety net
- **B-31** — resubscribing on reconnect ran serially before anything else started, freezing the chain tip (and so an in-flight draw) for the whole sweep - **B-31** — resubscribing on reconnect ran serially before anything else started, freezing the chain tip (and so an in-flight draw) for the whole sweep
- **B-42** — Swagger/ReDoc/the raw OpenAPI JSON enumerated the entire API surface, admin endpoints included, to anyone who requested them; now off by default and gated behind `ENABLE_API_DOCS`
- **B-43** — the Caddyfile sent no CSP, no `X-Frame-Options`/`frame-ancestors`, and no HSTS, on a page whose JWT lives in `localStorage`
- **B-44** — README's Quick start documented a bare `uvicorn --reload` workflow, and `docs/running-the-server.md` still had a matching "Locale / venv" section, both contradicting CLAUDE.md's Docker-only policy
- **B-45** — `/admin/rounds`/`/admin/audit-log`'s `limit` had no bounds (`-1` means "everything" on SQLite), and `/admin/pending-transactions` had no limit or status filter at all
- **B-46** — `secrets.compare_digest` on a `str` raises `TypeError` on non-ASCII input, turning an invalid admin token with non-ASCII characters into a 500 instead of a 403
- **B-47** — `raw_tx_hex` and `payload_json` were unbounded `String` columns (`VARCHAR` with no length) — fine on SQLite/PostgreSQL, rejected by backends like MySQL that require a length
- **B-32** — an RBF bump could retry forever below BIP125's relay-mandated minimum fee delta, with no ceiling on the fee rate either - **B-32** — an RBF bump could retry forever below BIP125's relay-mandated minimum fee delta, with no ceiling on the fee rate either
- **B-33** — `POST /auth/login` had no rate limiting, so a password could be brute-forced against an enumerable username list - **B-33** — `POST /auth/login` had no rate limiting, so a password could be brute-forced against an enumerable username list
- **B-34** — password change/reset didn't invalidate already-issued JWTs, so a stolen token survived a change meant to lock it out - **B-34** — password change/reset didn't invalidate already-issued JWTs, so a stolen token survived a change meant to lock it out
- **B-35** — API timestamps round-tripped as naive datetimes, so the frontend parsed them as local time instead of UTC - **B-35** — API timestamps round-tripped as naive datetimes, so the frontend parsed them as local time instead of UTC
- **B-36** — a stalled draw wait had no timeout, no log, and no audit trail, so a frozen round showed nothing in `/admin` - **B-36** — a stalled draw wait had no timeout, no log, and no audit trail, so a frozen round showed nothing in `/admin`
- **B-37** — a withdrawal covered by unconfirmed change answered "insufficient balance" instead of distinguishing it from actually having no funds - **B-37** — a withdrawal covered by unconfirmed change answered "insufficient balance" instead of distinguishing it from actually having no funds
- **B-38** — the SSE subscriber cap was global, so one client opening enough connections degraded every other user to polling
- **B-39** — SQLite ran without WAL or a `busy_timeout`, so a writer could block every reader and a second writer failed immediately instead of waiting
- **B-40** — `bump_fee` held a DB session open across N slow network calls, and computed a prevout's value from a server-reported float instead of an exact integer
- **B-41** — confirmation/reconciliation depended on a verbose `blockchain.transaction.get` reply many Electrum servers reject, and abandonment relied on fragile substring-matching of an error message
See git history for the fix-by-fix breakdown (commits `f13f685`, `50a43ae`, `933760e`, and the See git history for the fix-by-fix breakdown (commits `f13f685`, `50a43ae`, `933760e`, and the
B-28/B-29/B-30/B-31/B-32/B-33/B-34/B-35/B-36/B-37 fixes). Suite grew from 139 to 201 tests over the thirteen. B-28/B-29/B-30/B-31/B-32/B-33/B-34/B-35/B-36/B-37/B-38/B-39/B-40/B-41/B-42/B-43/B-44/B-45/B-46/B-47 fixes). Suite grew from 139 to 248 tests over the twenty-three.
A full-codebase audit on 2026-07-26 (commit `d4e0974`) found 24 bugs across every Python A full-codebase audit on 2026-07-26 (commit `d4e0974`) found 24 bugs across every Python
module under `app/`, both static frontends, and the Docker/Caddy deployment — 5 critical, module under `app/`, both static frontends, and the Docker/Caddy deployment — 5 critical,
+4 -4
View File
@@ -12,11 +12,11 @@ All 10 stages of the original build order are code-complete and unit-tested —
Verified on mainnet with real money: registration + address derivation, deposit crediting (1-conf), a real 10 PLM bet (broadcast → confirmed → change credited back), and one full round cycle (close → draw on a real block hash → 70/30 payout with sat math checked against the broadcast tx → confirmation → close → next round auto-opened). **Withdrawal and the RBF bump path have never been exercised against a live broadcast** — unit-tested only. Verified on mainnet with real money: registration + address derivation, deposit crediting (1-conf), a real 10 PLM bet (broadcast → confirmed → change credited back), and one full round cycle (close → draw on a real block hash → 70/30 payout with sat math checked against the broadcast tx → confirmation → close → next round auto-opened). **Withdrawal and the RBF bump path have never been exercised against a live broadcast** — unit-tested only.
**Read [BUGS.md](BUGS.md) before trusting any behaviour here.** Two audits: 2026-07-26 found 24 bugs (5 critical), all fixed; 2026-07-27 found 25 more (B-25 … B-49), of which **12 are still open** — no Critical or High remains, only Medium/Low: no WAL/`busy_timeout` under five concurrent SQLite writer tasks (B-39), a 500-subscriber SSE cap that doubles as a cheap DoS of the realtime feature (B-38), among others. BUGS.md is the live open list with a proposed fix per finding; "Known gaps" at the end of this file is for limitations accepted **by design** instead. Don't fix a BUGS.md item silently as a side effect of other work — each fix lands with its own regression test. **Read [BUGS.md](BUGS.md) before trusting any behaviour here.** Two audits: 2026-07-26 found 24 bugs (5 critical), all fixed; 2026-07-27 found 25 more (B-25 … B-49), of which **2 are still open** — no Critical, High or Medium remains, only Low/hygiene: no cap on input count in `select_utxos` (B-48) and rollback paths not publishing an SSE update (B-49). BUGS.md is the live open list with a proposed fix per finding; "Known gaps" at the end of this file is for limitations accepted **by design** instead. Don't fix a BUGS.md item silently as a side effect of other work — each fix lands with its own regression test.
Before writing code, read the "Architecture" section below in full plus the diagrams in [flowchart/](flowchart/): [platform-overview.mmd](flowchart/platform-overview.mmd) (the 5-phase flow) and [round-lifecycle.mmd](flowchart/round-lifecycle.mmd) (the round/draw lifecycle). Every node **and edge label** (conditions, retries, loops) is a behaviour that must be implemented as described. Regenerate the companion PDFs with `flowchart/render-pdf.sh <file>.mmd` after editing either. Before writing code, read the "Architecture" section below in full plus the diagrams in [flowchart/](flowchart/): [platform-overview.mmd](flowchart/platform-overview.mmd) (the 5-phase flow) and [round-lifecycle.mmd](flowchart/round-lifecycle.mmd) (the round/draw lifecycle). Every node **and edge label** (conditions, retries, loops) is a behaviour that must be implemented as described. Regenerate the companion PDFs with `flowchart/render-pdf.sh <file>.mmd` after editing either.
Human-facing guides are in [docs/](docs/), in Italian by explicit request (an exception to the English-only rule): [setup.md](docs/setup.md), [running-the-server.md](docs/running-the-server.md), [guida-utente.md](docs/guida-utente.md), [guida-admin.md](docs/guida-admin.md). **[README.md](README.md)'s Quick start is stale** (bare `uvicorn --reload`, `docker compose run … generate_master_key.py` — neither is supported; B-44); this file and `docs/setup.md` are authoritative. Human-facing guides are in [docs/](docs/), in Italian by explicit request (an exception to the English-only rule): [setup.md](docs/setup.md), [running-the-server.md](docs/running-the-server.md), [guida-utente.md](docs/guida-utente.md), [guida-admin.md](docs/guida-admin.md). README's Quick start and `docs/running-the-server.md` are Docker-only, matching this file — a bare `uvicorn --reload` workflow was removed from both (B-44).
## Commands ## Commands
@@ -55,7 +55,7 @@ docker compose down
`SITE_ADDRESS` unset → `localhost`, Caddy issues a self-signed cert from its internal CA (browser warning on first visit is expected; `curl -k`). `SITE_ADDRESS=lottery.example.com docker compose up -d` → real Let's Encrypt cert, automatically renewed (needs DNS pointing here and ports 80+443 reachable). `SITE_ADDRESS` unset → `localhost`, Caddy issues a self-signed cert from its internal CA (browser warning on first visit is expected; `curl -k`). `SITE_ADDRESS=lottery.example.com docker compose up -d` → real Let's Encrypt cert, automatically renewed (needs DNS pointing here and ports 80+443 reachable).
The `Caddyfile` sets **no** security headers — no CSP, HSTS or `X-Frame-Options` (B-43). `restart: unless-stopped` on `app` means a mid-round crash auto-restarts: `closing` and `paying_out` resume on their own, `drawing` does not (see Known gaps). The `Caddyfile` sends baseline security headers — HSTS, `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy`, and a CSP scoped to `default-src 'self'` plus the Google Fonts `@import` in `style.css`/`admin.css`. `script-src`/`style-src` need `'unsafe-inline'` because both SPAs use inline `onclick` handlers and `style=""` attributes throughout — removing those is a separate, larger refactor, not a header change. `restart: unless-stopped` on `app` means a mid-round crash auto-restarts: `closing` and `paying_out` resume on their own, `drawing` does not (see Known gaps).
## Tech stack ## Tech stack
@@ -233,7 +233,7 @@ Explicit design choices, not derivable from any single file — respect them:
## Known gaps / TODO ## Known gaps / TODO
Accepted **by design**. For actual bugs see [BUGS.md](BUGS.md) (18 open) — not duplicated here. Accepted **by design**. For actual bugs see [BUGS.md](BUGS.md) (2 open) — not duplicated here.
- **`drawing` doesn't resume after a restart.** `_tick()` handles `open`, `closing` and `paying_out` (the last via `_retry_payout_if_due`); nothing re-enters `_wait_for_next_block` after a crash. That wait is unbounded by design (the draw's entropy genuinely depends on a future block) but no longer silent — past `_DRAW_STALL_THRESHOLD_SECONDS` it logs progress and writes a `draw_stalled` audit entry, and `GET /rounds/current`'s `draw_waiting_since` surfaces it live (B-36). Restart-resumption itself remains the last prerequisite for running unattended. - **`drawing` doesn't resume after a restart.** `_tick()` handles `open`, `closing` and `paying_out` (the last via `_retry_payout_if_due`); nothing re-enters `_wait_for_next_block` after a crash. That wait is unbounded by design (the draw's entropy genuinely depends on a future block) but no longer silent — past `_DRAW_STALL_THRESHOLD_SECONDS` it logs progress and writes a `draw_stalled` audit entry, and `GET /rounds/current`'s `draw_waiting_since` surfaces it live (B-36). Restart-resumption itself remains the last prerequisite for running unattended.
- **RBF handles one shape only**: a single change output, back to the tx's own sender, big enough to absorb the increase. No extra-input fallback — an exact-amount tx or too-small change raises `RbfError`. Not permanent, though: an unbumpable tx that never confirms is eventually abandoned and its UTXOs released. - **RBF handles one shape only**: a single change output, back to the tx's own sender, big enough to absorb the increase. No extra-input fallback — an exact-amount tx or too-small change raises `RbfError`. Not permanent, though: an unbumpable tx that never confirms is eventually abandoned and its UTXOs released.
+16
View File
@@ -13,5 +13,21 @@
not path /rounds/stream not path /rounds/stream
} }
encode @not_sse gzip encode @not_sse gzip
# B-43: Caddy adds none of these on its own. The JWT lives in
# localStorage, so any XSS exfiltrates it — CSP is the main mitigation.
# script-src/style-src need 'unsafe-inline' because both SPAs
# (app/static/index.html, admin.html) use inline onclick handlers and
# style="" attributes throughout; removing those is a separate,
# larger refactor, not a header change. fonts.googleapis.com/gstatic.com
# are the one external asset (the Google Fonts @import in style.css/admin.css).
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
X-Frame-Options "DENY"
Referrer-Policy "strict-origin-when-cross-origin"
Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; font-src 'self' https://fonts.gstatic.com; img-src 'self'; connect-src 'self'; base-uri 'self'; form-action 'self'; frame-ancestors 'none'; object-src 'none'"
}
reverse_proxy app:8123 reverse_proxy app:8123
} }
+16 -17
View File
@@ -13,31 +13,30 @@ production-ready.
## Quick start ## Quick start
The server always runs via Docker (app + Caddy reverse proxy with automatic
TLS) — in dev and production alike, with only `SITE_ADDRESS` differing
between the two. There's no supported way to run `uvicorn` directly; the
venv is only for local tooling (tests, Alembic migrations, the one-time key
scripts) — see [CLAUDE.md](CLAUDE.md#commands).
```bash ```bash
cp .env.example .env # then fill in the generated secrets, see docs/setup.md cp .env.example .env # then fill in the generated secrets, see docs/setup.md
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
PYTHONPATH=. python scripts/generate_master_key.py
alembic upgrade head
uvicorn app.main:app --reload --port 8123
```
Open `http://127.0.0.1:8123/` for the test UI, `http://127.0.0.1:8123/admin`
for the admin dashboard, `http://127.0.0.1:8123/docs` for the interactive API
docs.
Or run the whole stack (app + Caddy reverse proxy with automatic TLS) via
Docker:
```bash
mkdir -p data/db data/keys data/logs mkdir -p data/db data/keys data/logs
docker compose run --rm app python scripts/generate_master_key.py docker compose run --rm app python scripts/generate_master_key.py
docker compose up -d --build docker compose up -d --build
``` ```
Open `https://localhost/` for the test UI, `https://localhost/admin` for the
admin dashboard (a self-signed-certificate warning on first visit is
expected in dev — accept it, or use `curl -k`). The interactive API docs at
`/docs` are disabled by default (they'd otherwise expose the whole API
surface, admin endpoints included) — set `ENABLE_API_DOCS=true` in `.env` to
enable them.
See [docs/setup.md](docs/setup.md) and See [docs/setup.md](docs/setup.md) and
[docs/running-the-server.md](docs/running-the-server.md) for the full [docs/running-the-server.md](docs/running-the-server.md) for the full
walkthrough (both workflows, dev vs. production TLS). walkthrough (secrets, master key generation, production TLS with a real
domain).
## Documentation ## Documentation
@@ -62,7 +61,7 @@ python -m pytest # all tests
python -m pytest tests/unit/test_hd.py # one file python -m pytest tests/unit/test_hd.py # one file
``` ```
76 unit tests cover HD derivation, PSBT building, the Electrum client, bets, 232 unit tests cover HD derivation, PSBT building, the Electrum client, bets,
deposits, withdrawals, the round/draw engine, RBF fee-bumping, admin config, deposits, withdrawals, the round/draw engine, RBF fee-bumping, admin config,
the pending-inclusive balance calculation, and the SSE push channel. No the pending-inclusive balance calculation, and the SSE push channel. No
automated integration tests against a live Electrum connection — mainnet automated integration tests against a live Electrum connection — mainnet
+17
View File
@@ -0,0 +1,17 @@
from fastapi import Request
def client_ip(request: Request) -> str:
"""The caller's real IP, from Caddy's X-Forwarded-For (see Caddyfile) —
request.client.host would otherwise be the reverse proxy's own address, not
the caller's. Falls back to request.client.host only if the header is
somehow missing (e.g. the app container hit directly, bypassing Caddy).
Shared by the login/registration throttles (B-33) and the SSE per-IP
subscriber cap (B-38) so the two can't drift into different notions of
"the client's IP".
"""
forwarded = request.headers.get("x-forwarded-for")
if forwarded:
return forwarded.split(",")[0].strip()
return request.client.host if request.client else "unknown"
+14 -7
View File
@@ -1,7 +1,7 @@
import json import json
import secrets import secrets
from fastapi import APIRouter, Depends, Header, HTTPException, status from fastapi import APIRouter, Depends, Header, HTTPException, Query, status
from pydantic import BaseModel, Field, field_validator from pydantic import BaseModel, Field, field_validator
from sqlalchemy import select from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
@@ -26,7 +26,9 @@ async def require_admin(x_admin_token: str = Header(default="")) -> None:
# anyone on an instance that never configured a token. # anyone on an instance that never configured a token.
if not settings.admin_token: if not settings.admin_token:
raise HTTPException(status.HTTP_403_FORBIDDEN, "invalid admin token") raise HTTPException(status.HTTP_403_FORBIDDEN, "invalid admin token")
if not secrets.compare_digest(x_admin_token, settings.admin_token): # compare_digest raises TypeError on a str containing non-ASCII characters
# (B-46) -- comparing the UTF-8 bytes instead accepts any input safely.
if not secrets.compare_digest(x_admin_token.encode(), settings.admin_token.encode()):
raise HTTPException(status.HTTP_403_FORBIDDEN, "invalid admin token") raise HTTPException(status.HTTP_403_FORBIDDEN, "invalid admin token")
@@ -242,7 +244,9 @@ class AdminRoundResponse(BaseModel):
@router.get("/rounds", response_model=list[AdminRoundResponse], dependencies=[Depends(require_admin)]) @router.get("/rounds", response_model=list[AdminRoundResponse], dependencies=[Depends(require_admin)])
async def list_rounds(session: AsyncSession = Depends(get_session), limit: int = 50) -> list[AdminRoundResponse]: async def list_rounds(
session: AsyncSession = Depends(get_session), limit: int = Query(default=50, ge=1, le=500)
) -> list[AdminRoundResponse]:
rounds = (await session.scalars(select(Round).order_by(Round.id.desc()).limit(limit))).all() rounds = (await session.scalars(select(Round).order_by(Round.id.desc()).limit(limit))).all()
winner_ids = {r.winner_user_id for r in rounds if r.winner_user_id is not None} winner_ids = {r.winner_user_id for r in rounds if r.winner_user_id is not None}
winners = {} winners = {}
@@ -282,7 +286,7 @@ class AdminAuditLogResponse(BaseModel):
"/audit-log", response_model=list[AdminAuditLogResponse], dependencies=[Depends(require_admin)] "/audit-log", response_model=list[AdminAuditLogResponse], dependencies=[Depends(require_admin)]
) )
async def list_audit_log( async def list_audit_log(
session: AsyncSession = Depends(get_session), limit: int = 200 session: AsyncSession = Depends(get_session), limit: int = Query(default=200, ge=1, le=500)
) -> list[AdminAuditLogResponse]: ) -> list[AdminAuditLogResponse]:
entries = (await session.scalars(select(AuditLog).order_by(AuditLog.id.desc()).limit(limit))).all() entries = (await session.scalars(select(AuditLog).order_by(AuditLog.id.desc()).limit(limit))).all()
return [ return [
@@ -319,10 +323,13 @@ class AdminPendingTransactionResponse(BaseModel):
) )
async def list_pending_transactions( async def list_pending_transactions(
session: AsyncSession = Depends(get_session), session: AsyncSession = Depends(get_session),
limit: int = Query(default=50, ge=1, le=500),
status_filter: str | None = Query(default=None, alias="status"),
) -> list[AdminPendingTransactionResponse]: ) -> list[AdminPendingTransactionResponse]:
entries = ( query = select(PendingTransaction).order_by(PendingTransaction.id.desc())
await session.scalars(select(PendingTransaction).order_by(PendingTransaction.id.desc())) if status_filter is not None:
).all() query = query.where(PendingTransaction.status == status_filter)
entries = (await session.scalars(query.limit(limit))).all()
return [ return [
AdminPendingTransactionResponse( AdminPendingTransactionResponse(
id=p.id, id=p.id,
+11 -3
View File
@@ -8,12 +8,13 @@ from pydantic import BaseModel
from sqlalchemy import func, select from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.api.client_ip import client_ip
from app.api.timeutil import isoformat_utc from app.api.timeutil import isoformat_utc
from app.auth.dependencies import get_optional_user from app.auth.dependencies import get_optional_user
from app.db.models import RoundParticipant, User from app.db.models import RoundParticipant, User
from app.db.session import get_session from app.db.session import get_session
from app.rounds.config import get_round_config from app.rounds.config import get_round_config
from app.rounds.events import RoundEventCapacityError, broadcaster from app.rounds.events import EVICTED, RoundEventCapacityError, broadcaster
from app.rounds.service import get_active_round from app.rounds.service import get_active_round
router = APIRouter(prefix="/rounds", tags=["rounds"]) router = APIRouter(prefix="/rounds", tags=["rounds"])
@@ -46,9 +47,14 @@ async def round_stream(request: Request) -> Response:
That's also what happens past MAX_SUBSCRIBERS (app/rounds/events.py): this That's also what happens past MAX_SUBSCRIBERS (app/rounds/events.py): this
returns 503 rather than opening a stream, and the browser's EventSource returns 503 rather than opening a stream, and the browser's EventSource
just retries later while the frontend keeps working off polling meanwhile. just retries later while the frontend keeps working off polling meanwhile.
Concurrent streams are additionally capped per client IP (B-38): past
MAX_SUBSCRIBERS_PER_IP, opening one more evicts that IP's own oldest
connection rather than refusing the new one or letting a single source
exhaust the global cap and degrade every other user.
""" """
try: try:
queue = broadcaster.subscribe() queue = broadcaster.subscribe(client_ip(request))
except RoundEventCapacityError: except RoundEventCapacityError:
return JSONResponse(status_code=503, content={"detail": "too many concurrent update streams"}) return JSONResponse(status_code=503, content={"detail": "too many concurrent update streams"})
@@ -59,7 +65,9 @@ async def round_stream(request: Request) -> Response:
if await request.is_disconnected(): if await request.is_disconnected():
break break
try: try:
await asyncio.wait_for(queue.get(), timeout=_SSE_DISCONNECT_CHECK_SECONDS) item = await asyncio.wait_for(queue.get(), timeout=_SSE_DISCONNECT_CHECK_SECONDS)
if item is EVICTED:
break # this IP opened another stream past its per-IP cap
yield "event: update\ndata: {}\n\n".format(json.dumps({})) yield "event: update\ndata: {}\n\n".format(json.dumps({}))
ticks_since_keepalive = 0 ticks_since_keepalive = 0
except asyncio.TimeoutError: except asyncio.TimeoutError:
+1 -10
View File
@@ -4,6 +4,7 @@ from sqlalchemy import func, select
from sqlalchemy.exc import IntegrityError from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.api.client_ip import client_ip as _client_ip
from app.api.errors import http_error from app.api.errors import http_error
from app.auth.rate_limit import AuthRateLimiters from app.auth.rate_limit import AuthRateLimiters
from app.auth.security import MIN_PASSWORD_LENGTH, create_access_token, hash_password, verify_password from app.auth.security import MIN_PASSWORD_LENGTH, create_access_token, hash_password, verify_password
@@ -34,16 +35,6 @@ def _rate_limiters(request: Request) -> AuthRateLimiters:
return request.app.state.auth_rate_limiters return request.app.state.auth_rate_limiters
def _client_ip(request: Request) -> str:
# Caddy (see Caddyfile) reverse-proxies every request, so request.client.host
# is the proxy's address, not the caller's — fall back to it only if the
# header is somehow missing (e.g. hitting the app container directly).
forwarded = request.headers.get("x-forwarded-for")
if forwarded:
return forwarded.split(",")[0].strip()
return request.client.host if request.client else "unknown"
def _rate_limited_error(retry_after: float): def _rate_limited_error(retry_after: float):
return http_error( return http_error(
status.HTTP_429_TOO_MANY_REQUESTS, status.HTTP_429_TOO_MANY_REQUESTS,
+5
View File
@@ -29,6 +29,11 @@ class Settings(BaseSettings):
jwt_expire_minutes: int = 60 * 24 jwt_expire_minutes: int = 60 * 24
admin_token: str = "" admin_token: str = ""
# Swagger/ReDoc/OpenAPI JSON expose the entire API surface (admin endpoints
# included) to anyone who requests them. Off by default (B-42) — set to true
# only for local development, never in production.
enable_api_docs: bool = False
# Every business/round parameter (bet amount, round duration/cooldown, # Every business/round parameter (bet amount, round duration/cooldown,
# min amount, fee rate, RBF timeout, fee address) lives in the round_config # min amount, fee rate, RBF timeout, fee address) lives in the round_config
# DB table instead (app/db/models.py RoundConfig, app/rounds/config.py) — # DB table instead (app/db/models.py RoundConfig, app/rounds/config.py) —
+36 -1
View File
@@ -1,9 +1,44 @@
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine from sqlalchemy import event
from sqlalchemy.ext.asyncio import AsyncEngine, async_sessionmaker, create_async_engine
from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import DeclarativeBase
from app.config import settings from app.config import settings
# How long a writer waits for a lock held by another writer before SQLite raises
# "database is locked" (B-39). A few seconds is enough to ride out this app's own
# five concurrent background tasks (scheduler, confirmation poller, RBF bumper,
# two reconcilers) plus HTTP handlers briefly overlapping a write.
_SQLITE_BUSY_TIMEOUT_MS = 5000
def _register_sqlite_pragmas(target_engine: AsyncEngine) -> None:
"""Without WAL, SQLite's default (rollback-journal) mode lets a writer block
every reader for the duration of its transaction, and a second writer arriving
while one is already active fails immediately rather than waiting at all —
realistic under this app's concurrency, and nothing previously handled it.
WAL lets readers and writers proceed without blocking each other, and
busy_timeout gives a second writer a real window to wait for the first
instead of an instant `OperationalError`.
No-op for any dialect other than sqlite (e.g. a future PostgreSQL
DATABASE_URL), which neither needs nor understands these pragmas.
"""
if target_engine.dialect.name != "sqlite":
return
@event.listens_for(target_engine.sync_engine, "connect")
def _set_sqlite_pragmas(dbapi_connection, connection_record) -> None:
cursor = dbapi_connection.cursor()
try:
cursor.execute("PRAGMA journal_mode=WAL")
cursor.execute("PRAGMA synchronous=NORMAL")
cursor.execute(f"PRAGMA busy_timeout={_SQLITE_BUSY_TIMEOUT_MS}")
finally:
cursor.close()
engine = create_async_engine(settings.database_url) engine = create_async_engine(settings.database_url)
_register_sqlite_pragmas(engine)
AsyncSessionLocal = async_sessionmaker(engine, expire_on_commit=False) AsyncSessionLocal = async_sessionmaker(engine, expire_on_commit=False)
+3 -3
View File
@@ -1,6 +1,6 @@
from datetime import datetime, timezone from datetime import datetime, timezone
from sqlalchemy import BigInteger, ForeignKey, Index, String, UniqueConstraint, text from sqlalchemy import BigInteger, ForeignKey, Index, String, Text, UniqueConstraint, text
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from app.db.base import Base from app.db.base import Base
@@ -154,7 +154,7 @@ class PendingTransaction(Base):
user_id: Mapped[int | None] = mapped_column(ForeignKey("users.id"), default=None) user_id: Mapped[int | None] = mapped_column(ForeignKey("users.id"), default=None)
current_txid: Mapped[str] = mapped_column(String(64)) current_txid: Mapped[str] = mapped_column(String(64))
fee_rate_sat_vb: Mapped[int] fee_rate_sat_vb: Mapped[int]
raw_tx_hex: Mapped[str] = mapped_column(String) raw_tx_hex: Mapped[str] = mapped_column(Text)
# The *first* broadcast — never rewritten by a bump — since this is what the # The *first* broadcast — never rewritten by a bump — since this is what the
# reconciler's abandon-after-N-hours grace period (app/tx/reconcile.py) measures # reconciler's abandon-after-N-hours grace period (app/tx/reconcile.py) measures
# from. Bumping used to overwrite this field, which reset that clock on every # from. Bumping used to overwrite this field, which reset that clock on every
@@ -193,7 +193,7 @@ class AuditLog(Base):
id: Mapped[int] = mapped_column(primary_key=True) id: Mapped[int] = mapped_column(primary_key=True)
event_type: Mapped[str] = mapped_column(String(32)) event_type: Mapped[str] = mapped_column(String(32))
payload_json: Mapped[str] = mapped_column(String) payload_json: Mapped[str] = mapped_column(Text)
user_id: Mapped[int | None] = mapped_column(ForeignKey("users.id"), default=None) user_id: Mapped[int | None] = mapped_column(ForeignKey("users.id"), default=None)
round_id: Mapped[int | None] = mapped_column(ForeignKey("rounds.id"), default=None) round_id: Mapped[int | None] = mapped_column(ForeignKey("rounds.id"), default=None)
created_at: Mapped[datetime] = mapped_column(default=utcnow) created_at: Mapped[datetime] = mapped_column(default=utcnow)
+11
View File
@@ -169,6 +169,17 @@ class ElectrumClient:
async def listunspent(self, scripthash: str) -> list[dict]: async def listunspent(self, scripthash: str) -> list[dict]:
return await self.request("blockchain.scripthash.listunspent", [scripthash]) return await self.request("blockchain.scripthash.listunspent", [scripthash])
async def get_history(self, scripthash: str) -> list[dict]:
"""Every transaction touching `scripthash`, each as {"tx_hash", "height"} —
height > 0 means confirmed at that height, height <= 0 means still in the
mempool. Used instead of blockchain.transaction.get's verbose=True mode
for confirmation/existence checks (B-41): several Electrum server
implementations and versions reject the verbose flag outright ("verbose
transactions are currently unsupported"), while get_history is a plain,
universally-supported method every server must implement.
"""
return await self.request("blockchain.scripthash.get_history", [scripthash])
async def broadcast(self, raw_tx_hex: str) -> str: async def broadcast(self, raw_tx_hex: str) -> str:
return await self.request("blockchain.transaction.broadcast", [raw_tx_hex]) return await self.request("blockchain.transaction.broadcast", [raw_tx_hex])
+10 -1
View File
@@ -84,7 +84,16 @@ async def lifespan(app: FastAPI):
await listener.client.close() await listener.client.close()
app = FastAPI(title="PLM Lottery", lifespan=lifespan) # Swagger/ReDoc/the raw OpenAPI JSON enumerate the entire API surface, admin
# endpoints included, to anyone who requests them (B-42) — disabled unless
# ENABLE_API_DOCS is explicitly set, which should only happen in development.
app = FastAPI(
title="PLM Lottery",
lifespan=lifespan,
docs_url="/docs" if settings.enable_api_docs else None,
redoc_url="/redoc" if settings.enable_api_docs else None,
openapi_url="/openapi.json" if settings.enable_api_docs else None,
)
app.include_router(auth_router) app.include_router(auth_router)
app.include_router(users_router) app.include_router(users_router)
app.include_router(bets_router) app.include_router(bets_router)
+54 -14
View File
@@ -1,16 +1,31 @@
import asyncio import asyncio
from collections import defaultdict
# Defensive cap on concurrent SSE subscribers. Expected load is on the order of # Defensive backstop on concurrent SSE subscribers overall, regardless of source
# ~100 concurrent users; this is set well above that so it never engages under # — expected load is on the order of ~100 concurrent users, so this is set well
# normal use — it exists purely so a runaway/DoS-y number of open connections # above that. The real defense against a single abusive source is the per-IP cap
# degrades (new connections fall back to polling, see round_stream()) instead # below (B-38): a global-only cap was trivially exhausted by one client opening
# of growing the in-memory subscriber set without bound. Revisit this number if # MAX_SUBSCRIBERS connections, degrading every other user to polling — the
# expected concurrency grows well past it. # comment used to call it "defensive"; it was actually the vector.
MAX_SUBSCRIBERS = 500 MAX_SUBSCRIBERS = 500
# How many concurrent streams a single client IP may hold. Deliberately small —
# a real browser tab needs at most one, occasionally two briefly across a
# reload — since this bounds one source's share of the global capacity, not a
# legitimate per-user concurrency limit.
MAX_SUBSCRIBERS_PER_IP = 5
# Put on a to-be-evicted subscriber's queue (B-38) to wake its generator
# (app/api/routes/rounds.py:round_stream) promptly so it closes the connection
# instead of lingering, silently uncounted, until the client's own network
# timeout or the next keep-alive tick.
EVICTED = object()
class RoundEventCapacityError(Exception): class RoundEventCapacityError(Exception):
"""Raised by subscribe() when MAX_SUBSCRIBERS is already reached.""" """Raised by subscribe() when MAX_SUBSCRIBERS — the global backstop — is
already reached. The per-IP cap never raises this; it evicts instead (see
subscribe())."""
class RoundEventBroadcaster: class RoundEventBroadcaster:
@@ -26,22 +41,47 @@ class RoundEventBroadcaster:
would need a shared channel (e.g. Redis pub/sub) instead. would need a shared channel (e.g. Redis pub/sub) instead.
""" """
def __init__(self, max_subscribers: int = MAX_SUBSCRIBERS): def __init__(self, max_subscribers: int = MAX_SUBSCRIBERS, max_per_ip: int = MAX_SUBSCRIBERS_PER_IP):
self._subscribers: set[asyncio.Queue] = set() self._ip_by_queue: dict[asyncio.Queue, str] = {}
self._queues_by_ip: dict[str, list[asyncio.Queue]] = defaultdict(list)
self.max_subscribers = max_subscribers self.max_subscribers = max_subscribers
self.max_per_ip = max_per_ip
def subscribe(self) -> asyncio.Queue: def subscribe(self, client_ip: str = "unknown") -> asyncio.Queue:
if len(self._subscribers) >= self.max_subscribers: if len(self._ip_by_queue) >= self.max_subscribers:
raise RoundEventCapacityError(f"already at the {self.max_subscribers}-subscriber cap") raise RoundEventCapacityError(f"already at the {self.max_subscribers}-subscriber cap")
ip_queues = self._queues_by_ip[client_ip]
if len(ip_queues) >= self.max_per_ip:
# B-38: evict this IP's own oldest connection rather than refusing
# the new one — bounds one source's footprint without turning a
# legitimate reconnect storm (a flaky network retrying EventSource)
# into an outright block, and without letting one abusive IP crowd
# out unrelated clients the way the old global-only cap did.
oldest = ip_queues.pop(0)
self._ip_by_queue.pop(oldest, None)
if not oldest.full():
oldest.put_nowait(EVICTED)
queue: asyncio.Queue = asyncio.Queue(maxsize=1) queue: asyncio.Queue = asyncio.Queue(maxsize=1)
self._subscribers.add(queue) self._ip_by_queue[queue] = client_ip
ip_queues.append(queue)
return queue return queue
def unsubscribe(self, queue: asyncio.Queue) -> None: def unsubscribe(self, queue: asyncio.Queue) -> None:
self._subscribers.discard(queue) client_ip = self._ip_by_queue.pop(queue, None)
if client_ip is None:
return
ip_queues = self._queues_by_ip.get(client_ip)
if ip_queues is None:
return
if queue in ip_queues:
ip_queues.remove(queue)
if not ip_queues:
self._queues_by_ip.pop(client_ip, None)
def publish(self) -> None: def publish(self) -> None:
for queue in self._subscribers: for queue in self._ip_by_queue:
if queue.full(): if queue.full():
continue # a not-yet-delivered notification already covers this one continue # a not-yet-delivered notification already covers this one
queue.put_nowait(None) queue.put_nowait(None)
+54 -26
View File
@@ -48,14 +48,14 @@ def should_bump(pending: PendingTransaction, now: datetime, timeout_seconds: int
return now >= pending.last_broadcast_at.replace(tzinfo=timezone.utc) + timedelta(seconds=timeout_seconds) return now >= pending.last_broadcast_at.replace(tzinfo=timezone.utc) + timedelta(seconds=timeout_seconds)
async def _signing_context(session: AsyncSession, pending: PendingTransaction) -> tuple: async def _signing_context(session: AsyncSession, kind: str, user_id: int | None) -> tuple:
"""Returns (signing_key, own_script, own_address) for the single sender that """Returns (signing_key, own_script, own_address) for the single sender that
controls every input of this tx — a user for bet/withdrawal, the pool for controls every input of this tx — a user for bet/withdrawal, the pool for
payout. All our builders only ever spend one address's UTXOs per tx.""" payout. All our builders only ever spend one address's UTXOs per tx."""
if pending.kind == "payout": if kind == "payout":
key = derive_pool_key() key = derive_pool_key()
else: else:
user = await session.get(User, pending.user_id) user = await session.get(User, user_id)
key = derive_user_key(user.derivation_index) key = derive_user_key(user.derivation_index)
own_script = script.p2wpkh(key.to_public()) own_script = script.p2wpkh(key.to_public())
own_address = own_script.address(network=PLM_MAINNET) own_address = own_script.address(network=PLM_MAINNET)
@@ -63,10 +63,19 @@ async def _signing_context(session: AsyncSession, pending: PendingTransaction) -
async def _prevout_amount(client: ElectrumClient, vin: TransactionInput) -> int: async def _prevout_amount(client: ElectrumClient, vin: TransactionInput) -> int:
"""The exact integer satoshi value of the output this input spends.
Parsed directly from the raw transaction via embit rather than asking the
server for its own float, whole-coin-denominated "value" field (verbose=True)
and converting with `* 100_000_000` — embit's TransactionOutput.value is
already an integer number of satoshis straight from the tx's binary
encoding, so this never touches floating point in a codebase that is
otherwise strictly integer-satoshi (B-40).
"""
txid_hex = vin.txid.hex() txid_hex = vin.txid.hex()
tx = await client.get_transaction(txid_hex, verbose=True) raw_hex = await client.get_transaction(txid_hex, verbose=False)
value_coins = tx["vout"][vin.vout]["value"] prevout_tx = Transaction.parse(bytes.fromhex(raw_hex))
return round(value_coins * 100_000_000) return prevout_tx.vout[vin.vout].value
def _find_change_output(tx: Transaction, change_address: str) -> int | None: def _find_change_output(tx: Transaction, change_address: str) -> int | None:
@@ -76,35 +85,55 @@ def _find_change_output(tx: Transaction, change_address: str) -> int | None:
return None return None
async def bump_fee(session: AsyncSession, client: ElectrumClient, pending: PendingTransaction) -> str: async def bump_fee(
"""Rebuild `pending`'s transaction with a higher fee (same inputs, same session_factory: async_sessionmaker, client: ElectrumClient, pending_id: int
recipient outputs, the extra fee taken from the change output) and ) -> str | None:
rebroadcast. Returns the new txid. """Rebuild pending_transaction `pending_id`'s transaction with a higher fee
(same inputs, same recipient outputs, the extra fee taken from the change
output) and rebroadcast. Returns the new txid, or None if there was nothing
to do (the row is gone or already left "pending" — a normal race with
confirmation, not an error).
Three phases, so no DB session is held across the network calls this needs
(one get_transaction per input, then a broadcast) — the same shape used
elsewhere for exactly this reason (B-18, rounds/scheduler.py:_trigger_payout;
B-31, electrum/listener.py:refresh_user) and now here too (B-40): read what's
needed and close the session, do the chain work, then reopen to persist.
Only handles the common case: exactly one change output paying back to the Only handles the common case: exactly one change output paying back to the
tx's own sender address, large enough to absorb the increase. If there's no tx's own sender address, large enough to absorb the increase. If there's no
such output (e.g. an exact-amount bet with no change), this raises RbfError — such output (e.g. an exact-amount bet with no change), this raises RbfError —
bumping such a tx would require selecting additional inputs, which isn't bumping such a tx would require selecting additional inputs, which isn't
implemented for the MVP; it needs manual operator intervention. Also raises implemented for the MVP; it needs manual operator intervention. Also raises
RbfError, rather than bumping, once `pending` is already at MAX_FEE_RATE_SAT_VB RbfError, rather than bumping, once the row is already at MAX_FEE_RATE_SAT_VB
(B-32) — the reconciler abandons it if it never confirms (B-27), instead of (B-32) — the reconciler abandons it if it never confirms (B-27), instead of
this retrying an ever-higher fee forever. this retrying an ever-higher fee forever.
""" """
# --- Phase 1: read what's needed, close the session before any network call ---
async with session_factory() as session:
pending = await session.get(PendingTransaction, pending_id)
if pending is None or pending.status != "pending":
logger.info("pending_transaction %s no longer pending; skipping bump", pending_id)
return None
if pending.fee_rate_sat_vb >= MAX_FEE_RATE_SAT_VB: if pending.fee_rate_sat_vb >= MAX_FEE_RATE_SAT_VB:
raise RbfError( raise RbfError(
f"pending_transaction {pending.id}: already at the maximum fee rate " f"pending_transaction {pending_id}: already at the maximum fee rate "
f"({MAX_FEE_RATE_SAT_VB} sat/vB) — refusing to bump further" f"({MAX_FEE_RATE_SAT_VB} sat/vB) — refusing to bump further"
) )
old_tx = Transaction.parse(bytes.fromhex(pending.raw_tx_hex)) kind = pending.kind
signing_key, own_script, own_address = await _signing_context(session, pending) current_fee_rate = pending.fee_rate_sat_vb
raw_tx_hex = pending.raw_tx_hex
signing_key, own_script, own_address = await _signing_context(session, kind, pending.user_id)
# --- Phase 2: chain reads, signing, and the broadcast — no DB session open ----
old_tx = Transaction.parse(bytes.fromhex(raw_tx_hex))
input_amounts = [await _prevout_amount(client, vin) for vin in old_tx.vin] input_amounts = [await _prevout_amount(client, vin) for vin in old_tx.vin]
total_in = sum(input_amounts) total_in = sum(input_amounts)
old_fee = total_in - sum(o.value for o in old_tx.vout) old_fee = total_in - sum(o.value for o in old_tx.vout)
vsize = estimate_vsize(len(old_tx.vin), len(old_tx.vout)) vsize = estimate_vsize(len(old_tx.vin), len(old_tx.vout))
target_fee_rate = min(pending.fee_rate_sat_vb + _FEE_RATE_INCREMENT, MAX_FEE_RATE_SAT_VB) target_fee_rate = min(current_fee_rate + _FEE_RATE_INCREMENT, MAX_FEE_RATE_SAT_VB)
target_fee = vsize * target_fee_rate target_fee = vsize * target_fee_rate
# BIP125 rule 4's minimum, in absolute sats for this tx's size — the floor # BIP125 rule 4's minimum, in absolute sats for this tx's size — the floor
# `fee_delta` must never go below, no matter what `target_fee - old_fee` comes # `fee_delta` must never go below, no matter what `target_fee - old_fee` comes
@@ -120,7 +149,7 @@ async def bump_fee(session: AsyncSession, client: ElectrumClient, pending: Pendi
change_index = _find_change_output(old_tx, own_address) change_index = _find_change_output(old_tx, own_address)
if change_index is None or old_tx.vout[change_index].value <= fee_delta: if change_index is None or old_tx.vout[change_index].value <= fee_delta:
raise RbfError(f"pending_transaction {pending.id}: no change output large enough to absorb a fee bump") raise RbfError(f"pending_transaction {pending_id}: no change output large enough to absorb a fee bump")
new_vout = list(old_tx.vout) new_vout = list(old_tx.vout)
bumped_change = new_vout[change_index].value - fee_delta bumped_change = new_vout[change_index].value - fee_delta
@@ -144,6 +173,9 @@ async def bump_fee(session: AsyncSession, client: ElectrumClient, pending: Pendi
new_txid = final_tx.txid().hex() new_txid = final_tx.txid().hex()
await client.broadcast(raw_hex) await client.broadcast(raw_hex)
# --- Phase 3: persist the outcome ----------------------------------------------
async with session_factory() as session:
pending = await session.get(PendingTransaction, pending_id)
old_txid = pending.current_txid old_txid = pending.current_txid
pending.replaced_by_txid = old_txid # points backwards: what current_txid replaced pending.replaced_by_txid = old_txid # points backwards: what current_txid replaced
pending.current_txid = new_txid pending.current_txid = new_txid
@@ -162,7 +194,7 @@ async def bump_fee(session: AsyncSession, client: ElectrumClient, pending: Pendi
await _retarget_txid_references(session, pending, old_txid, new_txid) await _retarget_txid_references(session, pending, old_txid, new_txid)
await session.commit() await session.commit()
logger.info("bumped %s pending_transaction %s: %s -> %s", pending.kind, pending.id, old_txid, new_txid) logger.info("bumped %s pending_transaction %s: %s -> %s", kind, pending_id, old_txid, new_txid)
return new_txid return new_txid
@@ -230,16 +262,12 @@ class RbfBumper:
candidates = ( candidates = (
await session.scalars(select(PendingTransaction).where(PendingTransaction.status == "pending")) await session.scalars(select(PendingTransaction).where(PendingTransaction.status == "pending"))
).all() ).all()
due = [p for p in candidates if should_bump(p, now, timeout_seconds=timeout_seconds)] due_ids = [p.id for p in candidates if should_bump(p, now, timeout_seconds=timeout_seconds)]
for pending in due: for pending_id in due_ids:
async with self._session_factory() as session:
row = await session.get(PendingTransaction, pending.id)
if row is None or row.status != "pending":
continue
try: try:
await bump_fee(session, client, row) await bump_fee(self._session_factory, client, pending_id)
except RbfError: except RbfError:
logger.exception("could not bump pending_transaction %s", row.id) logger.exception("could not bump pending_transaction %s", pending_id)
except Exception: except Exception:
logger.exception("unexpected error bumping pending_transaction %s", row.id) logger.exception("unexpected error bumping pending_transaction %s", pending_id)
+40 -10
View File
@@ -7,7 +7,9 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from app.db.models import PendingTransaction from app.db.models import PendingTransaction
from app.electrum.client import ElectrumClient from app.electrum.client import ElectrumClient
from app.electrum.scripthash import address_to_scripthash
from app.rounds.events import broadcaster from app.rounds.events import broadcaster
from app.tx.pending_address import own_address_for
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -31,24 +33,52 @@ async def poll_once(session_factory: async_sessionmaker, client: ElectrumClient)
candidates = ( candidates = (
await session.execute( await session.execute(
select( select(
PendingTransaction.id, PendingTransaction.current_txid, PendingTransaction.kind PendingTransaction.id,
PendingTransaction.current_txid,
PendingTransaction.kind,
PendingTransaction.user_id,
).where(PendingTransaction.status == "pending") ).where(PendingTransaction.status == "pending")
) )
).all() ).all()
confirmed = 0 # Resolved once per candidate while the session is still open, and cached
for pending_id, txid, kind in candidates: # by scripthash below — every "payout" row shares the same pool address,
# so this also avoids asking the server the same history twice per tick.
scripthash_by_id: dict[int, str] = {}
for pending_id, _txid, kind, user_id in candidates:
try: try:
tx = await client.get_transaction(txid, verbose=True) address = await own_address_for(session, kind, user_id)
scripthash_by_id[pending_id] = address_to_scripthash(address)
except Exception: except Exception:
# One unresolvable txid must not stop the others: a tx the server no logger.exception("could not derive the address for pending_transaction %s", pending_id)
# longer knows (dropped from the mempool, replaced) used to abort the
# whole pass, so nothing confirmed again until an operator intervened confirmed = 0
# (B-03). Abandoning such a row is app/tx/reconcile.py's job, not ours. history_cache: dict[str, list[dict]] = {}
logger.warning("could not check pending_transaction %s (txid %s)", pending_id, txid, exc_info=True) for pending_id, txid, kind, _user_id in candidates:
scripthash = scripthash_by_id.get(pending_id)
if scripthash is None:
continue # address derivation failed above; already logged
try:
if scripthash not in history_cache:
history_cache[scripthash] = await client.get_history(scripthash)
except Exception:
# One unresolvable scripthash must not stop the others: a tx the server
# no longer knows about (dropped from the mempool, replaced) used to
# abort the whole pass via a verbose blockchain.transaction.get call
# that some servers reject outright (B-41), so nothing confirmed again
# until an operator intervened (B-03). Abandoning such a row is
# app/tx/reconcile.py's job, not ours.
logger.warning("could not fetch history for pending_transaction %s (txid %s)", pending_id, txid, exc_info=True)
continue continue
if not tx or tx.get("confirmations", 0) < 1:
entry = next((e for e in history_cache[scripthash] if e.get("tx_hash") == txid), None)
# height > 0 means confirmed at that height; 0 or absent means still in
# the mempool (or the server doesn't know this txid at all yet) — either
# way, not confirmed, so keep waiting.
if entry is None or entry.get("height", 0) <= 0:
continue continue
async with session_factory() as session: async with session_factory() as session:
row = await session.get(PendingTransaction, pending_id) row = await session.get(PendingTransaction, pending_id)
if row is None or row.status != "pending": if row is None or row.status != "pending":
+22
View File
@@ -0,0 +1,22 @@
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import User
from app.wallet.hd import derive_pool_address, derive_user_address
async def own_address_for(session: AsyncSession, kind: str, user_id: int | None) -> str:
"""The address that owns every input of a PendingTransaction of this kind —
a user's own address for a bet/withdrawal, the pool address for a payout.
All our builders only ever spend one address's UTXOs per tx (see
tx/broadcast.py:_signing_context, which derives the same address alongside
the signing key it also needs).
Shared by tx/confirmation.py and tx/reconcile.py (B-41): both now check
blockchain.scripthash.get_history for this address instead of asking
blockchain.transaction.get for a verbose reply, so the two can't derive
different addresses for the same row.
"""
if kind == "payout":
return derive_pool_address()
user = await session.get(User, user_id)
return derive_user_address(user.derivation_index)
+32 -22
View File
@@ -36,7 +36,9 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from app.audit.log import write_audit_log from app.audit.log import write_audit_log
from app.db.models import PendingTransaction, Round, RoundParticipant, UtxoEvent, Withdrawal from app.db.models import PendingTransaction, Round, RoundParticipant, UtxoEvent, Withdrawal
from app.electrum.client import ElectrumClient from app.electrum.client import ElectrumClient
from app.electrum.scripthash import address_to_scripthash
from app.rounds.events import broadcaster from app.rounds.events import broadcaster
from app.tx.pending_address import own_address_for
from app.wallet.balance import recompute_balance from app.wallet.balance import recompute_balance
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -72,22 +74,21 @@ class PendingTransactionReconciler:
await asyncio.sleep(_POLL_INTERVAL_SECONDS) await asyncio.sleep(_POLL_INTERVAL_SECONDS)
async def _tx_exists_on_chain(client: ElectrumClient, txid: str) -> bool:
"""True if the server knows this txid at all (mempool or mined). An error reply
means "unknown", which is the answer we're looking for; a transport failure is
*not* — that raises, and the caller leaves the row alone until next time."""
try:
tx = await client.get_transaction(txid, verbose=True)
except Exception as exc:
message = str(exc).lower()
if "missing" in message or "not found" in message or "no such" in message or "unknown" in message:
return False
raise
return bool(tx)
async def reconcile_once(session_factory: async_sessionmaker, client: ElectrumClient) -> int: async def reconcile_once(session_factory: async_sessionmaker, client: ElectrumClient) -> int:
"""Returns how many rows were resolved (promoted or abandoned).""" """Returns how many rows were resolved (promoted or abandoned).
Existence is decided by checking whether a row's own address's history
(blockchain.scripthash.get_history) includes its txid at all — mempool or
mined — rather than asking blockchain.transaction.get for a verbose reply
(B-41): several Electrum server implementations and versions reject the
verbose flag outright, and the previous substring-matching on the error
text (looking for "missing", "not found", ...) was fragile as the basis for
a decision that releases funds. A transport failure fetching history still
raises and leaves the row alone until next time — get_history not
returning our txid is the only thing that means "gone". History is cached
per scripthash within one pass, since every "payout" row shares the same
pool address.
"""
now = datetime.now(timezone.utc) now = datetime.now(timezone.utc)
async with session_factory() as session: async with session_factory() as session:
rows = ( rows = (
@@ -95,16 +96,25 @@ async def reconcile_once(session_factory: async_sessionmaker, client: ElectrumCl
select(PendingTransaction).where(PendingTransaction.status.in_(("building", "pending"))) select(PendingTransaction).where(PendingTransaction.status.in_(("building", "pending")))
) )
).all() ).all()
candidates = [ candidates = []
(row.id, row.status, row.current_txid) for row in rows:
for row in rows if not _is_due(row, now):
if _is_due(row, now) continue
] try:
address = await own_address_for(session, row.kind, row.user_id)
scripthash = address_to_scripthash(address)
except Exception:
logger.exception("could not derive the address for pending_transaction %s", row.id)
continue
candidates.append((row.id, row.status, row.current_txid, scripthash))
resolved = 0 resolved = 0
for row_id, status, txid in candidates: history_cache: dict[str, list[dict]] = {}
for row_id, status, txid, scripthash in candidates:
try: try:
exists = await _tx_exists_on_chain(client, txid) if scripthash not in history_cache:
history_cache[scripthash] = await client.get_history(scripthash)
exists = any(entry.get("tx_hash") == txid for entry in history_cache[scripthash])
except Exception: except Exception:
# Transport/server problem — say nothing about this tx and try again on # Transport/server problem — say nothing about this tx and try again on
# the next pass rather than abandoning a tx that may be perfectly alive. # the next pass rather than abandoning a tx that may be perfectly alive.
+3 -2
View File
@@ -121,8 +121,9 @@ Eventi a cui vale la pena prestare attenzione:
## Alternative all'interfaccia grafica ## Alternative all'interfaccia grafica
Le stesse operazioni si possono fare da terminale o da Swagger UI Le stesse operazioni si possono fare da terminale o da Swagger UI
(`https://<host>/docs`, sezione `admin`), sempre passando `ADMIN_TOKEN` (`https://<host>/docs`, sezione `admin` — disponibile solo se `ENABLE_API_DOCS=true`
nell'header `X-Admin-Token`: è impostato in `.env`, disattivata di default perché espone l'intera API),
sempre passando `ADMIN_TOKEN` nell'header `X-Admin-Token`:
```bash ```bash
# leggere la configurazione # leggere la configurazione
+7 -17
View File
@@ -3,24 +3,14 @@
Presuppone che [setup.md](setup.md) sia già stato completato (`.env` pronto, Presuppone che [setup.md](setup.md) sia già stato completato (`.env` pronto,
master key generata, migrazioni applicate). master key generata, migrazioni applicate).
## Locale / venv (sviluppo rapido) Il server gira sempre via Docker, in sviluppo e in produzione allo stesso
modo — non esiste un modo supportato per lanciare `uvicorn` direttamente.
Il venv locale (`.venv/`) serve solo per i test, per scrivere le migrazioni
Alembic e per gli script una tantum di generazione chiavi (vedi
[setup.md](setup.md) e la sezione "Commands" di
[CLAUDE.md](../CLAUDE.md#commands)).
```bash ## Docker + Caddy (unico workflow supportato)
source .venv/bin/activate
uvicorn app.main:app --reload --port 8123
```
- App su `http://127.0.0.1:8123/`
- Pannello admin su `http://127.0.0.1:8123/admin`
- Log applicativi in `logs/app.log` (rotante, 10MB × 5 backup)
- Nessun TLS, nessun reverse proxy — solo per test locali sulla tua macchina.
Per fermarlo: `Ctrl+C`, oppure se lanciato in background con `nohup`:
```bash
pkill -f "uvicorn app.main:app"
```
## Docker + Caddy (consigliato, anche per i test con dominio/TLS)
```bash ```bash
mkdir -p data/db data/keys data/logs # una tantum, se non già presenti mkdir -p data/db data/keys data/logs # una tantum, se non già presenti
+5
View File
@@ -38,6 +38,11 @@ cp .env.example .env
Le altre chiavi di `.env` (`DATABASE_URL`, `ELECTRUM_HOST`/`PORT`/`USE_SSL`, Le altre chiavi di `.env` (`DATABASE_URL`, `ELECTRUM_HOST`/`PORT`/`USE_SSL`,
`MASTER_KEY_PATH`) hanno default sensati in `.env.example`. `MASTER_KEY_PATH`) hanno default sensati in `.env.example`.
`ENABLE_API_DOCS` (default `false`) controlla Swagger/ReDoc/l'OpenAPI JSON grezzo
su `/docs`, `/redoc` e `/openapi.json`: espongono l'intera superficie dell'API,
endpoint admin inclusi, quindi restano disattivati a meno di non impostarlo
esplicitamente a `true` — utile in locale, da evitare in produzione.
`ELECTRUM_FALLBACK_SERVERS` elenca i server di riserva, separati da virgola, nel `ELECTRUM_FALLBACK_SERVERS` elenca i server di riserva, separati da virgola, nel
formato `host:porta` (TLS, il caso normale) oppure `host:porta:notls`. Esempio: formato `host:porta` (TLS, il caso normale) oppure `host:porta:notls`. Esempio:
@@ -0,0 +1,38 @@
"""widen raw_tx_hex and payload_json to Text
Fixes B-47: both columns held arbitrary-length data (a raw signed transaction
hex, and a JSON audit payload) in an unbounded `String`, which SQLAlchemy
compiles to `VARCHAR` with no length. That's accepted by SQLite and
PostgreSQL but rejected by other backends (e.g. MySQL requires a length on
VARCHAR) — `Text` is the portable type for both.
Revision ID: 87a0c640355c
Revises: 9ef6a51509f7
Create Date: 2026-07-27
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '87a0c640355c'
down_revision: Union[str, Sequence[str], None] = '9ef6a51509f7'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
with op.batch_alter_table('audit_log') as batch_op:
batch_op.alter_column('payload_json', existing_type=sa.VARCHAR(), type_=sa.Text(), existing_nullable=False)
with op.batch_alter_table('pending_transactions') as batch_op:
batch_op.alter_column('raw_tx_hex', existing_type=sa.VARCHAR(), type_=sa.Text(), existing_nullable=False)
def downgrade() -> None:
with op.batch_alter_table('pending_transactions') as batch_op:
batch_op.alter_column('raw_tx_hex', existing_type=sa.Text(), type_=sa.VARCHAR(), existing_nullable=False)
with op.batch_alter_table('audit_log') as batch_op:
batch_op.alter_column('payload_json', existing_type=sa.Text(), type_=sa.VARCHAR(), existing_nullable=False)
+96
View File
@@ -67,6 +67,15 @@ async def test_admin_rejects_wrong_token(client):
assert resp.status_code == 403 assert resp.status_code == 403
async def test_admin_rejects_non_ascii_token_with_403_not_500(client):
"""B-46: secrets.compare_digest raises TypeError on a non-ASCII str, which
used to bubble up as a 500 instead of the expected 403. httpx encodes str
header values as ASCII client-side, so the raw UTF-8 bytes are passed
directly to reproduce what a real non-ASCII header on the wire looks like."""
resp = await client.get("/admin/config", headers={"X-Admin-Token": "café".encode("utf-8")})
assert resp.status_code == 403
async def test_admin_reads_and_updates_config(client): async def test_admin_reads_and_updates_config(client):
headers = {"X-Admin-Token": "test-admin-token"} headers = {"X-Admin-Token": "test-admin-token"}
@@ -277,3 +286,90 @@ async def test_pause_cannot_be_toggled_through_the_config_endpoint(client):
resp = await client.put("/admin/config", headers=headers, json={"paused": True}) resp = await client.put("/admin/config", headers=headers, json={"paused": True})
assert resp.status_code in (200, 422) # ignored or refused, but never applied assert resp.status_code in (200, 422) # ignored or refused, but never applied
assert (await client.get("/admin/config", headers=headers)).json()["paused"] is False assert (await client.get("/admin/config", headers=headers)).json()["paused"] is False
@pytest.mark.parametrize("endpoint", ["/admin/rounds", "/admin/audit-log", "/admin/pending-transactions"])
@pytest.mark.parametrize("bad_limit", [0, -1, 501])
async def test_admin_list_endpoints_reject_out_of_range_limit(client, endpoint, bad_limit):
"""B-45: `limit` had no bounds — `-1` means "everything" on SQLite, so an
unvalidated limit could dump the entire table in one response."""
headers = {"X-Admin-Token": "test-admin-token"}
resp = await client.get(endpoint, headers=headers, params={"limit": bad_limit})
assert resp.status_code == 422
async def test_admin_list_rounds_respects_limit(client):
from app.db import base as db_base
from app.db.models import Round
async with db_base.AsyncSessionLocal() as session:
session.add_all([Round(status="closed") for _ in range(3)])
await session.commit()
headers = {"X-Admin-Token": "test-admin-token"}
resp = await client.get("/admin/rounds", headers=headers, params={"limit": 2})
assert resp.status_code == 200
assert len(resp.json()) == 2
async def test_admin_audit_log_respects_limit(client):
from app.db import base as db_base
from app.audit.log import write_audit_log
async with db_base.AsyncSessionLocal() as session:
for _ in range(3):
await write_audit_log(session, "test_event", {})
await session.commit()
headers = {"X-Admin-Token": "test-admin-token"}
resp = await client.get("/admin/audit-log", headers=headers, params={"limit": 2})
assert resp.status_code == 200
assert len(resp.json()) == 2
async def _make_pending_transaction(session, *, kind="bet", status="pending"):
from app.db.models import PendingTransaction
import secrets as _secrets
tx = PendingTransaction(
kind=kind,
current_txid=_secrets.token_hex(32),
fee_rate_sat_vb=1,
raw_tx_hex="00",
status=status,
)
session.add(tx)
return tx
async def test_admin_pending_transactions_respects_limit(client):
from app.db import base as db_base
async with db_base.AsyncSessionLocal() as session:
for _ in range(3):
await _make_pending_transaction(session)
await session.commit()
headers = {"X-Admin-Token": "test-admin-token"}
resp = await client.get("/admin/pending-transactions", headers=headers, params={"limit": 2})
assert resp.status_code == 200
assert len(resp.json()) == 2
async def test_admin_pending_transactions_status_filter(client):
from app.db import base as db_base
async with db_base.AsyncSessionLocal() as session:
await _make_pending_transaction(session, status="pending")
await _make_pending_transaction(session, status="confirmed")
await _make_pending_transaction(session, status="failed")
await session.commit()
headers = {"X-Admin-Token": "test-admin-token"}
resp = await client.get(
"/admin/pending-transactions", headers=headers, params={"status": "confirmed"}
)
assert resp.status_code == 200
entries = resp.json()
assert len(entries) == 1
assert entries[0]["status"] == "confirmed"
+137 -21
View File
@@ -3,7 +3,7 @@ from datetime import datetime, timedelta, timezone
import pytest import pytest
from embit import script from embit import script
from embit.bip32 import HDKey from embit.bip32 import HDKey
from embit.transaction import Transaction from embit.transaction import Transaction, TransactionInput, TransactionOutput
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from app.config import settings from app.config import settings
@@ -58,12 +58,23 @@ def test_should_bump_measures_from_last_broadcast_not_first(monkeypatch):
class FakeClient: class FakeClient:
"""B-40: _prevout_amount now asks for the raw (non-verbose) transaction and
reads its output value as an integer via embit, rather than a verbose reply's
float "value" field — so this fake must hand back a real, parseable raw tx
whose vout[0] carries the requested amount (every test here spends vout 0 of
its fixture UTXO)."""
def __init__(self, prevout_values: dict[str, int]): def __init__(self, prevout_values: dict[str, int]):
self._prevout_values = prevout_values self._prevout_values = prevout_values
self.broadcasted: list[str] = [] self.broadcasted: list[str] = []
async def get_transaction(self, txid: str, verbose: bool = False) -> dict: async def get_transaction(self, txid: str, verbose: bool = False) -> str:
return {"vout": {0: {"value": self._prevout_values[txid] / 100_000_000}}} assert verbose is False
fake_prevout_tx = Transaction(
vin=[TransactionInput(b"\x00" * 32, 0)],
vout=[TransactionOutput(self._prevout_values[txid], script.Script(b"\x00\x14" + b"\x00" * 20))],
)
return fake_prevout_tx.serialize().hex()
async def broadcast(self, raw_tx_hex: str) -> str: async def broadcast(self, raw_tx_hex: str) -> str:
self.broadcasted.append(raw_tx_hex) self.broadcasted.append(raw_tx_hex)
@@ -130,9 +141,7 @@ async def test_bump_fee_shrinks_change_and_rebroadcasts(session_factory):
client = FakeClient({utxo_txid: utxo_amount}) client = FakeClient({utxo_txid: utxo_amount})
async with session_factory() as session: new_txid = await bump_fee(session_factory, client, pending_id)
row = await session.get(PendingTransaction, pending_id)
new_txid = await bump_fee(session, client, row)
assert client.broadcasted assert client.broadcasted
assert new_txid != built.txid assert new_txid != built.txid
@@ -197,9 +206,7 @@ async def test_bump_fee_leaves_broadcast_at_untouched(session_factory):
client = FakeClient({utxo_txid: utxo_amount}) client = FakeClient({utxo_txid: utxo_amount})
before_bump = datetime.now(timezone.utc) before_bump = datetime.now(timezone.utc)
async with session_factory() as session: await bump_fee(session_factory, client, pending_id)
row = await session.get(PendingTransaction, pending_id)
await bump_fee(session, client, row)
async with session_factory() as session: async with session_factory() as session:
row = await session.get(PendingTransaction, pending_id) row = await session.get(PendingTransaction, pending_id)
@@ -246,10 +253,8 @@ async def test_bump_fee_raises_when_no_change_output(session_factory):
client = FakeClient({utxo_txid: utxo_amount}) client = FakeClient({utxo_txid: utxo_amount})
async with session_factory() as session:
row = await session.get(PendingTransaction, pending_id)
with pytest.raises(RbfError): with pytest.raises(RbfError):
await bump_fee(session, client, row) await bump_fee(session_factory, client, pending_id)
async def test_bump_fee_retargets_every_stored_txid(session_factory): async def test_bump_fee_retargets_every_stored_txid(session_factory):
@@ -303,9 +308,7 @@ async def test_bump_fee_retargets_every_stored_txid(session_factory):
await session.commit() await session.commit()
pending_id = pending.id pending_id = pending.id
async with session_factory() as session: new_txid = await bump_fee(session_factory, FakeClient({utxo_txid: utxo_amount}), pending_id)
row = await session.get(PendingTransaction, pending_id)
new_txid = await bump_fee(session, FakeClient({utxo_txid: utxo_amount}), row)
async with session_factory() as session: async with session_factory() as session:
from sqlalchemy import select from sqlalchemy import select
@@ -378,9 +381,7 @@ async def test_bump_fee_meets_bip125_minimum_when_old_fee_already_exceeds_target
inflated_excess = 50_000 inflated_excess = 50_000
client = FakeClient({utxo_txid: utxo_amount + inflated_excess}) client = FakeClient({utxo_txid: utxo_amount + inflated_excess})
async with session_factory() as session: new_txid = await bump_fee(session_factory, client, pending_id)
row = await session.get(PendingTransaction, pending_id)
new_txid = await bump_fee(session, client, row)
assert client.broadcasted assert client.broadcasted
new_tx = Transaction.parse(bytes.fromhex(client.broadcasted[0])) new_tx = Transaction.parse(bytes.fromhex(client.broadcasted[0]))
@@ -442,9 +443,124 @@ async def test_bump_fee_refuses_once_at_the_max_fee_rate(session_factory):
client = FakeClient({utxo_txid: utxo_amount}) client = FakeClient({utxo_txid: utxo_amount})
async with session_factory() as session:
row = await session.get(PendingTransaction, pending_id)
with pytest.raises(RbfError): with pytest.raises(RbfError):
await bump_fee(session, client, row) await bump_fee(session_factory, client, pending_id)
assert not client.broadcasted assert not client.broadcasted
# --- B-40: bump_fee must not hold a DB session open across its network calls,
# and a row that's no longer pending by the time it runs is a quiet no-op. -------
async def test_bump_fee_holds_no_session_open_during_network_calls(session_factory):
"""The get_transaction-per-input reads and the broadcast must happen with no
DB session held open — the same shape used elsewhere for this reason (B-18,
electrum/listener.py's refresh_user for B-31) — otherwise a session sits
idle in the pool for the whole duration of what can be several slow network
round-trips."""
from app.wallet.hd import derive_user_address, derive_user_key
signer = derive_user_key(0)
my_address = derive_user_address(0)
from_script = script.p2wpkh(signer.to_public())
to_address = script.p2wpkh(_key(94).to_public()).address(network=PLM_MAINNET)
utxo_amount = 150_000_000
utxo_txid = "77" * 32
built = build_signed_transaction(
signing_key=signer,
from_script=from_script,
utxos=[Utxo(utxo_txid, 0, utxo_amount)],
to_address=to_address,
amount_sats=10_000_000,
change_address=my_address,
fee_rate_sat_vb=1,
)
async with session_factory() as session:
user = User(username="frank", password_hash="x", derivation_index=0, address=my_address)
session.add(user)
await session.commit()
pending = PendingTransaction(
kind="bet",
user_id=user.id,
current_txid=built.txid,
fee_rate_sat_vb=1,
raw_tx_hex=built.raw_hex,
status="pending",
broadcast_at=datetime.now(timezone.utc) - timedelta(seconds=1000),
)
session.add(pending)
await session.commit()
pending_id = pending.id
open_count = {"n": 0}
class _TrackedSession:
def __init__(self, inner):
self._inner = inner
async def __aenter__(self):
result = await self._inner.__aenter__()
open_count["n"] += 1
return result
async def __aexit__(self, *exc):
open_count["n"] -= 1
return await self._inner.__aexit__(*exc)
def tracking_session_factory():
return _TrackedSession(session_factory())
class TrackingClient(FakeClient):
async def get_transaction(self, txid, verbose=False):
assert open_count["n"] == 0, "a session was held open during a network call"
return await super().get_transaction(txid, verbose)
async def broadcast(self, raw_tx_hex):
assert open_count["n"] == 0, "a session was held open during the broadcast"
return await super().broadcast(raw_tx_hex)
client = TrackingClient({utxo_txid: utxo_amount})
await bump_fee(tracking_session_factory, client, pending_id)
assert client.broadcasted
assert open_count["n"] == 0 # nothing left open afterwards either
async def test_bump_fee_is_a_noop_when_no_longer_pending(session_factory):
"""A row can legitimately confirm (or otherwise leave "pending") between
being read as due and RbfBumper actually attempting the bump — a normal
race, not an error. Must return quietly rather than raising or touching
the network."""
async with session_factory() as session:
user = User(username="grace", password_hash="x", derivation_index=0, address="plm1qxxx")
session.add(user)
await session.commit()
pending = PendingTransaction(
kind="bet",
user_id=user.id,
current_txid="already-confirmed-txid",
fee_rate_sat_vb=1,
raw_tx_hex="00",
status="confirmed",
broadcast_at=datetime.now(timezone.utc) - timedelta(seconds=1000),
)
session.add(pending)
await session.commit()
pending_id = pending.id
client = FakeClient({})
result = await bump_fee(session_factory, client, pending_id)
assert result is None
assert not client.broadcasted
async def test_bump_fee_is_a_noop_when_the_row_is_gone(session_factory):
client = FakeClient({})
result = await bump_fee(session_factory, client, 999_999)
assert result is None
assert not client.broadcasted
@@ -0,0 +1,35 @@
"""B-43: the Caddyfile must keep sending baseline security headers. Caddy adds
none of these on its own, and the JWT lives in localStorage, so a regression
here silently reopens an XSS/clickjacking exposure with no test ever failing
in the Python suite (the Caddyfile isn't imported/exercised by anything else)."""
from pathlib import Path
CADDYFILE = (Path(__file__).parent.parent.parent / "Caddyfile").read_text()
def test_header_block_present():
assert "header {" in CADDYFILE
def test_hsts_is_set():
assert "Strict-Transport-Security" in CADDYFILE
assert "max-age=" in CADDYFILE
def test_nosniff_is_set():
assert 'X-Content-Type-Options "nosniff"' in CADDYFILE
def test_frame_ancestors_are_blocked():
assert 'X-Frame-Options "DENY"' in CADDYFILE
assert "frame-ancestors 'none'" in CADDYFILE
def test_referrer_policy_is_set():
assert "Referrer-Policy" in CADDYFILE
def test_csp_default_src_is_self():
assert "Content-Security-Policy" in CADDYFILE
assert "default-src 'self'" in CADDYFILE
+43
View File
@@ -0,0 +1,43 @@
"""app.api.client_ip is shared by the login/registration throttles (B-33) and
the SSE per-IP subscriber cap (B-38) — both depend on it correctly preferring
X-Forwarded-For (Caddy reverse-proxies every request, see Caddyfile) over
request.client.host, which would otherwise be the proxy's own address."""
from starlette.requests import Request
from app.api.client_ip import client_ip
def _request(*, forwarded: str | None = None, client_host: str | None = "127.0.0.1") -> Request:
headers = [(b"x-forwarded-for", forwarded.encode())] if forwarded else []
scope = {
"type": "http",
"headers": headers,
"client": (client_host, 12345) if client_host else None,
}
return Request(scope)
def test_client_ip_prefers_x_forwarded_for():
request = _request(forwarded="5.6.7.8", client_host="10.0.0.1")
assert client_ip(request) == "5.6.7.8"
def test_client_ip_takes_the_first_hop_of_a_forwarded_chain():
request = _request(forwarded="5.6.7.8, 10.0.0.1, 172.17.0.1")
assert client_ip(request) == "5.6.7.8"
def test_client_ip_strips_whitespace():
request = _request(forwarded=" 5.6.7.8 , 10.0.0.1")
assert client_ip(request) == "5.6.7.8"
def test_client_ip_falls_back_to_request_client_without_the_header():
request = _request(forwarded=None, client_host="10.0.0.1")
assert client_ip(request) == "10.0.0.1"
def test_client_ip_falls_back_to_unknown_with_neither():
request = _request(forwarded=None, client_host=None)
assert client_ip(request) == "unknown"
+132 -36
View File
@@ -1,41 +1,83 @@
import pytest import pytest
from sqlalchemy import select
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
import app.bets.confirmation # noqa: F401 (registers the "bet" handler) import app.bets.confirmation # noqa: F401 (registers the "bet" handler)
import app.rounds.confirmation # noqa: F401 (registers the "payout" handler) import app.rounds.confirmation # noqa: F401 (registers the "payout" handler)
from sqlalchemy import select
from app.config import settings
from app.db.base import Base from app.db.base import Base
from app.db.models import PendingTransaction, Round, RoundParticipant from app.db.models import PendingTransaction, Round, RoundParticipant, User
from app.electrum.scripthash import address_to_scripthash
from app.tx.confirmation import poll_once from app.tx.confirmation import poll_once
from app.wallet.hd import derive_user_address
class FakeClient: class FakeClient:
def __init__(self, confirmations_by_txid: dict[str, int]): """B-41: poll_once now asks blockchain.scripthash.get_history rather than a
self._confirmations = confirmations_by_txid verbose blockchain.transaction.get, so this hands back a flat history —
height > 0 means confirmed at that height, 0 (or absent) means still in the
mempool. The scripthash argument is ignored: every candidate's derived
address is looked up against the same known universe of txids, which is
fine since matching happens on tx_hash, not on which address asked."""
async def get_transaction(self, txid: str, verbose: bool = False) -> dict: def __init__(self, heights_by_txid: dict[str, int]):
return {"confirmations": self._confirmations.get(txid, 0)} self._heights = heights_by_txid
async def get_history(self, scripthash: str) -> list[dict]:
return [{"tx_hash": txid, "height": height} for txid, height in self._heights.items()]
@pytest.fixture @pytest.fixture
async def session_factory(): async def session_factory(tmp_path, monkeypatch):
# own_address_for (B-41) derives each row's address via the HD wallet, so
# poll_once now needs a real master key — same bootstrap test_broadcast.py
# and test_reconcile.py use.
monkeypatch.setattr(settings, "master_key_path", str(tmp_path / "master.xprv.enc"))
monkeypatch.setattr(
settings,
"xprv_encryption_key",
__import__("cryptography.fernet", fromlist=["Fernet"]).Fernet.generate_key().decode(),
)
from app.wallet import hd
hd._account_key = None
hd.generate_master_key()
engine = create_async_engine("sqlite+aiosqlite:///:memory:") engine = create_async_engine("sqlite+aiosqlite:///:memory:")
async with engine.begin() as conn: async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all) await conn.run_sync(Base.metadata.create_all)
yield async_sessionmaker(engine, expire_on_commit=False) yield async_sessionmaker(engine, expire_on_commit=False)
await engine.dispose() await engine.dispose()
hd._account_key = None
async def _make_user(session, derivation_index: int) -> User:
user = User(
username=f"user{derivation_index}",
password_hash="x",
derivation_index=derivation_index,
address=derive_user_address(derivation_index),
)
session.add(user)
await session.flush()
return user
async def test_bet_confirmation_marks_participant_confirmed(session_factory): async def test_bet_confirmation_marks_participant_confirmed(session_factory):
async with session_factory() as session: async with session_factory() as session:
user = await _make_user(session, 0)
session.add(Round(id=1, status="open")) session.add(Round(id=1, status="open"))
session.add( session.add(
RoundParticipant( RoundParticipant(
round_id=1, user_id=1, bet_amount_sats=1_000, bet_txid="tx1", status="broadcast" round_id=1, user_id=user.id, bet_amount_sats=1_000, bet_txid="tx1", status="broadcast"
) )
) )
session.add( session.add(
PendingTransaction(kind="bet", round_id=1, user_id=1, current_txid="tx1", fee_rate_sat_vb=1, raw_tx_hex="00", status="pending") PendingTransaction(
kind="bet", round_id=1, user_id=user.id, current_txid="tx1", fee_rate_sat_vb=1,
raw_tx_hex="00", status="pending",
)
) )
await session.commit() await session.commit()
@@ -53,9 +95,17 @@ async def test_bet_confirmation_marks_participant_confirmed(session_factory):
async def test_unconfirmed_tx_is_left_pending(session_factory): async def test_unconfirmed_tx_is_left_pending(session_factory):
async with session_factory() as session: async with session_factory() as session:
user = await _make_user(session, 0)
session.add(Round(id=2, status="open")) session.add(Round(id=2, status="open"))
session.add(RoundParticipant(round_id=2, user_id=1, bet_amount_sats=1_000, bet_txid="tx2", status="broadcast")) session.add(
session.add(PendingTransaction(kind="bet", round_id=2, user_id=1, current_txid="tx2", fee_rate_sat_vb=1, raw_tx_hex="00", status="pending")) RoundParticipant(round_id=2, user_id=user.id, bet_amount_sats=1_000, bet_txid="tx2", status="broadcast")
)
session.add(
PendingTransaction(
kind="bet", round_id=2, user_id=user.id, current_txid="tx2", fee_rate_sat_vb=1,
raw_tx_hex="00", status="pending",
)
)
await session.commit() await session.commit()
client = FakeClient({"tx2": 0}) client = FakeClient({"tx2": 0})
@@ -70,7 +120,11 @@ async def test_unconfirmed_tx_is_left_pending(session_factory):
async def test_payout_confirmation_closes_round(session_factory): async def test_payout_confirmation_closes_round(session_factory):
async with session_factory() as session: async with session_factory() as session:
session.add(Round(id=3, status="paying_out", payout_txid="tx3")) session.add(Round(id=3, status="paying_out", payout_txid="tx3"))
session.add(PendingTransaction(kind="payout", round_id=3, current_txid="tx3", fee_rate_sat_vb=1, raw_tx_hex="00", status="pending")) session.add(
PendingTransaction(
kind="payout", round_id=3, current_txid="tx3", fee_rate_sat_vb=1, raw_tx_hex="00", status="pending"
)
)
await session.commit() await session.commit()
client = FakeClient({"tx3": 2}) client = FakeClient({"tx3": 2})
@@ -83,43 +137,52 @@ async def test_payout_confirmation_closes_round(session_factory):
class ExplodingClient: class ExplodingClient:
"""Answers for one txid and raises for the other — a tx the server no longer """Answers for one address's history and raises for the other's — the
knows (dropped from the mempool, replaced by a bump).""" get_history equivalent of a server that no longer knows a particular tx
(dropped from the mempool, replaced by a bump)."""
def __init__(self, known: dict[str, int], exploding_txid: str): def __init__(self, heights_by_txid: dict[str, int], exploding_scripthash: str):
self._known = known self._heights = heights_by_txid
self._exploding = exploding_txid self._exploding = exploding_scripthash
async def get_transaction(self, txid: str, verbose: bool = False) -> dict: async def get_history(self, scripthash: str) -> list[dict]:
if txid == self._exploding: if scripthash == self._exploding:
raise RuntimeError("missing transaction") raise RuntimeError("server error")
return {"confirmations": self._known.get(txid, 0)} return [{"tx_hash": txid, "height": height} for txid, height in self._heights.items()]
async def test_one_unresolvable_txid_does_not_block_the_others(session_factory): async def test_one_unresolvable_candidate_does_not_block_the_others(session_factory):
"""B-03: the lookup used to be unguarded, so a single unknown txid aborted the """B-03: the lookup used to be unguarded, so a single failing candidate aborted
whole pass — nothing confirmed again until an operator intervened, which in turn the whole pass — nothing confirmed again until an operator intervened, which in
meant no round could ever close.""" turn meant no round could ever close. B-41 changed the failure unit from "one
txid" to "one address's history", but the isolation guarantee is the same."""
async with session_factory() as session: async with session_factory() as session:
good_user = await _make_user(session, 0)
gone_user = await _make_user(session, 1)
session.add(Round(id=10, status="open")) session.add(Round(id=10, status="open"))
session.add( session.add(
RoundParticipant(round_id=10, user_id=1, bet_amount_sats=1_000, bet_txid="good", status="broadcast") RoundParticipant(
) round_id=10, user_id=good_user.id, bet_amount_sats=1_000, bet_txid="good", status="broadcast"
session.add(
PendingTransaction(
kind="bet", round_id=10, user_id=2, current_txid="gone", fee_rate_sat_vb=1, raw_tx_hex="00",
status="pending",
) )
) )
session.add( session.add(
PendingTransaction( PendingTransaction(
kind="bet", round_id=10, user_id=1, current_txid="good", fee_rate_sat_vb=1, raw_tx_hex="00", kind="bet", round_id=10, user_id=gone_user.id, current_txid="gone", fee_rate_sat_vb=1,
status="pending", raw_tx_hex="00", status="pending",
)
)
session.add(
PendingTransaction(
kind="bet", round_id=10, user_id=good_user.id, current_txid="good", fee_rate_sat_vb=1,
raw_tx_hex="00", status="pending",
) )
) )
await session.commit() await session.commit()
confirmed = await poll_once(session_factory, ExplodingClient({"good": 1}, exploding_txid="gone")) exploding_scripthash = address_to_scripthash(derive_user_address(1))
confirmed = await poll_once(
session_factory, ExplodingClient({"good": 1}, exploding_scripthash=exploding_scripthash)
)
assert confirmed == 1 # the healthy one still got processed assert confirmed == 1 # the healthy one still got processed
async with session_factory() as session: async with session_factory() as session:
@@ -135,15 +198,16 @@ async def test_bet_confirms_after_an_rbf_bump_changed_the_txid(session_factory):
a txid no participant carried — the participant stayed "broadcast" forever and a txid no participant carried — the participant stayed "broadcast" forever and
the round could never close. It now resolves by (round_id, user_id).""" the round could never close. It now resolves by (round_id, user_id)."""
async with session_factory() as session: async with session_factory() as session:
user = await _make_user(session, 0)
session.add(Round(id=11, status="open")) session.add(Round(id=11, status="open"))
session.add( session.add(
RoundParticipant( RoundParticipant(
round_id=11, user_id=7, bet_amount_sats=1_000, bet_txid="old-txid", status="broadcast" round_id=11, user_id=user.id, bet_amount_sats=1_000, bet_txid="old-txid", status="broadcast"
) )
) )
session.add( session.add(
PendingTransaction( PendingTransaction(
kind="bet", round_id=11, user_id=7, current_txid="bumped-txid", fee_rate_sat_vb=2, kind="bet", round_id=11, user_id=user.id, current_txid="bumped-txid", fee_rate_sat_vb=2,
raw_tx_hex="00", status="pending", replaced_by_txid="old-txid", raw_tx_hex="00", status="pending", replaced_by_txid="old-txid",
) )
) )
@@ -171,3 +235,35 @@ async def test_payout_confirms_after_an_rbf_bump_changed_the_txid(session_factor
async with session_factory() as session: async with session_factory() as session:
assert (await session.get(Round, 12)).status == "closed" assert (await session.get(Round, 12)).status == "closed"
async def test_poll_once_caches_history_per_scripthash(session_factory):
"""Two pending bets from the same user share one address — fetching its
history twice in one pass would be wasteful."""
async with session_factory() as session:
user = await _make_user(session, 0)
session.add(Round(id=20, status="open"))
session.add(
PendingTransaction(
kind="bet", round_id=20, user_id=user.id, current_txid="tx-a", fee_rate_sat_vb=1,
raw_tx_hex="00", status="pending",
)
)
session.add(
PendingTransaction(
kind="withdrawal", user_id=user.id, current_txid="tx-b", fee_rate_sat_vb=1,
raw_tx_hex="00", status="pending",
)
)
await session.commit()
call_count = {"n": 0}
class CountingClient:
async def get_history(self, scripthash: str) -> list[dict]:
call_count["n"] += 1
return [{"tx_hash": "tx-a", "height": 0}, {"tx_hash": "tx-b", "height": 0}]
await poll_once(session_factory, CountingClient())
assert call_count["n"] == 1
+61
View File
@@ -0,0 +1,61 @@
"""Regression tests for B-39: SQLite must run in WAL mode with a busy_timeout,
since this app has five concurrent background tasks plus every HTTP handler
sharing one database file, and the default rollback-journal mode lets a writer
block every reader and fails a second writer immediately instead of waiting."""
import pytest
from sqlalchemy.ext.asyncio import create_async_engine
from app.db.base import _SQLITE_BUSY_TIMEOUT_MS, _register_sqlite_pragmas
@pytest.fixture
async def sqlite_engine(tmp_path):
# WAL needs a real file (it writes a companion -wal/-shm file alongside it) —
# ":memory:" wouldn't exercise the same path.
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path}/test.db")
yield engine
await engine.dispose()
async def _pragma(engine, name: str):
async with engine.connect() as conn:
result = await conn.exec_driver_sql(f"PRAGMA {name}")
return result.fetchone()[0]
async def test_register_sqlite_pragmas_enables_wal_and_busy_timeout(sqlite_engine):
_register_sqlite_pragmas(sqlite_engine)
assert (await _pragma(sqlite_engine, "journal_mode")).lower() == "wal"
assert await _pragma(sqlite_engine, "busy_timeout") == _SQLITE_BUSY_TIMEOUT_MS
assert await _pragma(sqlite_engine, "synchronous") == 1 # NORMAL
async def test_register_sqlite_pragmas_applies_to_every_new_connection(sqlite_engine):
"""The pool can open more than one underlying DBAPI connection over the
engine's lifetime — the pragmas must be re-applied to each one, not just
the first, or a later connection would silently fall back to SQLite's
defaults."""
_register_sqlite_pragmas(sqlite_engine)
async with sqlite_engine.connect() as first:
await first.exec_driver_sql("PRAGMA journal_mode")
async with sqlite_engine.connect() as second:
result = await second.exec_driver_sql("PRAGMA busy_timeout")
assert result.fetchone()[0] == _SQLITE_BUSY_TIMEOUT_MS
def test_register_sqlite_pragmas_is_a_noop_for_other_dialects():
"""Must not touch (or crash on) a non-sqlite engine — e.g. a future
PostgreSQL DATABASE_URL, which neither needs nor understands these
pragmas."""
class _FakeDialect:
name = "postgresql"
class _FakeEngine:
dialect = _FakeDialect()
_register_sqlite_pragmas(_FakeEngine()) # must not raise
+28
View File
@@ -0,0 +1,28 @@
"""B-42: Swagger/ReDoc/OpenAPI JSON must not be reachable unless explicitly enabled —
they enumerate the whole API surface, admin endpoints included."""
import importlib
from app.config import settings
def _reload_main():
import app.main
return importlib.reload(app.main)
def test_docs_disabled_by_default(monkeypatch):
monkeypatch.setattr(settings, "enable_api_docs", False)
main = _reload_main()
assert main.app.docs_url is None
assert main.app.redoc_url is None
assert main.app.openapi_url is None
def test_docs_enabled_when_configured(monkeypatch):
monkeypatch.setattr(settings, "enable_api_docs", True)
main = _reload_main()
assert main.app.docs_url == "/docs"
assert main.app.redoc_url == "/redoc"
assert main.app.openapi_url == "/openapi.json"
+18
View File
@@ -0,0 +1,18 @@
"""B-44: README and docs/running-the-server.md must not document a bare
`uvicorn --reload` workflow — the server always runs via Docker, in dev and
production alike (CLAUDE.md's "Commands" section), and the two files had
drifted back to contradicting that policy."""
from pathlib import Path
REPO_ROOT = Path(__file__).parent.parent.parent
def test_readme_has_no_bare_uvicorn_command():
readme = (REPO_ROOT / "README.md").read_text()
assert "uvicorn app.main:app --reload" not in readme
def test_running_the_server_doc_has_no_bare_uvicorn_command():
doc = (REPO_ROOT / "docs" / "running-the-server.md").read_text()
assert "uvicorn app.main:app --reload" not in doc
+16
View File
@@ -0,0 +1,16 @@
"""B-47: raw_tx_hex (a full raw signed transaction hex) and payload_json (an
arbitrary audit payload) must stay `Text`, not a bare `String`/`VARCHAR` with
no length -- SQLite and PostgreSQL accept that, but other backends (e.g.
MySQL) require a length on VARCHAR and would reject it."""
from sqlalchemy import Text
from app.db.models import AuditLog, PendingTransaction
def test_pending_transaction_raw_tx_hex_is_text():
assert isinstance(PendingTransaction.__table__.c.raw_tx_hex.type, Text)
def test_audit_log_payload_json_is_text():
assert isinstance(AuditLog.__table__.c.payload_json.type, Text)
+55
View File
@@ -0,0 +1,55 @@
"""B-41: own_address_for is the single place tx/confirmation.py and
tx/reconcile.py derive a PendingTransaction's own address from — a payout's
address must always be the pool's, everything else the actual user's."""
import pytest
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from app.config import settings
from app.db.base import Base
from app.db.models import User
from app.tx.pending_address import own_address_for
@pytest.fixture
async def session_factory(tmp_path, monkeypatch):
monkeypatch.setattr(settings, "master_key_path", str(tmp_path / "master.xprv.enc"))
monkeypatch.setattr(
settings,
"xprv_encryption_key",
__import__("cryptography.fernet", fromlist=["Fernet"]).Fernet.generate_key().decode(),
)
from app.wallet import hd
hd._account_key = None
hd.generate_master_key()
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
yield async_sessionmaker(engine, expire_on_commit=False)
await engine.dispose()
hd._account_key = None
async def test_payout_uses_the_pool_address_regardless_of_user_id(session_factory):
from app.wallet.hd import derive_pool_address
async with session_factory() as session:
address = await own_address_for(session, "payout", None)
assert address == derive_pool_address()
@pytest.mark.parametrize("kind", ["bet", "withdrawal"])
async def test_bet_and_withdrawal_use_the_users_own_address(session_factory, kind):
from app.wallet.hd import derive_user_address
async with session_factory() as session:
user = User(username="alice", password_hash="x", derivation_index=3, address=derive_user_address(3))
session.add(user)
await session.flush()
address = await own_address_for(session, kind, user.id)
assert address == derive_user_address(3)
+88 -11
View File
@@ -1,5 +1,10 @@
"""Regression tests for B-04 (and the "building" half of B-08): a transaction that """Regression tests for B-04 (and the "building" half of B-08): a transaction that
never made it onto the chain must give the coins back instead of freezing them.""" never made it onto the chain must give the coins back instead of freezing them.
Also covers B-41: existence/reconciliation checks go through
blockchain.scripthash.get_history rather than a verbose blockchain.transaction.get
reply, so the fake clients below implement get_history.
"""
import pytest import pytest
from embit import script from embit import script
@@ -7,37 +12,60 @@ from embit.transaction import Transaction, TransactionInput, TransactionOutput
from sqlalchemy import select from sqlalchemy import select
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from app.config import settings
from app.db.base import Base from app.db.base import Base
from app.db.models import AuditLog, PendingTransaction, RoundParticipant, User, UtxoEvent, Withdrawal from app.db.models import AuditLog, PendingTransaction, RoundParticipant, User, UtxoEvent, Withdrawal
from app.tx.reconcile import reconcile_once from app.tx.reconcile import reconcile_once
class UnknownTxClient: class UnknownTxClient:
"""A server that doesn't know any of the txids it's asked about.""" """A server whose history for any address never includes our txid."""
async def get_transaction(self, txid: str, verbose: bool = False): async def get_history(self, scripthash: str) -> list[dict]:
raise RuntimeError(f"missing transaction {txid}") return []
class KnownTxClient: class KnownTxClient:
async def get_transaction(self, txid: str, verbose: bool = False): """A server whose history for the address includes our txid — mined or
return {"txid": txid, "confirmations": 0} still in the mempool doesn't matter for existence, only for confirmation
(which is tx/confirmation.py's concern, not reconcile.py's)."""
def __init__(self, txid: str = "betxid"):
self._txid = txid
async def get_history(self, scripthash: str) -> list[dict]:
return [{"tx_hash": self._txid, "height": 100}]
class BrokenClient: class BrokenClient:
"""A transport failure — says nothing about whether the tx exists.""" """A transport failure — says nothing about whether the tx exists."""
async def get_transaction(self, txid: str, verbose: bool = False): async def get_history(self, scripthash: str) -> list[dict]:
raise ConnectionResetError("connection reset") raise ConnectionResetError("connection reset")
@pytest.fixture @pytest.fixture
async def session_factory(): async def session_factory(tmp_path, monkeypatch):
# own_address_for (B-41) derives each row's address via the HD wallet rather
# than trusting the DB's address column, so reconcile_once now needs a real
# master key set up — same bootstrap test_broadcast.py uses.
monkeypatch.setattr(settings, "master_key_path", str(tmp_path / "master.xprv.enc"))
monkeypatch.setattr(
settings,
"xprv_encryption_key",
__import__("cryptography.fernet", fromlist=["Fernet"]).Fernet.generate_key().decode(),
)
from app.wallet import hd
hd._account_key = None
hd.generate_master_key()
engine = create_async_engine("sqlite+aiosqlite:///:memory:") engine = create_async_engine("sqlite+aiosqlite:///:memory:")
async with engine.begin() as conn: async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all) await conn.run_sync(Base.metadata.create_all)
yield async_sessionmaker(engine, expire_on_commit=False) yield async_sessionmaker(engine, expire_on_commit=False)
await engine.dispose() await engine.dispose()
hd._account_key = None
# A real (unsigned) transaction spending one input, built rather than hand-written # A real (unsigned) transaction spending one input, built rather than hand-written
@@ -66,11 +94,19 @@ async def _seed_bet(
participant_status: str, participant_status: str,
age_seconds: int, age_seconds: int,
last_broadcast_age_seconds: int | None = None, last_broadcast_age_seconds: int | None = None,
derivation_index: int = 0,
): ):
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from app.wallet.hd import derive_user_address
async with session_factory() as session: async with session_factory() as session:
user = User(username="u", password_hash="x", derivation_index=0, address="plm1qtest") user = User(
username="u",
password_hash="x",
derivation_index=derivation_index,
address=derive_user_address(derivation_index),
)
session.add(user) session.add(user)
await session.flush() await session.flush()
session.add( session.add(
@@ -145,7 +181,7 @@ async def test_promotes_a_building_row_whose_tx_did_reach_the_chain(session_fact
session_factory, pending_status="building", participant_status="building", age_seconds=300 session_factory, pending_status="building", participant_status="building", age_seconds=300
) )
resolved = await reconcile_once(session_factory, KnownTxClient()) resolved = await reconcile_once(session_factory, KnownTxClient("betxid"))
assert resolved == 1 assert resolved == 1
async with session_factory() as session: async with session_factory() as session:
@@ -219,8 +255,10 @@ async def test_abandoned_withdrawal_is_marked_failed_and_kept(session_factory):
they can see it didn't go through.""" they can see it didn't go through."""
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from app.wallet.hd import derive_user_address
async with session_factory() as session: async with session_factory() as session:
user = User(username="w", password_hash="x", derivation_index=1, address="plm1qtest2") user = User(username="w", password_hash="x", derivation_index=1, address=derive_user_address(1))
session.add(user) session.add(user)
await session.flush() await session.flush()
session.add( session.add(
@@ -264,3 +302,42 @@ async def test_abandoned_withdrawal_is_marked_failed_and_kept(session_factory):
assert withdrawal.status == "failed" assert withdrawal.status == "failed"
assert withdrawal.txid is None assert withdrawal.txid is None
assert (await session.scalars(select(UtxoEvent))).one().spent_txid is None assert (await session.scalars(select(UtxoEvent))).one().spent_txid is None
# --- B-41: existence checks now use get_history and share it across candidates
# sharing the same address, instead of a per-tx verbose blockchain.transaction.get. --
async def test_reconcile_once_caches_history_per_scripthash(session_factory):
"""Two payout PendingTransaction rows always share the same pool address —
fetching its history twice in one pass would be wasteful and, at scale
across many candidates on one address, needlessly slow the whole tick."""
from datetime import datetime, timedelta, timezone
async with session_factory() as session:
old = datetime.now(timezone.utc) - timedelta(hours=7)
session.add(
PendingTransaction(
kind="payout", round_id=1, current_txid="payout-a", fee_rate_sat_vb=1,
raw_tx_hex=_RAW_TX, status="pending", broadcast_at=old, last_broadcast_at=old,
)
)
session.add(
PendingTransaction(
kind="payout", round_id=2, current_txid="payout-b", fee_rate_sat_vb=1,
raw_tx_hex=_RAW_TX, status="pending", broadcast_at=old, last_broadcast_at=old,
)
)
await session.commit()
call_count = {"n": 0}
class CountingClient:
async def get_history(self, scripthash: str) -> list[dict]:
call_count["n"] += 1
return [{"tx_hash": "payout-a", "height": 100}, {"tx_hash": "payout-b", "height": 100}]
resolved = await reconcile_once(session_factory, CountingClient())
assert resolved == 0 # both exist — nothing to abandon or promote (already "pending")
assert call_count["n"] == 1 # one call covered both rows sharing the pool address
+69 -1
View File
@@ -2,7 +2,7 @@ import asyncio
import pytest import pytest
from app.rounds.events import RoundEventBroadcaster, RoundEventCapacityError from app.rounds.events import EVICTED, RoundEventBroadcaster, RoundEventCapacityError
async def test_publish_wakes_up_subscriber(): async def test_publish_wakes_up_subscriber():
@@ -60,3 +60,71 @@ async def test_unsubscribe_frees_a_capacity_slot():
broadcaster.unsubscribe(queue) broadcaster.unsubscribe(queue)
broadcaster.subscribe() # no longer at capacity broadcaster.subscribe() # no longer at capacity
# --- B-38: a single IP must not be able to exhaust the global cap and degrade
# every other user to polling. ----------------------------------------------------
async def test_subscribe_evicts_the_same_ips_oldest_connection_past_its_cap():
"""Past MAX_SUBSCRIBERS_PER_IP, one more stream from the *same* IP evicts
that IP's own oldest connection rather than being refused — bounds one
source's footprint without an outright block."""
broadcaster = RoundEventBroadcaster(max_subscribers=100, max_per_ip=2)
first = broadcaster.subscribe("1.2.3.4")
second = broadcaster.subscribe("1.2.3.4")
third = broadcaster.subscribe("1.2.3.4") # past the per-IP cap of 2
assert await asyncio.wait_for(first.get(), timeout=1) is EVICTED
assert second.empty() # untouched — only the oldest was evicted
assert third is not None
async def test_subscribe_does_not_evict_across_different_ips():
"""A different IP hitting its own cap must never evict an unrelated IP's
connection — that would let one abusive source crowd out real users, which
is exactly what the global-only cap used to allow."""
broadcaster = RoundEventBroadcaster(max_subscribers=100, max_per_ip=1)
other_ip_queue = broadcaster.subscribe("9.9.9.9")
broadcaster.subscribe("1.2.3.4")
broadcaster.subscribe("1.2.3.4") # evicts 1.2.3.4's own oldest, not 9.9.9.9's
assert other_ip_queue.empty()
async def test_subscribe_still_enforces_the_global_cap_across_many_ips():
"""The per-IP cap doesn't replace the global backstop — spreading across
enough distinct IPs must still eventually hit MAX_SUBSCRIBERS."""
broadcaster = RoundEventBroadcaster(max_subscribers=3, max_per_ip=1)
broadcaster.subscribe("1.1.1.1")
broadcaster.subscribe("2.2.2.2")
broadcaster.subscribe("3.3.3.3")
with pytest.raises(RoundEventCapacityError):
broadcaster.subscribe("4.4.4.4")
async def test_unsubscribe_clears_the_per_ip_tracking_too():
"""Regression guard: unsubscribe must forget the queue's IP association, or
a churned-through connection would keep counting against that IP's cap
forever."""
broadcaster = RoundEventBroadcaster(max_subscribers=100, max_per_ip=1)
queue = broadcaster.subscribe("1.2.3.4")
broadcaster.unsubscribe(queue)
broadcaster.subscribe("1.2.3.4") # must not evict anything — nothing left to evict
assert queue.empty()
async def test_subscribe_defaults_to_a_shared_ip_when_none_given():
"""Existing callers (and most tests) that don't care about IP isolation
still share one implicit bucket rather than needing every call updated."""
broadcaster = RoundEventBroadcaster(max_subscribers=100, max_per_ip=2)
first = broadcaster.subscribe()
broadcaster.subscribe()
broadcaster.subscribe() # past the default bucket's cap of 2 — evicts, doesn't raise
assert await asyncio.wait_for(first.get(), timeout=1) is EVICTED