Compare commits
10
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6a90136b50 | ||
|
|
31bc9a327f | ||
|
|
6045c89ed0 | ||
|
|
a574db0d93 | ||
|
|
d60da11603 | ||
|
|
22e3cfb2be | ||
|
|
4124dc08e6 | ||
|
|
08c566d547 | ||
|
|
97545ad91f | ||
|
|
702b37b319 |
No files matched your search
@@ -30,3 +30,8 @@ ADMIN_TOKEN=
|
||||
# Every business/round parameter (bet amount, round duration/cooldown, min
|
||||
# amount, fee rate, RBF timeout, fee address) is configured live from the
|
||||
# admin panel (/admin) instead of here — see docs/guida-admin.md.
|
||||
|
||||
# Swagger/ReDoc/the raw OpenAPI JSON expose the entire API surface — admin
|
||||
# endpoints included — to anyone who requests them. Off by default; set to
|
||||
# true only for local development, never in production.
|
||||
ENABLE_API_DOCS=false
|
||||
@@ -1,11 +1,11 @@
|
||||
# Known bugs
|
||||
|
||||
A second full-codebase audit on 2026-07-27 found **25 further issues** (4 critical, 6 high,
|
||||
7 medium, 8 low), listed below as B-33 … B-49. B-25 through B-37 are fixed (see "Previously
|
||||
fixed" below) — no Critical-severity finding remains open; the other 12 are High/Medium/Low.
|
||||
The 139-test suite was green at the time of the audit, so none of these were caught by existing
|
||||
coverage — every fix lands with a regression test (the thirteen fixes so far brought the suite
|
||||
from 139 to 201).
|
||||
7 medium, 8 low), listed below as B-48 … B-49. B-25 through B-47 are fixed (see "Previously
|
||||
fixed" below) — no Critical-, High- or Medium-severity finding remains open; the remaining 2 are
|
||||
Low/hygiene. The 139-test suite was green at the time of the audit, so none of these were caught
|
||||
by existing coverage — every fix lands with a regression test (the twenty-three fixes so far
|
||||
brought the suite from 139 to 248).
|
||||
|
||||
The recurring pattern across the open findings is worth stating once: the code is rigorous
|
||||
about the failure modes that have actually been hit, and silent about the ones that have not.
|
||||
@@ -16,113 +16,8 @@ admin auth, single-process assumptions, no user-facing history, etc.) are docume
|
||||
|
||||
---
|
||||
|
||||
## Medium
|
||||
|
||||
### B-38 — The 500-subscriber SSE cap is a zero-cost DoS of the realtime feature
|
||||
|
||||
`GET /rounds/stream` requires no authentication and each connection takes a slot on a
|
||||
**global** counter (`rounds/events.py:33-38`). Anyone opening 500 connections degrades every
|
||||
real user to polling. The comment describes it as a defensive cap; it is in fact the vector,
|
||||
not the defence.
|
||||
|
||||
**Proposed fix.** Cap per client IP (and, once available, per authenticated user) rather than
|
||||
globally, and evict the oldest idle subscriber instead of refusing new ones. The reverse proxy
|
||||
is the right place for the connection-count limit — Caddy can enforce it before the request
|
||||
reaches the app.
|
||||
|
||||
### B-39 — SQLite with no WAL, no `busy_timeout`, and five concurrent writer tasks
|
||||
|
||||
`db/base.py:6` calls `create_async_engine(settings.database_url)` with no `connect_args`, and
|
||||
there is no `PRAGMA` anywhere in the repo (verified by grep). Without `journal_mode=WAL`
|
||||
readers block writers, and the concurrent writers are five background tasks plus every HTTP
|
||||
handler. `database is locked` under load is realistic, and nothing handles it.
|
||||
|
||||
**Proposed fix.** Set `journal_mode=WAL`, `synchronous=NORMAL` and a `busy_timeout` of a few
|
||||
seconds on connect (a `connect` event listener on the engine, applied only for the SQLite
|
||||
dialect), and retry `OperationalError: database is locked` in the background loops. Longer
|
||||
term this is an argument for PostgreSQL, which the single-process constraints in CLAUDE.md
|
||||
also point at.
|
||||
|
||||
### B-40 — `bump_fee` holds a DB session open across N network calls
|
||||
|
||||
`tx/broadcast.py:80` issues one `get_transaction` **per input** (up to 15s each) and then a
|
||||
`broadcast`, all with the session open. This is precisely the pattern B-18 removed from
|
||||
`_trigger_payout` via its three-phase structure; it survives here.
|
||||
|
||||
Side note in the same function: `_prevout_amount` does `round(value_coins * 100_000_000)` on a
|
||||
float from the server — acceptable at these magnitudes, but it is floating-point money
|
||||
arithmetic in a codebase that is otherwise strictly integer-satoshi.
|
||||
|
||||
**Proposed fix.** Restructure into the same three phases: read what is needed and close the
|
||||
session, do the chain work, then reopen to persist. For the float: prefer the raw (non-verbose)
|
||||
transaction and parse the output value as an integer with `embit`, which is what
|
||||
`reconcile.py:_release_inputs` already does for inputs.
|
||||
|
||||
### B-41 — All confirmation logic depends on `verbose=True`, which is not universally supported
|
||||
|
||||
`poll_once`, `reconcile._tx_exists_on_chain` and `bump_fee` all call
|
||||
`blockchain.transaction.get(txid, True)`. Several Electrum server implementations and versions
|
||||
reject the verbose flag ("verbose transactions are currently unsupported"). Falling back onto
|
||||
such a server means **no confirmations, no reconciliation, no bumps** — and the code would read
|
||||
that as a transport error and stay silent.
|
||||
|
||||
Related: `reconcile.py:83` decides whether to **abandon a transaction** by substring-matching
|
||||
the error text (`"missing"`, `"not found"`, `"no such"`, `"unknown"`). It works against
|
||||
ElectrumX; it is fragile as the basis for a decision that releases funds.
|
||||
|
||||
**Proposed fix.** Use `blockchain.transaction.get_merkle` (or the scripthash history) for
|
||||
confirmation and existence checks — both are portable and give the confirming height directly.
|
||||
Probe verbose support once at connect time and record it on the client, so an unsupported
|
||||
server is detected loudly at session start rather than silently mid-operation.
|
||||
|
||||
---
|
||||
|
||||
## Low / hygiene
|
||||
|
||||
### B-42 — `/docs` exposed in production
|
||||
|
||||
FastAPI mounts Swagger by default, so the entire API surface — `/admin` included — is publicly
|
||||
enumerable. The README advertises it.
|
||||
**Fix:** `docs_url=None, redoc_url=None, openapi_url=None` in production (env-gated), or place
|
||||
them behind `require_admin`.
|
||||
|
||||
### B-43 — No HTTP security headers
|
||||
|
||||
The [Caddyfile](Caddyfile) sets no CSP, no `X-Frame-Options`/`frame-ancestors`, and no HSTS
|
||||
(Caddy does not add it on its own). The JWT lives in `localStorage`, so any XSS exfiltrates
|
||||
it, and the page is iframeable.
|
||||
**Fix:** a `header` block in the Caddyfile with `Strict-Transport-Security`,
|
||||
`X-Content-Type-Options: nosniff`, `Referrer-Policy` and a CSP tight enough for two static
|
||||
pages with no external assets (`default-src 'self'`).
|
||||
|
||||
### B-44 — README and CLAUDE.md contradict each other
|
||||
|
||||
The README says to run `uvicorn --reload` directly and
|
||||
`docker compose run --rm app python scripts/generate_master_key.py`; CLAUDE.md says explicitly
|
||||
that neither is supported. Whoever opens the repo reads the README first.
|
||||
**Fix:** align the README's Quick start with the Docker-only workflow documented in
|
||||
CLAUDE.md and `docs/setup.md`.
|
||||
|
||||
### B-45 — Unvalidated and unpaginated admin list endpoints
|
||||
|
||||
`limit: int = 50` on `/admin/rounds` and `/admin/audit-log` has no bounds (`-1` means
|
||||
"everything" on SQLite), and `/admin/pending-transactions` has no limit at all — it grows
|
||||
without end.
|
||||
**Fix:** `Query(default=50, ge=1, le=500)` on both, and the same treatment plus a status filter
|
||||
on the pending-transaction list.
|
||||
|
||||
### B-46 — `secrets.compare_digest` on a `str` raises on non-ASCII input
|
||||
|
||||
`api/routes/admin.py:27` raises `TypeError` — a 500 instead of a 403 — when the header contains
|
||||
non-ASCII characters.
|
||||
**Fix:** compare the UTF-8 encoded bytes of both sides.
|
||||
|
||||
### B-47 — Unbounded `String` columns for large text
|
||||
|
||||
`raw_tx_hex` (`db/models.py:146`) and `payload_json` (`:178`) should be `Text`. It works on
|
||||
SQLite and PostgreSQL and breaks elsewhere.
|
||||
**Fix:** switch both to `Text` in a migration.
|
||||
|
||||
### B-48 — No cap on input count in `select_utxos`
|
||||
|
||||
A user with hundreds of small UTXOs builds a huge transaction whose fee — deducted from the bet
|
||||
@@ -150,15 +45,25 @@ already does.
|
||||
- **B-29** — a UTXO absent from one server's `listunspent` was marked spent immediately, irreversibly, on a single unauthenticated reply
|
||||
- **B-30** — a lost scripthash subscription meant a user's deposits were never credited, with no periodic safety net
|
||||
- **B-31** — resubscribing on reconnect ran serially before anything else started, freezing the chain tip (and so an in-flight draw) for the whole sweep
|
||||
- **B-42** — Swagger/ReDoc/the raw OpenAPI JSON enumerated the entire API surface, admin endpoints included, to anyone who requested them; now off by default and gated behind `ENABLE_API_DOCS`
|
||||
- **B-43** — the Caddyfile sent no CSP, no `X-Frame-Options`/`frame-ancestors`, and no HSTS, on a page whose JWT lives in `localStorage`
|
||||
- **B-44** — README's Quick start documented a bare `uvicorn --reload` workflow, and `docs/running-the-server.md` still had a matching "Locale / venv" section, both contradicting CLAUDE.md's Docker-only policy
|
||||
- **B-45** — `/admin/rounds`/`/admin/audit-log`'s `limit` had no bounds (`-1` means "everything" on SQLite), and `/admin/pending-transactions` had no limit or status filter at all
|
||||
- **B-46** — `secrets.compare_digest` on a `str` raises `TypeError` on non-ASCII input, turning an invalid admin token with non-ASCII characters into a 500 instead of a 403
|
||||
- **B-47** — `raw_tx_hex` and `payload_json` were unbounded `String` columns (`VARCHAR` with no length) — fine on SQLite/PostgreSQL, rejected by backends like MySQL that require a length
|
||||
- **B-32** — an RBF bump could retry forever below BIP125's relay-mandated minimum fee delta, with no ceiling on the fee rate either
|
||||
- **B-33** — `POST /auth/login` had no rate limiting, so a password could be brute-forced against an enumerable username list
|
||||
- **B-34** — password change/reset didn't invalidate already-issued JWTs, so a stolen token survived a change meant to lock it out
|
||||
- **B-35** — API timestamps round-tripped as naive datetimes, so the frontend parsed them as local time instead of UTC
|
||||
- **B-36** — a stalled draw wait had no timeout, no log, and no audit trail, so a frozen round showed nothing in `/admin`
|
||||
- **B-37** — a withdrawal covered by unconfirmed change answered "insufficient balance" instead of distinguishing it from actually having no funds
|
||||
- **B-38** — the SSE subscriber cap was global, so one client opening enough connections degraded every other user to polling
|
||||
- **B-39** — SQLite ran without WAL or a `busy_timeout`, so a writer could block every reader and a second writer failed immediately instead of waiting
|
||||
- **B-40** — `bump_fee` held a DB session open across N slow network calls, and computed a prevout's value from a server-reported float instead of an exact integer
|
||||
- **B-41** — confirmation/reconciliation depended on a verbose `blockchain.transaction.get` reply many Electrum servers reject, and abandonment relied on fragile substring-matching of an error message
|
||||
|
||||
See git history for the fix-by-fix breakdown (commits `f13f685`, `50a43ae`, `933760e`, and the
|
||||
B-28/B-29/B-30/B-31/B-32/B-33/B-34/B-35/B-36/B-37 fixes). Suite grew from 139 to 201 tests over the thirteen.
|
||||
B-28/B-29/B-30/B-31/B-32/B-33/B-34/B-35/B-36/B-37/B-38/B-39/B-40/B-41/B-42/B-43/B-44/B-45/B-46/B-47 fixes). Suite grew from 139 to 248 tests over the twenty-three.
|
||||
|
||||
A full-codebase audit on 2026-07-26 (commit `d4e0974`) found 24 bugs across every Python
|
||||
module under `app/`, both static frontends, and the Docker/Caddy deployment — 5 critical,
|
||||
|
||||
@@ -12,11 +12,11 @@ All 10 stages of the original build order are code-complete and unit-tested —
|
||||
|
||||
Verified on mainnet with real money: registration + address derivation, deposit crediting (1-conf), a real 10 PLM bet (broadcast → confirmed → change credited back), and one full round cycle (close → draw on a real block hash → 70/30 payout with sat math checked against the broadcast tx → confirmation → close → next round auto-opened). **Withdrawal and the RBF bump path have never been exercised against a live broadcast** — unit-tested only.
|
||||
|
||||
**Read [BUGS.md](BUGS.md) before trusting any behaviour here.** Two audits: 2026-07-26 found 24 bugs (5 critical), all fixed; 2026-07-27 found 25 more (B-25 … B-49), of which **12 are still open** — no Critical or High remains, only Medium/Low: no WAL/`busy_timeout` under five concurrent SQLite writer tasks (B-39), a 500-subscriber SSE cap that doubles as a cheap DoS of the realtime feature (B-38), among others. BUGS.md is the live open list with a proposed fix per finding; "Known gaps" at the end of this file is for limitations accepted **by design** instead. Don't fix a BUGS.md item silently as a side effect of other work — each fix lands with its own regression test.
|
||||
**Read [BUGS.md](BUGS.md) before trusting any behaviour here.** Two audits: 2026-07-26 found 24 bugs (5 critical), all fixed; 2026-07-27 found 25 more (B-25 … B-49), of which **2 are still open** — no Critical, High or Medium remains, only Low/hygiene: no cap on input count in `select_utxos` (B-48) and rollback paths not publishing an SSE update (B-49). BUGS.md is the live open list with a proposed fix per finding; "Known gaps" at the end of this file is for limitations accepted **by design** instead. Don't fix a BUGS.md item silently as a side effect of other work — each fix lands with its own regression test.
|
||||
|
||||
Before writing code, read the "Architecture" section below in full plus the diagrams in [flowchart/](flowchart/): [platform-overview.mmd](flowchart/platform-overview.mmd) (the 5-phase flow) and [round-lifecycle.mmd](flowchart/round-lifecycle.mmd) (the round/draw lifecycle). Every node **and edge label** (conditions, retries, loops) is a behaviour that must be implemented as described. Regenerate the companion PDFs with `flowchart/render-pdf.sh <file>.mmd` after editing either.
|
||||
|
||||
Human-facing guides are in [docs/](docs/), in Italian by explicit request (an exception to the English-only rule): [setup.md](docs/setup.md), [running-the-server.md](docs/running-the-server.md), [guida-utente.md](docs/guida-utente.md), [guida-admin.md](docs/guida-admin.md). **[README.md](README.md)'s Quick start is stale** (bare `uvicorn --reload`, `docker compose run … generate_master_key.py` — neither is supported; B-44); this file and `docs/setup.md` are authoritative.
|
||||
Human-facing guides are in [docs/](docs/), in Italian by explicit request (an exception to the English-only rule): [setup.md](docs/setup.md), [running-the-server.md](docs/running-the-server.md), [guida-utente.md](docs/guida-utente.md), [guida-admin.md](docs/guida-admin.md). README's Quick start and `docs/running-the-server.md` are Docker-only, matching this file — a bare `uvicorn --reload` workflow was removed from both (B-44).
|
||||
|
||||
## Commands
|
||||
|
||||
@@ -55,7 +55,7 @@ docker compose down
|
||||
|
||||
`SITE_ADDRESS` unset → `localhost`, Caddy issues a self-signed cert from its internal CA (browser warning on first visit is expected; `curl -k`). `SITE_ADDRESS=lottery.example.com docker compose up -d` → real Let's Encrypt cert, automatically renewed (needs DNS pointing here and ports 80+443 reachable).
|
||||
|
||||
The `Caddyfile` sets **no** security headers — no CSP, HSTS or `X-Frame-Options` (B-43). `restart: unless-stopped` on `app` means a mid-round crash auto-restarts: `closing` and `paying_out` resume on their own, `drawing` does not (see Known gaps).
|
||||
The `Caddyfile` sends baseline security headers — HSTS, `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Referrer-Policy`, and a CSP scoped to `default-src 'self'` plus the Google Fonts `@import` in `style.css`/`admin.css`. `script-src`/`style-src` need `'unsafe-inline'` because both SPAs use inline `onclick` handlers and `style=""` attributes throughout — removing those is a separate, larger refactor, not a header change. `restart: unless-stopped` on `app` means a mid-round crash auto-restarts: `closing` and `paying_out` resume on their own, `drawing` does not (see Known gaps).
|
||||
|
||||
## Tech stack
|
||||
|
||||
@@ -233,7 +233,7 @@ Explicit design choices, not derivable from any single file — respect them:
|
||||
|
||||
## Known gaps / TODO
|
||||
|
||||
Accepted **by design**. For actual bugs see [BUGS.md](BUGS.md) (18 open) — not duplicated here.
|
||||
Accepted **by design**. For actual bugs see [BUGS.md](BUGS.md) (2 open) — not duplicated here.
|
||||
|
||||
- **`drawing` doesn't resume after a restart.** `_tick()` handles `open`, `closing` and `paying_out` (the last via `_retry_payout_if_due`); nothing re-enters `_wait_for_next_block` after a crash. That wait is unbounded by design (the draw's entropy genuinely depends on a future block) but no longer silent — past `_DRAW_STALL_THRESHOLD_SECONDS` it logs progress and writes a `draw_stalled` audit entry, and `GET /rounds/current`'s `draw_waiting_since` surfaces it live (B-36). Restart-resumption itself remains the last prerequisite for running unattended.
|
||||
- **RBF handles one shape only**: a single change output, back to the tx's own sender, big enough to absorb the increase. No extra-input fallback — an exact-amount tx or too-small change raises `RbfError`. Not permanent, though: an unbumpable tx that never confirms is eventually abandoned and its UTXOs released.
|
||||
|
||||
@@ -13,5 +13,21 @@
|
||||
not path /rounds/stream
|
||||
}
|
||||
encode @not_sse gzip
|
||||
|
||||
# B-43: Caddy adds none of these on its own. The JWT lives in
|
||||
# localStorage, so any XSS exfiltrates it — CSP is the main mitigation.
|
||||
# script-src/style-src need 'unsafe-inline' because both SPAs
|
||||
# (app/static/index.html, admin.html) use inline onclick handlers and
|
||||
# style="" attributes throughout; removing those is a separate,
|
||||
# larger refactor, not a header change. fonts.googleapis.com/gstatic.com
|
||||
# are the one external asset (the Google Fonts @import in style.css/admin.css).
|
||||
header {
|
||||
Strict-Transport-Security "max-age=31536000; includeSubDomains"
|
||||
X-Content-Type-Options "nosniff"
|
||||
X-Frame-Options "DENY"
|
||||
Referrer-Policy "strict-origin-when-cross-origin"
|
||||
Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; font-src 'self' https://fonts.gstatic.com; img-src 'self'; connect-src 'self'; base-uri 'self'; form-action 'self'; frame-ancestors 'none'; object-src 'none'"
|
||||
}
|
||||
|
||||
reverse_proxy app:8123
|
||||
}
|
||||
@@ -13,31 +13,30 @@ production-ready.
|
||||
|
||||
## Quick start
|
||||
|
||||
The server always runs via Docker (app + Caddy reverse proxy with automatic
|
||||
TLS) — in dev and production alike, with only `SITE_ADDRESS` differing
|
||||
between the two. There's no supported way to run `uvicorn` directly; the
|
||||
venv is only for local tooling (tests, Alembic migrations, the one-time key
|
||||
scripts) — see [CLAUDE.md](CLAUDE.md#commands).
|
||||
|
||||
```bash
|
||||
cp .env.example .env # then fill in the generated secrets, see docs/setup.md
|
||||
python3 -m venv .venv && source .venv/bin/activate
|
||||
pip install -e ".[dev]"
|
||||
PYTHONPATH=. python scripts/generate_master_key.py
|
||||
alembic upgrade head
|
||||
uvicorn app.main:app --reload --port 8123
|
||||
```
|
||||
|
||||
Open `http://127.0.0.1:8123/` for the test UI, `http://127.0.0.1:8123/admin`
|
||||
for the admin dashboard, `http://127.0.0.1:8123/docs` for the interactive API
|
||||
docs.
|
||||
|
||||
Or run the whole stack (app + Caddy reverse proxy with automatic TLS) via
|
||||
Docker:
|
||||
|
||||
```bash
|
||||
mkdir -p data/db data/keys data/logs
|
||||
docker compose run --rm app python scripts/generate_master_key.py
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
Open `https://localhost/` for the test UI, `https://localhost/admin` for the
|
||||
admin dashboard (a self-signed-certificate warning on first visit is
|
||||
expected in dev — accept it, or use `curl -k`). The interactive API docs at
|
||||
`/docs` are disabled by default (they'd otherwise expose the whole API
|
||||
surface, admin endpoints included) — set `ENABLE_API_DOCS=true` in `.env` to
|
||||
enable them.
|
||||
|
||||
See [docs/setup.md](docs/setup.md) and
|
||||
[docs/running-the-server.md](docs/running-the-server.md) for the full
|
||||
walkthrough (both workflows, dev vs. production TLS).
|
||||
walkthrough (secrets, master key generation, production TLS with a real
|
||||
domain).
|
||||
|
||||
## Documentation
|
||||
|
||||
@@ -62,7 +61,7 @@ python -m pytest # all tests
|
||||
python -m pytest tests/unit/test_hd.py # one file
|
||||
```
|
||||
|
||||
76 unit tests cover HD derivation, PSBT building, the Electrum client, bets,
|
||||
232 unit tests cover HD derivation, PSBT building, the Electrum client, bets,
|
||||
deposits, withdrawals, the round/draw engine, RBF fee-bumping, admin config,
|
||||
the pending-inclusive balance calculation, and the SSE push channel. No
|
||||
automated integration tests against a live Electrum connection — mainnet
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
from fastapi import Request
|
||||
|
||||
|
||||
def client_ip(request: Request) -> str:
|
||||
"""The caller's real IP, from Caddy's X-Forwarded-For (see Caddyfile) —
|
||||
request.client.host would otherwise be the reverse proxy's own address, not
|
||||
the caller's. Falls back to request.client.host only if the header is
|
||||
somehow missing (e.g. the app container hit directly, bypassing Caddy).
|
||||
|
||||
Shared by the login/registration throttles (B-33) and the SSE per-IP
|
||||
subscriber cap (B-38) so the two can't drift into different notions of
|
||||
"the client's IP".
|
||||
"""
|
||||
forwarded = request.headers.get("x-forwarded-for")
|
||||
if forwarded:
|
||||
return forwarded.split(",")[0].strip()
|
||||
return request.client.host if request.client else "unknown"
|
||||
+14
-7
@@ -1,7 +1,7 @@
|
||||
import json
|
||||
import secrets
|
||||
|
||||
from fastapi import APIRouter, Depends, Header, HTTPException, status
|
||||
from fastapi import APIRouter, Depends, Header, HTTPException, Query, status
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
@@ -26,7 +26,9 @@ async def require_admin(x_admin_token: str = Header(default="")) -> None:
|
||||
# anyone on an instance that never configured a token.
|
||||
if not settings.admin_token:
|
||||
raise HTTPException(status.HTTP_403_FORBIDDEN, "invalid admin token")
|
||||
if not secrets.compare_digest(x_admin_token, settings.admin_token):
|
||||
# compare_digest raises TypeError on a str containing non-ASCII characters
|
||||
# (B-46) -- comparing the UTF-8 bytes instead accepts any input safely.
|
||||
if not secrets.compare_digest(x_admin_token.encode(), settings.admin_token.encode()):
|
||||
raise HTTPException(status.HTTP_403_FORBIDDEN, "invalid admin token")
|
||||
|
||||
|
||||
@@ -242,7 +244,9 @@ class AdminRoundResponse(BaseModel):
|
||||
|
||||
|
||||
@router.get("/rounds", response_model=list[AdminRoundResponse], dependencies=[Depends(require_admin)])
|
||||
async def list_rounds(session: AsyncSession = Depends(get_session), limit: int = 50) -> list[AdminRoundResponse]:
|
||||
async def list_rounds(
|
||||
session: AsyncSession = Depends(get_session), limit: int = Query(default=50, ge=1, le=500)
|
||||
) -> list[AdminRoundResponse]:
|
||||
rounds = (await session.scalars(select(Round).order_by(Round.id.desc()).limit(limit))).all()
|
||||
winner_ids = {r.winner_user_id for r in rounds if r.winner_user_id is not None}
|
||||
winners = {}
|
||||
@@ -282,7 +286,7 @@ class AdminAuditLogResponse(BaseModel):
|
||||
"/audit-log", response_model=list[AdminAuditLogResponse], dependencies=[Depends(require_admin)]
|
||||
)
|
||||
async def list_audit_log(
|
||||
session: AsyncSession = Depends(get_session), limit: int = 200
|
||||
session: AsyncSession = Depends(get_session), limit: int = Query(default=200, ge=1, le=500)
|
||||
) -> list[AdminAuditLogResponse]:
|
||||
entries = (await session.scalars(select(AuditLog).order_by(AuditLog.id.desc()).limit(limit))).all()
|
||||
return [
|
||||
@@ -319,10 +323,13 @@ class AdminPendingTransactionResponse(BaseModel):
|
||||
)
|
||||
async def list_pending_transactions(
|
||||
session: AsyncSession = Depends(get_session),
|
||||
limit: int = Query(default=50, ge=1, le=500),
|
||||
status_filter: str | None = Query(default=None, alias="status"),
|
||||
) -> list[AdminPendingTransactionResponse]:
|
||||
entries = (
|
||||
await session.scalars(select(PendingTransaction).order_by(PendingTransaction.id.desc()))
|
||||
).all()
|
||||
query = select(PendingTransaction).order_by(PendingTransaction.id.desc())
|
||||
if status_filter is not None:
|
||||
query = query.where(PendingTransaction.status == status_filter)
|
||||
entries = (await session.scalars(query.limit(limit))).all()
|
||||
return [
|
||||
AdminPendingTransactionResponse(
|
||||
id=p.id,
|
||||
|
||||
@@ -8,12 +8,13 @@ from pydantic import BaseModel
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.api.client_ip import client_ip
|
||||
from app.api.timeutil import isoformat_utc
|
||||
from app.auth.dependencies import get_optional_user
|
||||
from app.db.models import RoundParticipant, User
|
||||
from app.db.session import get_session
|
||||
from app.rounds.config import get_round_config
|
||||
from app.rounds.events import RoundEventCapacityError, broadcaster
|
||||
from app.rounds.events import EVICTED, RoundEventCapacityError, broadcaster
|
||||
from app.rounds.service import get_active_round
|
||||
|
||||
router = APIRouter(prefix="/rounds", tags=["rounds"])
|
||||
@@ -46,9 +47,14 @@ async def round_stream(request: Request) -> Response:
|
||||
That's also what happens past MAX_SUBSCRIBERS (app/rounds/events.py): this
|
||||
returns 503 rather than opening a stream, and the browser's EventSource
|
||||
just retries later while the frontend keeps working off polling meanwhile.
|
||||
|
||||
Concurrent streams are additionally capped per client IP (B-38): past
|
||||
MAX_SUBSCRIBERS_PER_IP, opening one more evicts that IP's own oldest
|
||||
connection rather than refusing the new one or letting a single source
|
||||
exhaust the global cap and degrade every other user.
|
||||
"""
|
||||
try:
|
||||
queue = broadcaster.subscribe()
|
||||
queue = broadcaster.subscribe(client_ip(request))
|
||||
except RoundEventCapacityError:
|
||||
return JSONResponse(status_code=503, content={"detail": "too many concurrent update streams"})
|
||||
|
||||
@@ -59,7 +65,9 @@ async def round_stream(request: Request) -> Response:
|
||||
if await request.is_disconnected():
|
||||
break
|
||||
try:
|
||||
await asyncio.wait_for(queue.get(), timeout=_SSE_DISCONNECT_CHECK_SECONDS)
|
||||
item = await asyncio.wait_for(queue.get(), timeout=_SSE_DISCONNECT_CHECK_SECONDS)
|
||||
if item is EVICTED:
|
||||
break # this IP opened another stream past its per-IP cap
|
||||
yield "event: update\ndata: {}\n\n".format(json.dumps({}))
|
||||
ticks_since_keepalive = 0
|
||||
except asyncio.TimeoutError:
|
||||
|
||||
+1
-10
@@ -4,6 +4,7 @@ from sqlalchemy import func, select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.api.client_ip import client_ip as _client_ip
|
||||
from app.api.errors import http_error
|
||||
from app.auth.rate_limit import AuthRateLimiters
|
||||
from app.auth.security import MIN_PASSWORD_LENGTH, create_access_token, hash_password, verify_password
|
||||
@@ -34,16 +35,6 @@ def _rate_limiters(request: Request) -> AuthRateLimiters:
|
||||
return request.app.state.auth_rate_limiters
|
||||
|
||||
|
||||
def _client_ip(request: Request) -> str:
|
||||
# Caddy (see Caddyfile) reverse-proxies every request, so request.client.host
|
||||
# is the proxy's address, not the caller's — fall back to it only if the
|
||||
# header is somehow missing (e.g. hitting the app container directly).
|
||||
forwarded = request.headers.get("x-forwarded-for")
|
||||
if forwarded:
|
||||
return forwarded.split(",")[0].strip()
|
||||
return request.client.host if request.client else "unknown"
|
||||
|
||||
|
||||
def _rate_limited_error(retry_after: float):
|
||||
return http_error(
|
||||
status.HTTP_429_TOO_MANY_REQUESTS,
|
||||
|
||||
@@ -29,6 +29,11 @@ class Settings(BaseSettings):
|
||||
jwt_expire_minutes: int = 60 * 24
|
||||
admin_token: str = ""
|
||||
|
||||
# Swagger/ReDoc/OpenAPI JSON expose the entire API surface (admin endpoints
|
||||
# included) to anyone who requests them. Off by default (B-42) — set to true
|
||||
# only for local development, never in production.
|
||||
enable_api_docs: bool = False
|
||||
|
||||
# Every business/round parameter (bet amount, round duration/cooldown,
|
||||
# min amount, fee rate, RBF timeout, fee address) lives in the round_config
|
||||
# DB table instead (app/db/models.py RoundConfig, app/rounds/config.py) —
|
||||
|
||||
+36
-1
@@ -1,9 +1,44 @@
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||
from sqlalchemy import event
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine, async_sessionmaker, create_async_engine
|
||||
from sqlalchemy.orm import DeclarativeBase
|
||||
|
||||
from app.config import settings
|
||||
|
||||
# How long a writer waits for a lock held by another writer before SQLite raises
|
||||
# "database is locked" (B-39). A few seconds is enough to ride out this app's own
|
||||
# five concurrent background tasks (scheduler, confirmation poller, RBF bumper,
|
||||
# two reconcilers) plus HTTP handlers briefly overlapping a write.
|
||||
_SQLITE_BUSY_TIMEOUT_MS = 5000
|
||||
|
||||
|
||||
def _register_sqlite_pragmas(target_engine: AsyncEngine) -> None:
|
||||
"""Without WAL, SQLite's default (rollback-journal) mode lets a writer block
|
||||
every reader for the duration of its transaction, and a second writer arriving
|
||||
while one is already active fails immediately rather than waiting at all —
|
||||
realistic under this app's concurrency, and nothing previously handled it.
|
||||
WAL lets readers and writers proceed without blocking each other, and
|
||||
busy_timeout gives a second writer a real window to wait for the first
|
||||
instead of an instant `OperationalError`.
|
||||
|
||||
No-op for any dialect other than sqlite (e.g. a future PostgreSQL
|
||||
DATABASE_URL), which neither needs nor understands these pragmas.
|
||||
"""
|
||||
if target_engine.dialect.name != "sqlite":
|
||||
return
|
||||
|
||||
@event.listens_for(target_engine.sync_engine, "connect")
|
||||
def _set_sqlite_pragmas(dbapi_connection, connection_record) -> None:
|
||||
cursor = dbapi_connection.cursor()
|
||||
try:
|
||||
cursor.execute("PRAGMA journal_mode=WAL")
|
||||
cursor.execute("PRAGMA synchronous=NORMAL")
|
||||
cursor.execute(f"PRAGMA busy_timeout={_SQLITE_BUSY_TIMEOUT_MS}")
|
||||
finally:
|
||||
cursor.close()
|
||||
|
||||
|
||||
engine = create_async_engine(settings.database_url)
|
||||
_register_sqlite_pragmas(engine)
|
||||
AsyncSessionLocal = async_sessionmaker(engine, expire_on_commit=False)
|
||||
|
||||
|
||||
|
||||
+3
-3
@@ -1,6 +1,6 @@
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import BigInteger, ForeignKey, Index, String, UniqueConstraint, text
|
||||
from sqlalchemy import BigInteger, ForeignKey, Index, String, Text, UniqueConstraint, text
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.db.base import Base
|
||||
@@ -154,7 +154,7 @@ class PendingTransaction(Base):
|
||||
user_id: Mapped[int | None] = mapped_column(ForeignKey("users.id"), default=None)
|
||||
current_txid: Mapped[str] = mapped_column(String(64))
|
||||
fee_rate_sat_vb: Mapped[int]
|
||||
raw_tx_hex: Mapped[str] = mapped_column(String)
|
||||
raw_tx_hex: Mapped[str] = mapped_column(Text)
|
||||
# The *first* broadcast — never rewritten by a bump — since this is what the
|
||||
# reconciler's abandon-after-N-hours grace period (app/tx/reconcile.py) measures
|
||||
# from. Bumping used to overwrite this field, which reset that clock on every
|
||||
@@ -193,7 +193,7 @@ class AuditLog(Base):
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
event_type: Mapped[str] = mapped_column(String(32))
|
||||
payload_json: Mapped[str] = mapped_column(String)
|
||||
payload_json: Mapped[str] = mapped_column(Text)
|
||||
user_id: Mapped[int | None] = mapped_column(ForeignKey("users.id"), default=None)
|
||||
round_id: Mapped[int | None] = mapped_column(ForeignKey("rounds.id"), default=None)
|
||||
created_at: Mapped[datetime] = mapped_column(default=utcnow)
|
||||
@@ -169,6 +169,17 @@ class ElectrumClient:
|
||||
async def listunspent(self, scripthash: str) -> list[dict]:
|
||||
return await self.request("blockchain.scripthash.listunspent", [scripthash])
|
||||
|
||||
async def get_history(self, scripthash: str) -> list[dict]:
|
||||
"""Every transaction touching `scripthash`, each as {"tx_hash", "height"} —
|
||||
height > 0 means confirmed at that height, height <= 0 means still in the
|
||||
mempool. Used instead of blockchain.transaction.get's verbose=True mode
|
||||
for confirmation/existence checks (B-41): several Electrum server
|
||||
implementations and versions reject the verbose flag outright ("verbose
|
||||
transactions are currently unsupported"), while get_history is a plain,
|
||||
universally-supported method every server must implement.
|
||||
"""
|
||||
return await self.request("blockchain.scripthash.get_history", [scripthash])
|
||||
|
||||
async def broadcast(self, raw_tx_hex: str) -> str:
|
||||
return await self.request("blockchain.transaction.broadcast", [raw_tx_hex])
|
||||
|
||||
|
||||
+10
-1
@@ -84,7 +84,16 @@ async def lifespan(app: FastAPI):
|
||||
await listener.client.close()
|
||||
|
||||
|
||||
app = FastAPI(title="PLM Lottery", lifespan=lifespan)
|
||||
# Swagger/ReDoc/the raw OpenAPI JSON enumerate the entire API surface, admin
|
||||
# endpoints included, to anyone who requests them (B-42) — disabled unless
|
||||
# ENABLE_API_DOCS is explicitly set, which should only happen in development.
|
||||
app = FastAPI(
|
||||
title="PLM Lottery",
|
||||
lifespan=lifespan,
|
||||
docs_url="/docs" if settings.enable_api_docs else None,
|
||||
redoc_url="/redoc" if settings.enable_api_docs else None,
|
||||
openapi_url="/openapi.json" if settings.enable_api_docs else None,
|
||||
)
|
||||
app.include_router(auth_router)
|
||||
app.include_router(users_router)
|
||||
app.include_router(bets_router)
|
||||
|
||||
+54
-14
@@ -1,16 +1,31 @@
|
||||
import asyncio
|
||||
from collections import defaultdict
|
||||
|
||||
# Defensive cap on concurrent SSE subscribers. Expected load is on the order of
|
||||
# ~100 concurrent users; this is set well above that so it never engages under
|
||||
# normal use — it exists purely so a runaway/DoS-y number of open connections
|
||||
# degrades (new connections fall back to polling, see round_stream()) instead
|
||||
# of growing the in-memory subscriber set without bound. Revisit this number if
|
||||
# expected concurrency grows well past it.
|
||||
# Defensive backstop on concurrent SSE subscribers overall, regardless of source
|
||||
# — expected load is on the order of ~100 concurrent users, so this is set well
|
||||
# above that. The real defense against a single abusive source is the per-IP cap
|
||||
# below (B-38): a global-only cap was trivially exhausted by one client opening
|
||||
# MAX_SUBSCRIBERS connections, degrading every other user to polling — the
|
||||
# comment used to call it "defensive"; it was actually the vector.
|
||||
MAX_SUBSCRIBERS = 500
|
||||
|
||||
# How many concurrent streams a single client IP may hold. Deliberately small —
|
||||
# a real browser tab needs at most one, occasionally two briefly across a
|
||||
# reload — since this bounds one source's share of the global capacity, not a
|
||||
# legitimate per-user concurrency limit.
|
||||
MAX_SUBSCRIBERS_PER_IP = 5
|
||||
|
||||
# Put on a to-be-evicted subscriber's queue (B-38) to wake its generator
|
||||
# (app/api/routes/rounds.py:round_stream) promptly so it closes the connection
|
||||
# instead of lingering, silently uncounted, until the client's own network
|
||||
# timeout or the next keep-alive tick.
|
||||
EVICTED = object()
|
||||
|
||||
|
||||
class RoundEventCapacityError(Exception):
|
||||
"""Raised by subscribe() when MAX_SUBSCRIBERS is already reached."""
|
||||
"""Raised by subscribe() when MAX_SUBSCRIBERS — the global backstop — is
|
||||
already reached. The per-IP cap never raises this; it evicts instead (see
|
||||
subscribe())."""
|
||||
|
||||
|
||||
class RoundEventBroadcaster:
|
||||
@@ -26,22 +41,47 @@ class RoundEventBroadcaster:
|
||||
would need a shared channel (e.g. Redis pub/sub) instead.
|
||||
"""
|
||||
|
||||
def __init__(self, max_subscribers: int = MAX_SUBSCRIBERS):
|
||||
self._subscribers: set[asyncio.Queue] = set()
|
||||
def __init__(self, max_subscribers: int = MAX_SUBSCRIBERS, max_per_ip: int = MAX_SUBSCRIBERS_PER_IP):
|
||||
self._ip_by_queue: dict[asyncio.Queue, str] = {}
|
||||
self._queues_by_ip: dict[str, list[asyncio.Queue]] = defaultdict(list)
|
||||
self.max_subscribers = max_subscribers
|
||||
self.max_per_ip = max_per_ip
|
||||
|
||||
def subscribe(self) -> asyncio.Queue:
|
||||
if len(self._subscribers) >= self.max_subscribers:
|
||||
def subscribe(self, client_ip: str = "unknown") -> asyncio.Queue:
|
||||
if len(self._ip_by_queue) >= self.max_subscribers:
|
||||
raise RoundEventCapacityError(f"already at the {self.max_subscribers}-subscriber cap")
|
||||
|
||||
ip_queues = self._queues_by_ip[client_ip]
|
||||
if len(ip_queues) >= self.max_per_ip:
|
||||
# B-38: evict this IP's own oldest connection rather than refusing
|
||||
# the new one — bounds one source's footprint without turning a
|
||||
# legitimate reconnect storm (a flaky network retrying EventSource)
|
||||
# into an outright block, and without letting one abusive IP crowd
|
||||
# out unrelated clients the way the old global-only cap did.
|
||||
oldest = ip_queues.pop(0)
|
||||
self._ip_by_queue.pop(oldest, None)
|
||||
if not oldest.full():
|
||||
oldest.put_nowait(EVICTED)
|
||||
|
||||
queue: asyncio.Queue = asyncio.Queue(maxsize=1)
|
||||
self._subscribers.add(queue)
|
||||
self._ip_by_queue[queue] = client_ip
|
||||
ip_queues.append(queue)
|
||||
return queue
|
||||
|
||||
def unsubscribe(self, queue: asyncio.Queue) -> None:
|
||||
self._subscribers.discard(queue)
|
||||
client_ip = self._ip_by_queue.pop(queue, None)
|
||||
if client_ip is None:
|
||||
return
|
||||
ip_queues = self._queues_by_ip.get(client_ip)
|
||||
if ip_queues is None:
|
||||
return
|
||||
if queue in ip_queues:
|
||||
ip_queues.remove(queue)
|
||||
if not ip_queues:
|
||||
self._queues_by_ip.pop(client_ip, None)
|
||||
|
||||
def publish(self) -> None:
|
||||
for queue in self._subscribers:
|
||||
for queue in self._ip_by_queue:
|
||||
if queue.full():
|
||||
continue # a not-yet-delivered notification already covers this one
|
||||
queue.put_nowait(None)
|
||||
|
||||
+54
-26
@@ -48,14 +48,14 @@ def should_bump(pending: PendingTransaction, now: datetime, timeout_seconds: int
|
||||
return now >= pending.last_broadcast_at.replace(tzinfo=timezone.utc) + timedelta(seconds=timeout_seconds)
|
||||
|
||||
|
||||
async def _signing_context(session: AsyncSession, pending: PendingTransaction) -> tuple:
|
||||
async def _signing_context(session: AsyncSession, kind: str, user_id: int | None) -> tuple:
|
||||
"""Returns (signing_key, own_script, own_address) for the single sender that
|
||||
controls every input of this tx — a user for bet/withdrawal, the pool for
|
||||
payout. All our builders only ever spend one address's UTXOs per tx."""
|
||||
if pending.kind == "payout":
|
||||
if kind == "payout":
|
||||
key = derive_pool_key()
|
||||
else:
|
||||
user = await session.get(User, pending.user_id)
|
||||
user = await session.get(User, user_id)
|
||||
key = derive_user_key(user.derivation_index)
|
||||
own_script = script.p2wpkh(key.to_public())
|
||||
own_address = own_script.address(network=PLM_MAINNET)
|
||||
@@ -63,10 +63,19 @@ async def _signing_context(session: AsyncSession, pending: PendingTransaction) -
|
||||
|
||||
|
||||
async def _prevout_amount(client: ElectrumClient, vin: TransactionInput) -> int:
|
||||
"""The exact integer satoshi value of the output this input spends.
|
||||
|
||||
Parsed directly from the raw transaction via embit rather than asking the
|
||||
server for its own float, whole-coin-denominated "value" field (verbose=True)
|
||||
and converting with `* 100_000_000` — embit's TransactionOutput.value is
|
||||
already an integer number of satoshis straight from the tx's binary
|
||||
encoding, so this never touches floating point in a codebase that is
|
||||
otherwise strictly integer-satoshi (B-40).
|
||||
"""
|
||||
txid_hex = vin.txid.hex()
|
||||
tx = await client.get_transaction(txid_hex, verbose=True)
|
||||
value_coins = tx["vout"][vin.vout]["value"]
|
||||
return round(value_coins * 100_000_000)
|
||||
raw_hex = await client.get_transaction(txid_hex, verbose=False)
|
||||
prevout_tx = Transaction.parse(bytes.fromhex(raw_hex))
|
||||
return prevout_tx.vout[vin.vout].value
|
||||
|
||||
|
||||
def _find_change_output(tx: Transaction, change_address: str) -> int | None:
|
||||
@@ -76,35 +85,55 @@ def _find_change_output(tx: Transaction, change_address: str) -> int | None:
|
||||
return None
|
||||
|
||||
|
||||
async def bump_fee(session: AsyncSession, client: ElectrumClient, pending: PendingTransaction) -> str:
|
||||
"""Rebuild `pending`'s transaction with a higher fee (same inputs, same
|
||||
recipient outputs, the extra fee taken from the change output) and
|
||||
rebroadcast. Returns the new txid.
|
||||
async def bump_fee(
|
||||
session_factory: async_sessionmaker, client: ElectrumClient, pending_id: int
|
||||
) -> str | None:
|
||||
"""Rebuild pending_transaction `pending_id`'s transaction with a higher fee
|
||||
(same inputs, same recipient outputs, the extra fee taken from the change
|
||||
output) and rebroadcast. Returns the new txid, or None if there was nothing
|
||||
to do (the row is gone or already left "pending" — a normal race with
|
||||
confirmation, not an error).
|
||||
|
||||
Three phases, so no DB session is held across the network calls this needs
|
||||
(one get_transaction per input, then a broadcast) — the same shape used
|
||||
elsewhere for exactly this reason (B-18, rounds/scheduler.py:_trigger_payout;
|
||||
B-31, electrum/listener.py:refresh_user) and now here too (B-40): read what's
|
||||
needed and close the session, do the chain work, then reopen to persist.
|
||||
|
||||
Only handles the common case: exactly one change output paying back to the
|
||||
tx's own sender address, large enough to absorb the increase. If there's no
|
||||
such output (e.g. an exact-amount bet with no change), this raises RbfError —
|
||||
bumping such a tx would require selecting additional inputs, which isn't
|
||||
implemented for the MVP; it needs manual operator intervention. Also raises
|
||||
RbfError, rather than bumping, once `pending` is already at MAX_FEE_RATE_SAT_VB
|
||||
RbfError, rather than bumping, once the row is already at MAX_FEE_RATE_SAT_VB
|
||||
(B-32) — the reconciler abandons it if it never confirms (B-27), instead of
|
||||
this retrying an ever-higher fee forever.
|
||||
"""
|
||||
# --- Phase 1: read what's needed, close the session before any network call ---
|
||||
async with session_factory() as session:
|
||||
pending = await session.get(PendingTransaction, pending_id)
|
||||
if pending is None or pending.status != "pending":
|
||||
logger.info("pending_transaction %s no longer pending; skipping bump", pending_id)
|
||||
return None
|
||||
if pending.fee_rate_sat_vb >= MAX_FEE_RATE_SAT_VB:
|
||||
raise RbfError(
|
||||
f"pending_transaction {pending.id}: already at the maximum fee rate "
|
||||
f"pending_transaction {pending_id}: already at the maximum fee rate "
|
||||
f"({MAX_FEE_RATE_SAT_VB} sat/vB) — refusing to bump further"
|
||||
)
|
||||
|
||||
old_tx = Transaction.parse(bytes.fromhex(pending.raw_tx_hex))
|
||||
signing_key, own_script, own_address = await _signing_context(session, pending)
|
||||
kind = pending.kind
|
||||
current_fee_rate = pending.fee_rate_sat_vb
|
||||
raw_tx_hex = pending.raw_tx_hex
|
||||
signing_key, own_script, own_address = await _signing_context(session, kind, pending.user_id)
|
||||
|
||||
# --- Phase 2: chain reads, signing, and the broadcast — no DB session open ----
|
||||
old_tx = Transaction.parse(bytes.fromhex(raw_tx_hex))
|
||||
input_amounts = [await _prevout_amount(client, vin) for vin in old_tx.vin]
|
||||
total_in = sum(input_amounts)
|
||||
old_fee = total_in - sum(o.value for o in old_tx.vout)
|
||||
vsize = estimate_vsize(len(old_tx.vin), len(old_tx.vout))
|
||||
|
||||
target_fee_rate = min(pending.fee_rate_sat_vb + _FEE_RATE_INCREMENT, MAX_FEE_RATE_SAT_VB)
|
||||
target_fee_rate = min(current_fee_rate + _FEE_RATE_INCREMENT, MAX_FEE_RATE_SAT_VB)
|
||||
target_fee = vsize * target_fee_rate
|
||||
# BIP125 rule 4's minimum, in absolute sats for this tx's size — the floor
|
||||
# `fee_delta` must never go below, no matter what `target_fee - old_fee` comes
|
||||
@@ -120,7 +149,7 @@ async def bump_fee(session: AsyncSession, client: ElectrumClient, pending: Pendi
|
||||
|
||||
change_index = _find_change_output(old_tx, own_address)
|
||||
if change_index is None or old_tx.vout[change_index].value <= fee_delta:
|
||||
raise RbfError(f"pending_transaction {pending.id}: no change output large enough to absorb a fee bump")
|
||||
raise RbfError(f"pending_transaction {pending_id}: no change output large enough to absorb a fee bump")
|
||||
|
||||
new_vout = list(old_tx.vout)
|
||||
bumped_change = new_vout[change_index].value - fee_delta
|
||||
@@ -144,6 +173,9 @@ async def bump_fee(session: AsyncSession, client: ElectrumClient, pending: Pendi
|
||||
new_txid = final_tx.txid().hex()
|
||||
await client.broadcast(raw_hex)
|
||||
|
||||
# --- Phase 3: persist the outcome ----------------------------------------------
|
||||
async with session_factory() as session:
|
||||
pending = await session.get(PendingTransaction, pending_id)
|
||||
old_txid = pending.current_txid
|
||||
pending.replaced_by_txid = old_txid # points backwards: what current_txid replaced
|
||||
pending.current_txid = new_txid
|
||||
@@ -162,7 +194,7 @@ async def bump_fee(session: AsyncSession, client: ElectrumClient, pending: Pendi
|
||||
await _retarget_txid_references(session, pending, old_txid, new_txid)
|
||||
await session.commit()
|
||||
|
||||
logger.info("bumped %s pending_transaction %s: %s -> %s", pending.kind, pending.id, old_txid, new_txid)
|
||||
logger.info("bumped %s pending_transaction %s: %s -> %s", kind, pending_id, old_txid, new_txid)
|
||||
return new_txid
|
||||
|
||||
|
||||
@@ -230,16 +262,12 @@ class RbfBumper:
|
||||
candidates = (
|
||||
await session.scalars(select(PendingTransaction).where(PendingTransaction.status == "pending"))
|
||||
).all()
|
||||
due = [p for p in candidates if should_bump(p, now, timeout_seconds=timeout_seconds)]
|
||||
due_ids = [p.id for p in candidates if should_bump(p, now, timeout_seconds=timeout_seconds)]
|
||||
|
||||
for pending in due:
|
||||
async with self._session_factory() as session:
|
||||
row = await session.get(PendingTransaction, pending.id)
|
||||
if row is None or row.status != "pending":
|
||||
continue
|
||||
for pending_id in due_ids:
|
||||
try:
|
||||
await bump_fee(session, client, row)
|
||||
await bump_fee(self._session_factory, client, pending_id)
|
||||
except RbfError:
|
||||
logger.exception("could not bump pending_transaction %s", row.id)
|
||||
logger.exception("could not bump pending_transaction %s", pending_id)
|
||||
except Exception:
|
||||
logger.exception("unexpected error bumping pending_transaction %s", row.id)
|
||||
logger.exception("unexpected error bumping pending_transaction %s", pending_id)
|
||||
+40
-10
@@ -7,7 +7,9 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
from app.db.models import PendingTransaction
|
||||
from app.electrum.client import ElectrumClient
|
||||
from app.electrum.scripthash import address_to_scripthash
|
||||
from app.rounds.events import broadcaster
|
||||
from app.tx.pending_address import own_address_for
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -31,24 +33,52 @@ async def poll_once(session_factory: async_sessionmaker, client: ElectrumClient)
|
||||
candidates = (
|
||||
await session.execute(
|
||||
select(
|
||||
PendingTransaction.id, PendingTransaction.current_txid, PendingTransaction.kind
|
||||
PendingTransaction.id,
|
||||
PendingTransaction.current_txid,
|
||||
PendingTransaction.kind,
|
||||
PendingTransaction.user_id,
|
||||
).where(PendingTransaction.status == "pending")
|
||||
)
|
||||
).all()
|
||||
|
||||
confirmed = 0
|
||||
for pending_id, txid, kind in candidates:
|
||||
# Resolved once per candidate while the session is still open, and cached
|
||||
# by scripthash below — every "payout" row shares the same pool address,
|
||||
# so this also avoids asking the server the same history twice per tick.
|
||||
scripthash_by_id: dict[int, str] = {}
|
||||
for pending_id, _txid, kind, user_id in candidates:
|
||||
try:
|
||||
tx = await client.get_transaction(txid, verbose=True)
|
||||
address = await own_address_for(session, kind, user_id)
|
||||
scripthash_by_id[pending_id] = address_to_scripthash(address)
|
||||
except Exception:
|
||||
# One unresolvable txid must not stop the others: a tx the server no
|
||||
# longer knows (dropped from the mempool, replaced) used to abort the
|
||||
# whole pass, so nothing confirmed again until an operator intervened
|
||||
# (B-03). Abandoning such a row is app/tx/reconcile.py's job, not ours.
|
||||
logger.warning("could not check pending_transaction %s (txid %s)", pending_id, txid, exc_info=True)
|
||||
logger.exception("could not derive the address for pending_transaction %s", pending_id)
|
||||
|
||||
confirmed = 0
|
||||
history_cache: dict[str, list[dict]] = {}
|
||||
for pending_id, txid, kind, _user_id in candidates:
|
||||
scripthash = scripthash_by_id.get(pending_id)
|
||||
if scripthash is None:
|
||||
continue # address derivation failed above; already logged
|
||||
|
||||
try:
|
||||
if scripthash not in history_cache:
|
||||
history_cache[scripthash] = await client.get_history(scripthash)
|
||||
except Exception:
|
||||
# One unresolvable scripthash must not stop the others: a tx the server
|
||||
# no longer knows about (dropped from the mempool, replaced) used to
|
||||
# abort the whole pass via a verbose blockchain.transaction.get call
|
||||
# that some servers reject outright (B-41), so nothing confirmed again
|
||||
# until an operator intervened (B-03). Abandoning such a row is
|
||||
# app/tx/reconcile.py's job, not ours.
|
||||
logger.warning("could not fetch history for pending_transaction %s (txid %s)", pending_id, txid, exc_info=True)
|
||||
continue
|
||||
if not tx or tx.get("confirmations", 0) < 1:
|
||||
|
||||
entry = next((e for e in history_cache[scripthash] if e.get("tx_hash") == txid), None)
|
||||
# height > 0 means confirmed at that height; 0 or absent means still in
|
||||
# the mempool (or the server doesn't know this txid at all yet) — either
|
||||
# way, not confirmed, so keep waiting.
|
||||
if entry is None or entry.get("height", 0) <= 0:
|
||||
continue
|
||||
|
||||
async with session_factory() as session:
|
||||
row = await session.get(PendingTransaction, pending_id)
|
||||
if row is None or row.status != "pending":
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.db.models import User
|
||||
from app.wallet.hd import derive_pool_address, derive_user_address
|
||||
|
||||
|
||||
async def own_address_for(session: AsyncSession, kind: str, user_id: int | None) -> str:
|
||||
"""The address that owns every input of a PendingTransaction of this kind —
|
||||
a user's own address for a bet/withdrawal, the pool address for a payout.
|
||||
All our builders only ever spend one address's UTXOs per tx (see
|
||||
tx/broadcast.py:_signing_context, which derives the same address alongside
|
||||
the signing key it also needs).
|
||||
|
||||
Shared by tx/confirmation.py and tx/reconcile.py (B-41): both now check
|
||||
blockchain.scripthash.get_history for this address instead of asking
|
||||
blockchain.transaction.get for a verbose reply, so the two can't derive
|
||||
different addresses for the same row.
|
||||
"""
|
||||
if kind == "payout":
|
||||
return derive_pool_address()
|
||||
user = await session.get(User, user_id)
|
||||
return derive_user_address(user.derivation_index)
|
||||
+32
-22
@@ -36,7 +36,9 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
from app.audit.log import write_audit_log
|
||||
from app.db.models import PendingTransaction, Round, RoundParticipant, UtxoEvent, Withdrawal
|
||||
from app.electrum.client import ElectrumClient
|
||||
from app.electrum.scripthash import address_to_scripthash
|
||||
from app.rounds.events import broadcaster
|
||||
from app.tx.pending_address import own_address_for
|
||||
from app.wallet.balance import recompute_balance
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -72,22 +74,21 @@ class PendingTransactionReconciler:
|
||||
await asyncio.sleep(_POLL_INTERVAL_SECONDS)
|
||||
|
||||
|
||||
async def _tx_exists_on_chain(client: ElectrumClient, txid: str) -> bool:
|
||||
"""True if the server knows this txid at all (mempool or mined). An error reply
|
||||
means "unknown", which is the answer we're looking for; a transport failure is
|
||||
*not* — that raises, and the caller leaves the row alone until next time."""
|
||||
try:
|
||||
tx = await client.get_transaction(txid, verbose=True)
|
||||
except Exception as exc:
|
||||
message = str(exc).lower()
|
||||
if "missing" in message or "not found" in message or "no such" in message or "unknown" in message:
|
||||
return False
|
||||
raise
|
||||
return bool(tx)
|
||||
|
||||
|
||||
async def reconcile_once(session_factory: async_sessionmaker, client: ElectrumClient) -> int:
|
||||
"""Returns how many rows were resolved (promoted or abandoned)."""
|
||||
"""Returns how many rows were resolved (promoted or abandoned).
|
||||
|
||||
Existence is decided by checking whether a row's own address's history
|
||||
(blockchain.scripthash.get_history) includes its txid at all — mempool or
|
||||
mined — rather than asking blockchain.transaction.get for a verbose reply
|
||||
(B-41): several Electrum server implementations and versions reject the
|
||||
verbose flag outright, and the previous substring-matching on the error
|
||||
text (looking for "missing", "not found", ...) was fragile as the basis for
|
||||
a decision that releases funds. A transport failure fetching history still
|
||||
raises and leaves the row alone until next time — get_history not
|
||||
returning our txid is the only thing that means "gone". History is cached
|
||||
per scripthash within one pass, since every "payout" row shares the same
|
||||
pool address.
|
||||
"""
|
||||
now = datetime.now(timezone.utc)
|
||||
async with session_factory() as session:
|
||||
rows = (
|
||||
@@ -95,16 +96,25 @@ async def reconcile_once(session_factory: async_sessionmaker, client: ElectrumCl
|
||||
select(PendingTransaction).where(PendingTransaction.status.in_(("building", "pending")))
|
||||
)
|
||||
).all()
|
||||
candidates = [
|
||||
(row.id, row.status, row.current_txid)
|
||||
for row in rows
|
||||
if _is_due(row, now)
|
||||
]
|
||||
candidates = []
|
||||
for row in rows:
|
||||
if not _is_due(row, now):
|
||||
continue
|
||||
try:
|
||||
address = await own_address_for(session, row.kind, row.user_id)
|
||||
scripthash = address_to_scripthash(address)
|
||||
except Exception:
|
||||
logger.exception("could not derive the address for pending_transaction %s", row.id)
|
||||
continue
|
||||
candidates.append((row.id, row.status, row.current_txid, scripthash))
|
||||
|
||||
resolved = 0
|
||||
for row_id, status, txid in candidates:
|
||||
history_cache: dict[str, list[dict]] = {}
|
||||
for row_id, status, txid, scripthash in candidates:
|
||||
try:
|
||||
exists = await _tx_exists_on_chain(client, txid)
|
||||
if scripthash not in history_cache:
|
||||
history_cache[scripthash] = await client.get_history(scripthash)
|
||||
exists = any(entry.get("tx_hash") == txid for entry in history_cache[scripthash])
|
||||
except Exception:
|
||||
# Transport/server problem — say nothing about this tx and try again on
|
||||
# the next pass rather than abandoning a tx that may be perfectly alive.
|
||||
|
||||
+3
-2
@@ -121,8 +121,9 @@ Eventi a cui vale la pena prestare attenzione:
|
||||
## Alternative all'interfaccia grafica
|
||||
|
||||
Le stesse operazioni si possono fare da terminale o da Swagger UI
|
||||
(`https://<host>/docs`, sezione `admin`), sempre passando `ADMIN_TOKEN`
|
||||
nell'header `X-Admin-Token`:
|
||||
(`https://<host>/docs`, sezione `admin` — disponibile solo se `ENABLE_API_DOCS=true`
|
||||
è impostato in `.env`, disattivata di default perché espone l'intera API),
|
||||
sempre passando `ADMIN_TOKEN` nell'header `X-Admin-Token`:
|
||||
|
||||
```bash
|
||||
# leggere la configurazione
|
||||
|
||||
@@ -3,24 +3,14 @@
|
||||
Presuppone che [setup.md](setup.md) sia già stato completato (`.env` pronto,
|
||||
master key generata, migrazioni applicate).
|
||||
|
||||
## Locale / venv (sviluppo rapido)
|
||||
Il server gira sempre via Docker, in sviluppo e in produzione allo stesso
|
||||
modo — non esiste un modo supportato per lanciare `uvicorn` direttamente.
|
||||
Il venv locale (`.venv/`) serve solo per i test, per scrivere le migrazioni
|
||||
Alembic e per gli script una tantum di generazione chiavi (vedi
|
||||
[setup.md](setup.md) e la sezione "Commands" di
|
||||
[CLAUDE.md](../CLAUDE.md#commands)).
|
||||
|
||||
```bash
|
||||
source .venv/bin/activate
|
||||
uvicorn app.main:app --reload --port 8123
|
||||
```
|
||||
|
||||
- App su `http://127.0.0.1:8123/`
|
||||
- Pannello admin su `http://127.0.0.1:8123/admin`
|
||||
- Log applicativi in `logs/app.log` (rotante, 10MB × 5 backup)
|
||||
- Nessun TLS, nessun reverse proxy — solo per test locali sulla tua macchina.
|
||||
|
||||
Per fermarlo: `Ctrl+C`, oppure se lanciato in background con `nohup`:
|
||||
```bash
|
||||
pkill -f "uvicorn app.main:app"
|
||||
```
|
||||
|
||||
## Docker + Caddy (consigliato, anche per i test con dominio/TLS)
|
||||
## Docker + Caddy (unico workflow supportato)
|
||||
|
||||
```bash
|
||||
mkdir -p data/db data/keys data/logs # una tantum, se non già presenti
|
||||
|
||||
@@ -38,6 +38,11 @@ cp .env.example .env
|
||||
Le altre chiavi di `.env` (`DATABASE_URL`, `ELECTRUM_HOST`/`PORT`/`USE_SSL`,
|
||||
`MASTER_KEY_PATH`) hanno default sensati in `.env.example`.
|
||||
|
||||
`ENABLE_API_DOCS` (default `false`) controlla Swagger/ReDoc/l'OpenAPI JSON grezzo
|
||||
su `/docs`, `/redoc` e `/openapi.json`: espongono l'intera superficie dell'API,
|
||||
endpoint admin inclusi, quindi restano disattivati a meno di non impostarlo
|
||||
esplicitamente a `true` — utile in locale, da evitare in produzione.
|
||||
|
||||
`ELECTRUM_FALLBACK_SERVERS` elenca i server di riserva, separati da virgola, nel
|
||||
formato `host:porta` (TLS, il caso normale) oppure `host:porta:notls`. Esempio:
|
||||
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
"""widen raw_tx_hex and payload_json to Text
|
||||
|
||||
Fixes B-47: both columns held arbitrary-length data (a raw signed transaction
|
||||
hex, and a JSON audit payload) in an unbounded `String`, which SQLAlchemy
|
||||
compiles to `VARCHAR` with no length. That's accepted by SQLite and
|
||||
PostgreSQL but rejected by other backends (e.g. MySQL requires a length on
|
||||
VARCHAR) — `Text` is the portable type for both.
|
||||
|
||||
Revision ID: 87a0c640355c
|
||||
Revises: 9ef6a51509f7
|
||||
Create Date: 2026-07-27
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = '87a0c640355c'
|
||||
down_revision: Union[str, Sequence[str], None] = '9ef6a51509f7'
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
with op.batch_alter_table('audit_log') as batch_op:
|
||||
batch_op.alter_column('payload_json', existing_type=sa.VARCHAR(), type_=sa.Text(), existing_nullable=False)
|
||||
with op.batch_alter_table('pending_transactions') as batch_op:
|
||||
batch_op.alter_column('raw_tx_hex', existing_type=sa.VARCHAR(), type_=sa.Text(), existing_nullable=False)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
with op.batch_alter_table('pending_transactions') as batch_op:
|
||||
batch_op.alter_column('raw_tx_hex', existing_type=sa.Text(), type_=sa.VARCHAR(), existing_nullable=False)
|
||||
with op.batch_alter_table('audit_log') as batch_op:
|
||||
batch_op.alter_column('payload_json', existing_type=sa.Text(), type_=sa.VARCHAR(), existing_nullable=False)
|
||||
@@ -67,6 +67,15 @@ async def test_admin_rejects_wrong_token(client):
|
||||
assert resp.status_code == 403
|
||||
|
||||
|
||||
async def test_admin_rejects_non_ascii_token_with_403_not_500(client):
|
||||
"""B-46: secrets.compare_digest raises TypeError on a non-ASCII str, which
|
||||
used to bubble up as a 500 instead of the expected 403. httpx encodes str
|
||||
header values as ASCII client-side, so the raw UTF-8 bytes are passed
|
||||
directly to reproduce what a real non-ASCII header on the wire looks like."""
|
||||
resp = await client.get("/admin/config", headers={"X-Admin-Token": "café".encode("utf-8")})
|
||||
assert resp.status_code == 403
|
||||
|
||||
|
||||
async def test_admin_reads_and_updates_config(client):
|
||||
headers = {"X-Admin-Token": "test-admin-token"}
|
||||
|
||||
@@ -277,3 +286,90 @@ async def test_pause_cannot_be_toggled_through_the_config_endpoint(client):
|
||||
resp = await client.put("/admin/config", headers=headers, json={"paused": True})
|
||||
assert resp.status_code in (200, 422) # ignored or refused, but never applied
|
||||
assert (await client.get("/admin/config", headers=headers)).json()["paused"] is False
|
||||
|
||||
|
||||
@pytest.mark.parametrize("endpoint", ["/admin/rounds", "/admin/audit-log", "/admin/pending-transactions"])
|
||||
@pytest.mark.parametrize("bad_limit", [0, -1, 501])
|
||||
async def test_admin_list_endpoints_reject_out_of_range_limit(client, endpoint, bad_limit):
|
||||
"""B-45: `limit` had no bounds — `-1` means "everything" on SQLite, so an
|
||||
unvalidated limit could dump the entire table in one response."""
|
||||
headers = {"X-Admin-Token": "test-admin-token"}
|
||||
resp = await client.get(endpoint, headers=headers, params={"limit": bad_limit})
|
||||
assert resp.status_code == 422
|
||||
|
||||
|
||||
async def test_admin_list_rounds_respects_limit(client):
|
||||
from app.db import base as db_base
|
||||
from app.db.models import Round
|
||||
|
||||
async with db_base.AsyncSessionLocal() as session:
|
||||
session.add_all([Round(status="closed") for _ in range(3)])
|
||||
await session.commit()
|
||||
|
||||
headers = {"X-Admin-Token": "test-admin-token"}
|
||||
resp = await client.get("/admin/rounds", headers=headers, params={"limit": 2})
|
||||
assert resp.status_code == 200
|
||||
assert len(resp.json()) == 2
|
||||
|
||||
|
||||
async def test_admin_audit_log_respects_limit(client):
|
||||
from app.db import base as db_base
|
||||
from app.audit.log import write_audit_log
|
||||
|
||||
async with db_base.AsyncSessionLocal() as session:
|
||||
for _ in range(3):
|
||||
await write_audit_log(session, "test_event", {})
|
||||
await session.commit()
|
||||
|
||||
headers = {"X-Admin-Token": "test-admin-token"}
|
||||
resp = await client.get("/admin/audit-log", headers=headers, params={"limit": 2})
|
||||
assert resp.status_code == 200
|
||||
assert len(resp.json()) == 2
|
||||
|
||||
|
||||
async def _make_pending_transaction(session, *, kind="bet", status="pending"):
|
||||
from app.db.models import PendingTransaction
|
||||
import secrets as _secrets
|
||||
|
||||
tx = PendingTransaction(
|
||||
kind=kind,
|
||||
current_txid=_secrets.token_hex(32),
|
||||
fee_rate_sat_vb=1,
|
||||
raw_tx_hex="00",
|
||||
status=status,
|
||||
)
|
||||
session.add(tx)
|
||||
return tx
|
||||
|
||||
|
||||
async def test_admin_pending_transactions_respects_limit(client):
|
||||
from app.db import base as db_base
|
||||
|
||||
async with db_base.AsyncSessionLocal() as session:
|
||||
for _ in range(3):
|
||||
await _make_pending_transaction(session)
|
||||
await session.commit()
|
||||
|
||||
headers = {"X-Admin-Token": "test-admin-token"}
|
||||
resp = await client.get("/admin/pending-transactions", headers=headers, params={"limit": 2})
|
||||
assert resp.status_code == 200
|
||||
assert len(resp.json()) == 2
|
||||
|
||||
|
||||
async def test_admin_pending_transactions_status_filter(client):
|
||||
from app.db import base as db_base
|
||||
|
||||
async with db_base.AsyncSessionLocal() as session:
|
||||
await _make_pending_transaction(session, status="pending")
|
||||
await _make_pending_transaction(session, status="confirmed")
|
||||
await _make_pending_transaction(session, status="failed")
|
||||
await session.commit()
|
||||
|
||||
headers = {"X-Admin-Token": "test-admin-token"}
|
||||
resp = await client.get(
|
||||
"/admin/pending-transactions", headers=headers, params={"status": "confirmed"}
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
entries = resp.json()
|
||||
assert len(entries) == 1
|
||||
assert entries[0]["status"] == "confirmed"
|
||||
+137
-21
@@ -3,7 +3,7 @@ from datetime import datetime, timedelta, timezone
|
||||
import pytest
|
||||
from embit import script
|
||||
from embit.bip32 import HDKey
|
||||
from embit.transaction import Transaction
|
||||
from embit.transaction import Transaction, TransactionInput, TransactionOutput
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||
|
||||
from app.config import settings
|
||||
@@ -58,12 +58,23 @@ def test_should_bump_measures_from_last_broadcast_not_first(monkeypatch):
|
||||
|
||||
|
||||
class FakeClient:
|
||||
"""B-40: _prevout_amount now asks for the raw (non-verbose) transaction and
|
||||
reads its output value as an integer via embit, rather than a verbose reply's
|
||||
float "value" field — so this fake must hand back a real, parseable raw tx
|
||||
whose vout[0] carries the requested amount (every test here spends vout 0 of
|
||||
its fixture UTXO)."""
|
||||
|
||||
def __init__(self, prevout_values: dict[str, int]):
|
||||
self._prevout_values = prevout_values
|
||||
self.broadcasted: list[str] = []
|
||||
|
||||
async def get_transaction(self, txid: str, verbose: bool = False) -> dict:
|
||||
return {"vout": {0: {"value": self._prevout_values[txid] / 100_000_000}}}
|
||||
async def get_transaction(self, txid: str, verbose: bool = False) -> str:
|
||||
assert verbose is False
|
||||
fake_prevout_tx = Transaction(
|
||||
vin=[TransactionInput(b"\x00" * 32, 0)],
|
||||
vout=[TransactionOutput(self._prevout_values[txid], script.Script(b"\x00\x14" + b"\x00" * 20))],
|
||||
)
|
||||
return fake_prevout_tx.serialize().hex()
|
||||
|
||||
async def broadcast(self, raw_tx_hex: str) -> str:
|
||||
self.broadcasted.append(raw_tx_hex)
|
||||
@@ -130,9 +141,7 @@ async def test_bump_fee_shrinks_change_and_rebroadcasts(session_factory):
|
||||
|
||||
client = FakeClient({utxo_txid: utxo_amount})
|
||||
|
||||
async with session_factory() as session:
|
||||
row = await session.get(PendingTransaction, pending_id)
|
||||
new_txid = await bump_fee(session, client, row)
|
||||
new_txid = await bump_fee(session_factory, client, pending_id)
|
||||
|
||||
assert client.broadcasted
|
||||
assert new_txid != built.txid
|
||||
@@ -197,9 +206,7 @@ async def test_bump_fee_leaves_broadcast_at_untouched(session_factory):
|
||||
client = FakeClient({utxo_txid: utxo_amount})
|
||||
before_bump = datetime.now(timezone.utc)
|
||||
|
||||
async with session_factory() as session:
|
||||
row = await session.get(PendingTransaction, pending_id)
|
||||
await bump_fee(session, client, row)
|
||||
await bump_fee(session_factory, client, pending_id)
|
||||
|
||||
async with session_factory() as session:
|
||||
row = await session.get(PendingTransaction, pending_id)
|
||||
@@ -246,10 +253,8 @@ async def test_bump_fee_raises_when_no_change_output(session_factory):
|
||||
|
||||
client = FakeClient({utxo_txid: utxo_amount})
|
||||
|
||||
async with session_factory() as session:
|
||||
row = await session.get(PendingTransaction, pending_id)
|
||||
with pytest.raises(RbfError):
|
||||
await bump_fee(session, client, row)
|
||||
await bump_fee(session_factory, client, pending_id)
|
||||
|
||||
|
||||
async def test_bump_fee_retargets_every_stored_txid(session_factory):
|
||||
@@ -303,9 +308,7 @@ async def test_bump_fee_retargets_every_stored_txid(session_factory):
|
||||
await session.commit()
|
||||
pending_id = pending.id
|
||||
|
||||
async with session_factory() as session:
|
||||
row = await session.get(PendingTransaction, pending_id)
|
||||
new_txid = await bump_fee(session, FakeClient({utxo_txid: utxo_amount}), row)
|
||||
new_txid = await bump_fee(session_factory, FakeClient({utxo_txid: utxo_amount}), pending_id)
|
||||
|
||||
async with session_factory() as session:
|
||||
from sqlalchemy import select
|
||||
@@ -378,9 +381,7 @@ async def test_bump_fee_meets_bip125_minimum_when_old_fee_already_exceeds_target
|
||||
inflated_excess = 50_000
|
||||
client = FakeClient({utxo_txid: utxo_amount + inflated_excess})
|
||||
|
||||
async with session_factory() as session:
|
||||
row = await session.get(PendingTransaction, pending_id)
|
||||
new_txid = await bump_fee(session, client, row)
|
||||
new_txid = await bump_fee(session_factory, client, pending_id)
|
||||
|
||||
assert client.broadcasted
|
||||
new_tx = Transaction.parse(bytes.fromhex(client.broadcasted[0]))
|
||||
@@ -442,9 +443,124 @@ async def test_bump_fee_refuses_once_at_the_max_fee_rate(session_factory):
|
||||
|
||||
client = FakeClient({utxo_txid: utxo_amount})
|
||||
|
||||
async with session_factory() as session:
|
||||
row = await session.get(PendingTransaction, pending_id)
|
||||
with pytest.raises(RbfError):
|
||||
await bump_fee(session, client, row)
|
||||
await bump_fee(session_factory, client, pending_id)
|
||||
|
||||
assert not client.broadcasted
|
||||
|
||||
|
||||
# --- B-40: bump_fee must not hold a DB session open across its network calls,
|
||||
# and a row that's no longer pending by the time it runs is a quiet no-op. -------
|
||||
|
||||
|
||||
async def test_bump_fee_holds_no_session_open_during_network_calls(session_factory):
|
||||
"""The get_transaction-per-input reads and the broadcast must happen with no
|
||||
DB session held open — the same shape used elsewhere for this reason (B-18,
|
||||
electrum/listener.py's refresh_user for B-31) — otherwise a session sits
|
||||
idle in the pool for the whole duration of what can be several slow network
|
||||
round-trips."""
|
||||
from app.wallet.hd import derive_user_address, derive_user_key
|
||||
|
||||
signer = derive_user_key(0)
|
||||
my_address = derive_user_address(0)
|
||||
from_script = script.p2wpkh(signer.to_public())
|
||||
to_address = script.p2wpkh(_key(94).to_public()).address(network=PLM_MAINNET)
|
||||
|
||||
utxo_amount = 150_000_000
|
||||
utxo_txid = "77" * 32
|
||||
built = build_signed_transaction(
|
||||
signing_key=signer,
|
||||
from_script=from_script,
|
||||
utxos=[Utxo(utxo_txid, 0, utxo_amount)],
|
||||
to_address=to_address,
|
||||
amount_sats=10_000_000,
|
||||
change_address=my_address,
|
||||
fee_rate_sat_vb=1,
|
||||
)
|
||||
|
||||
async with session_factory() as session:
|
||||
user = User(username="frank", password_hash="x", derivation_index=0, address=my_address)
|
||||
session.add(user)
|
||||
await session.commit()
|
||||
pending = PendingTransaction(
|
||||
kind="bet",
|
||||
user_id=user.id,
|
||||
current_txid=built.txid,
|
||||
fee_rate_sat_vb=1,
|
||||
raw_tx_hex=built.raw_hex,
|
||||
status="pending",
|
||||
broadcast_at=datetime.now(timezone.utc) - timedelta(seconds=1000),
|
||||
)
|
||||
session.add(pending)
|
||||
await session.commit()
|
||||
pending_id = pending.id
|
||||
|
||||
open_count = {"n": 0}
|
||||
|
||||
class _TrackedSession:
|
||||
def __init__(self, inner):
|
||||
self._inner = inner
|
||||
|
||||
async def __aenter__(self):
|
||||
result = await self._inner.__aenter__()
|
||||
open_count["n"] += 1
|
||||
return result
|
||||
|
||||
async def __aexit__(self, *exc):
|
||||
open_count["n"] -= 1
|
||||
return await self._inner.__aexit__(*exc)
|
||||
|
||||
def tracking_session_factory():
|
||||
return _TrackedSession(session_factory())
|
||||
|
||||
class TrackingClient(FakeClient):
|
||||
async def get_transaction(self, txid, verbose=False):
|
||||
assert open_count["n"] == 0, "a session was held open during a network call"
|
||||
return await super().get_transaction(txid, verbose)
|
||||
|
||||
async def broadcast(self, raw_tx_hex):
|
||||
assert open_count["n"] == 0, "a session was held open during the broadcast"
|
||||
return await super().broadcast(raw_tx_hex)
|
||||
|
||||
client = TrackingClient({utxo_txid: utxo_amount})
|
||||
await bump_fee(tracking_session_factory, client, pending_id)
|
||||
|
||||
assert client.broadcasted
|
||||
assert open_count["n"] == 0 # nothing left open afterwards either
|
||||
|
||||
|
||||
async def test_bump_fee_is_a_noop_when_no_longer_pending(session_factory):
|
||||
"""A row can legitimately confirm (or otherwise leave "pending") between
|
||||
being read as due and RbfBumper actually attempting the bump — a normal
|
||||
race, not an error. Must return quietly rather than raising or touching
|
||||
the network."""
|
||||
async with session_factory() as session:
|
||||
user = User(username="grace", password_hash="x", derivation_index=0, address="plm1qxxx")
|
||||
session.add(user)
|
||||
await session.commit()
|
||||
pending = PendingTransaction(
|
||||
kind="bet",
|
||||
user_id=user.id,
|
||||
current_txid="already-confirmed-txid",
|
||||
fee_rate_sat_vb=1,
|
||||
raw_tx_hex="00",
|
||||
status="confirmed",
|
||||
broadcast_at=datetime.now(timezone.utc) - timedelta(seconds=1000),
|
||||
)
|
||||
session.add(pending)
|
||||
await session.commit()
|
||||
pending_id = pending.id
|
||||
|
||||
client = FakeClient({})
|
||||
|
||||
result = await bump_fee(session_factory, client, pending_id)
|
||||
|
||||
assert result is None
|
||||
assert not client.broadcasted
|
||||
|
||||
|
||||
async def test_bump_fee_is_a_noop_when_the_row_is_gone(session_factory):
|
||||
client = FakeClient({})
|
||||
result = await bump_fee(session_factory, client, 999_999)
|
||||
assert result is None
|
||||
assert not client.broadcasted
|
||||
@@ -0,0 +1,35 @@
|
||||
"""B-43: the Caddyfile must keep sending baseline security headers. Caddy adds
|
||||
none of these on its own, and the JWT lives in localStorage, so a regression
|
||||
here silently reopens an XSS/clickjacking exposure with no test ever failing
|
||||
in the Python suite (the Caddyfile isn't imported/exercised by anything else)."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
CADDYFILE = (Path(__file__).parent.parent.parent / "Caddyfile").read_text()
|
||||
|
||||
|
||||
def test_header_block_present():
|
||||
assert "header {" in CADDYFILE
|
||||
|
||||
|
||||
def test_hsts_is_set():
|
||||
assert "Strict-Transport-Security" in CADDYFILE
|
||||
assert "max-age=" in CADDYFILE
|
||||
|
||||
|
||||
def test_nosniff_is_set():
|
||||
assert 'X-Content-Type-Options "nosniff"' in CADDYFILE
|
||||
|
||||
|
||||
def test_frame_ancestors_are_blocked():
|
||||
assert 'X-Frame-Options "DENY"' in CADDYFILE
|
||||
assert "frame-ancestors 'none'" in CADDYFILE
|
||||
|
||||
|
||||
def test_referrer_policy_is_set():
|
||||
assert "Referrer-Policy" in CADDYFILE
|
||||
|
||||
|
||||
def test_csp_default_src_is_self():
|
||||
assert "Content-Security-Policy" in CADDYFILE
|
||||
assert "default-src 'self'" in CADDYFILE
|
||||
@@ -0,0 +1,43 @@
|
||||
"""app.api.client_ip is shared by the login/registration throttles (B-33) and
|
||||
the SSE per-IP subscriber cap (B-38) — both depend on it correctly preferring
|
||||
X-Forwarded-For (Caddy reverse-proxies every request, see Caddyfile) over
|
||||
request.client.host, which would otherwise be the proxy's own address."""
|
||||
|
||||
from starlette.requests import Request
|
||||
|
||||
from app.api.client_ip import client_ip
|
||||
|
||||
|
||||
def _request(*, forwarded: str | None = None, client_host: str | None = "127.0.0.1") -> Request:
|
||||
headers = [(b"x-forwarded-for", forwarded.encode())] if forwarded else []
|
||||
scope = {
|
||||
"type": "http",
|
||||
"headers": headers,
|
||||
"client": (client_host, 12345) if client_host else None,
|
||||
}
|
||||
return Request(scope)
|
||||
|
||||
|
||||
def test_client_ip_prefers_x_forwarded_for():
|
||||
request = _request(forwarded="5.6.7.8", client_host="10.0.0.1")
|
||||
assert client_ip(request) == "5.6.7.8"
|
||||
|
||||
|
||||
def test_client_ip_takes_the_first_hop_of_a_forwarded_chain():
|
||||
request = _request(forwarded="5.6.7.8, 10.0.0.1, 172.17.0.1")
|
||||
assert client_ip(request) == "5.6.7.8"
|
||||
|
||||
|
||||
def test_client_ip_strips_whitespace():
|
||||
request = _request(forwarded=" 5.6.7.8 , 10.0.0.1")
|
||||
assert client_ip(request) == "5.6.7.8"
|
||||
|
||||
|
||||
def test_client_ip_falls_back_to_request_client_without_the_header():
|
||||
request = _request(forwarded=None, client_host="10.0.0.1")
|
||||
assert client_ip(request) == "10.0.0.1"
|
||||
|
||||
|
||||
def test_client_ip_falls_back_to_unknown_with_neither():
|
||||
request = _request(forwarded=None, client_host=None)
|
||||
assert client_ip(request) == "unknown"
|
||||
+132
-36
@@ -1,41 +1,83 @@
|
||||
import pytest
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||
|
||||
import app.bets.confirmation # noqa: F401 (registers the "bet" handler)
|
||||
import app.rounds.confirmation # noqa: F401 (registers the "payout" handler)
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.config import settings
|
||||
from app.db.base import Base
|
||||
from app.db.models import PendingTransaction, Round, RoundParticipant
|
||||
from app.db.models import PendingTransaction, Round, RoundParticipant, User
|
||||
from app.electrum.scripthash import address_to_scripthash
|
||||
from app.tx.confirmation import poll_once
|
||||
from app.wallet.hd import derive_user_address
|
||||
|
||||
|
||||
class FakeClient:
|
||||
def __init__(self, confirmations_by_txid: dict[str, int]):
|
||||
self._confirmations = confirmations_by_txid
|
||||
"""B-41: poll_once now asks blockchain.scripthash.get_history rather than a
|
||||
verbose blockchain.transaction.get, so this hands back a flat history —
|
||||
height > 0 means confirmed at that height, 0 (or absent) means still in the
|
||||
mempool. The scripthash argument is ignored: every candidate's derived
|
||||
address is looked up against the same known universe of txids, which is
|
||||
fine since matching happens on tx_hash, not on which address asked."""
|
||||
|
||||
async def get_transaction(self, txid: str, verbose: bool = False) -> dict:
|
||||
return {"confirmations": self._confirmations.get(txid, 0)}
|
||||
def __init__(self, heights_by_txid: dict[str, int]):
|
||||
self._heights = heights_by_txid
|
||||
|
||||
async def get_history(self, scripthash: str) -> list[dict]:
|
||||
return [{"tx_hash": txid, "height": height} for txid, height in self._heights.items()]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def session_factory():
|
||||
async def session_factory(tmp_path, monkeypatch):
|
||||
# own_address_for (B-41) derives each row's address via the HD wallet, so
|
||||
# poll_once now needs a real master key — same bootstrap test_broadcast.py
|
||||
# and test_reconcile.py use.
|
||||
monkeypatch.setattr(settings, "master_key_path", str(tmp_path / "master.xprv.enc"))
|
||||
monkeypatch.setattr(
|
||||
settings,
|
||||
"xprv_encryption_key",
|
||||
__import__("cryptography.fernet", fromlist=["Fernet"]).Fernet.generate_key().decode(),
|
||||
)
|
||||
from app.wallet import hd
|
||||
|
||||
hd._account_key = None
|
||||
hd.generate_master_key()
|
||||
|
||||
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
yield async_sessionmaker(engine, expire_on_commit=False)
|
||||
await engine.dispose()
|
||||
hd._account_key = None
|
||||
|
||||
|
||||
async def _make_user(session, derivation_index: int) -> User:
|
||||
user = User(
|
||||
username=f"user{derivation_index}",
|
||||
password_hash="x",
|
||||
derivation_index=derivation_index,
|
||||
address=derive_user_address(derivation_index),
|
||||
)
|
||||
session.add(user)
|
||||
await session.flush()
|
||||
return user
|
||||
|
||||
|
||||
async def test_bet_confirmation_marks_participant_confirmed(session_factory):
|
||||
async with session_factory() as session:
|
||||
user = await _make_user(session, 0)
|
||||
session.add(Round(id=1, status="open"))
|
||||
session.add(
|
||||
RoundParticipant(
|
||||
round_id=1, user_id=1, bet_amount_sats=1_000, bet_txid="tx1", status="broadcast"
|
||||
round_id=1, user_id=user.id, bet_amount_sats=1_000, bet_txid="tx1", status="broadcast"
|
||||
)
|
||||
)
|
||||
session.add(
|
||||
PendingTransaction(kind="bet", round_id=1, user_id=1, current_txid="tx1", fee_rate_sat_vb=1, raw_tx_hex="00", status="pending")
|
||||
PendingTransaction(
|
||||
kind="bet", round_id=1, user_id=user.id, current_txid="tx1", fee_rate_sat_vb=1,
|
||||
raw_tx_hex="00", status="pending",
|
||||
)
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
@@ -53,9 +95,17 @@ async def test_bet_confirmation_marks_participant_confirmed(session_factory):
|
||||
|
||||
async def test_unconfirmed_tx_is_left_pending(session_factory):
|
||||
async with session_factory() as session:
|
||||
user = await _make_user(session, 0)
|
||||
session.add(Round(id=2, status="open"))
|
||||
session.add(RoundParticipant(round_id=2, user_id=1, bet_amount_sats=1_000, bet_txid="tx2", status="broadcast"))
|
||||
session.add(PendingTransaction(kind="bet", round_id=2, user_id=1, current_txid="tx2", fee_rate_sat_vb=1, raw_tx_hex="00", status="pending"))
|
||||
session.add(
|
||||
RoundParticipant(round_id=2, user_id=user.id, bet_amount_sats=1_000, bet_txid="tx2", status="broadcast")
|
||||
)
|
||||
session.add(
|
||||
PendingTransaction(
|
||||
kind="bet", round_id=2, user_id=user.id, current_txid="tx2", fee_rate_sat_vb=1,
|
||||
raw_tx_hex="00", status="pending",
|
||||
)
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
client = FakeClient({"tx2": 0})
|
||||
@@ -70,7 +120,11 @@ async def test_unconfirmed_tx_is_left_pending(session_factory):
|
||||
async def test_payout_confirmation_closes_round(session_factory):
|
||||
async with session_factory() as session:
|
||||
session.add(Round(id=3, status="paying_out", payout_txid="tx3"))
|
||||
session.add(PendingTransaction(kind="payout", round_id=3, current_txid="tx3", fee_rate_sat_vb=1, raw_tx_hex="00", status="pending"))
|
||||
session.add(
|
||||
PendingTransaction(
|
||||
kind="payout", round_id=3, current_txid="tx3", fee_rate_sat_vb=1, raw_tx_hex="00", status="pending"
|
||||
)
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
client = FakeClient({"tx3": 2})
|
||||
@@ -83,43 +137,52 @@ async def test_payout_confirmation_closes_round(session_factory):
|
||||
|
||||
|
||||
class ExplodingClient:
|
||||
"""Answers for one txid and raises for the other — a tx the server no longer
|
||||
knows (dropped from the mempool, replaced by a bump)."""
|
||||
"""Answers for one address's history and raises for the other's — the
|
||||
get_history equivalent of a server that no longer knows a particular tx
|
||||
(dropped from the mempool, replaced by a bump)."""
|
||||
|
||||
def __init__(self, known: dict[str, int], exploding_txid: str):
|
||||
self._known = known
|
||||
self._exploding = exploding_txid
|
||||
def __init__(self, heights_by_txid: dict[str, int], exploding_scripthash: str):
|
||||
self._heights = heights_by_txid
|
||||
self._exploding = exploding_scripthash
|
||||
|
||||
async def get_transaction(self, txid: str, verbose: bool = False) -> dict:
|
||||
if txid == self._exploding:
|
||||
raise RuntimeError("missing transaction")
|
||||
return {"confirmations": self._known.get(txid, 0)}
|
||||
async def get_history(self, scripthash: str) -> list[dict]:
|
||||
if scripthash == self._exploding:
|
||||
raise RuntimeError("server error")
|
||||
return [{"tx_hash": txid, "height": height} for txid, height in self._heights.items()]
|
||||
|
||||
|
||||
async def test_one_unresolvable_txid_does_not_block_the_others(session_factory):
|
||||
"""B-03: the lookup used to be unguarded, so a single unknown txid aborted the
|
||||
whole pass — nothing confirmed again until an operator intervened, which in turn
|
||||
meant no round could ever close."""
|
||||
async def test_one_unresolvable_candidate_does_not_block_the_others(session_factory):
|
||||
"""B-03: the lookup used to be unguarded, so a single failing candidate aborted
|
||||
the whole pass — nothing confirmed again until an operator intervened, which in
|
||||
turn meant no round could ever close. B-41 changed the failure unit from "one
|
||||
txid" to "one address's history", but the isolation guarantee is the same."""
|
||||
async with session_factory() as session:
|
||||
good_user = await _make_user(session, 0)
|
||||
gone_user = await _make_user(session, 1)
|
||||
session.add(Round(id=10, status="open"))
|
||||
session.add(
|
||||
RoundParticipant(round_id=10, user_id=1, bet_amount_sats=1_000, bet_txid="good", status="broadcast")
|
||||
)
|
||||
session.add(
|
||||
PendingTransaction(
|
||||
kind="bet", round_id=10, user_id=2, current_txid="gone", fee_rate_sat_vb=1, raw_tx_hex="00",
|
||||
status="pending",
|
||||
RoundParticipant(
|
||||
round_id=10, user_id=good_user.id, bet_amount_sats=1_000, bet_txid="good", status="broadcast"
|
||||
)
|
||||
)
|
||||
session.add(
|
||||
PendingTransaction(
|
||||
kind="bet", round_id=10, user_id=1, current_txid="good", fee_rate_sat_vb=1, raw_tx_hex="00",
|
||||
status="pending",
|
||||
kind="bet", round_id=10, user_id=gone_user.id, current_txid="gone", fee_rate_sat_vb=1,
|
||||
raw_tx_hex="00", status="pending",
|
||||
)
|
||||
)
|
||||
session.add(
|
||||
PendingTransaction(
|
||||
kind="bet", round_id=10, user_id=good_user.id, current_txid="good", fee_rate_sat_vb=1,
|
||||
raw_tx_hex="00", status="pending",
|
||||
)
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
confirmed = await poll_once(session_factory, ExplodingClient({"good": 1}, exploding_txid="gone"))
|
||||
exploding_scripthash = address_to_scripthash(derive_user_address(1))
|
||||
confirmed = await poll_once(
|
||||
session_factory, ExplodingClient({"good": 1}, exploding_scripthash=exploding_scripthash)
|
||||
)
|
||||
assert confirmed == 1 # the healthy one still got processed
|
||||
|
||||
async with session_factory() as session:
|
||||
@@ -135,15 +198,16 @@ async def test_bet_confirms_after_an_rbf_bump_changed_the_txid(session_factory):
|
||||
a txid no participant carried — the participant stayed "broadcast" forever and
|
||||
the round could never close. It now resolves by (round_id, user_id)."""
|
||||
async with session_factory() as session:
|
||||
user = await _make_user(session, 0)
|
||||
session.add(Round(id=11, status="open"))
|
||||
session.add(
|
||||
RoundParticipant(
|
||||
round_id=11, user_id=7, bet_amount_sats=1_000, bet_txid="old-txid", status="broadcast"
|
||||
round_id=11, user_id=user.id, bet_amount_sats=1_000, bet_txid="old-txid", status="broadcast"
|
||||
)
|
||||
)
|
||||
session.add(
|
||||
PendingTransaction(
|
||||
kind="bet", round_id=11, user_id=7, current_txid="bumped-txid", fee_rate_sat_vb=2,
|
||||
kind="bet", round_id=11, user_id=user.id, current_txid="bumped-txid", fee_rate_sat_vb=2,
|
||||
raw_tx_hex="00", status="pending", replaced_by_txid="old-txid",
|
||||
)
|
||||
)
|
||||
@@ -171,3 +235,35 @@ async def test_payout_confirms_after_an_rbf_bump_changed_the_txid(session_factor
|
||||
|
||||
async with session_factory() as session:
|
||||
assert (await session.get(Round, 12)).status == "closed"
|
||||
|
||||
|
||||
async def test_poll_once_caches_history_per_scripthash(session_factory):
|
||||
"""Two pending bets from the same user share one address — fetching its
|
||||
history twice in one pass would be wasteful."""
|
||||
async with session_factory() as session:
|
||||
user = await _make_user(session, 0)
|
||||
session.add(Round(id=20, status="open"))
|
||||
session.add(
|
||||
PendingTransaction(
|
||||
kind="bet", round_id=20, user_id=user.id, current_txid="tx-a", fee_rate_sat_vb=1,
|
||||
raw_tx_hex="00", status="pending",
|
||||
)
|
||||
)
|
||||
session.add(
|
||||
PendingTransaction(
|
||||
kind="withdrawal", user_id=user.id, current_txid="tx-b", fee_rate_sat_vb=1,
|
||||
raw_tx_hex="00", status="pending",
|
||||
)
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
call_count = {"n": 0}
|
||||
|
||||
class CountingClient:
|
||||
async def get_history(self, scripthash: str) -> list[dict]:
|
||||
call_count["n"] += 1
|
||||
return [{"tx_hash": "tx-a", "height": 0}, {"tx_hash": "tx-b", "height": 0}]
|
||||
|
||||
await poll_once(session_factory, CountingClient())
|
||||
|
||||
assert call_count["n"] == 1
|
||||
@@ -0,0 +1,61 @@
|
||||
"""Regression tests for B-39: SQLite must run in WAL mode with a busy_timeout,
|
||||
since this app has five concurrent background tasks plus every HTTP handler
|
||||
sharing one database file, and the default rollback-journal mode lets a writer
|
||||
block every reader and fails a second writer immediately instead of waiting."""
|
||||
|
||||
import pytest
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
|
||||
from app.db.base import _SQLITE_BUSY_TIMEOUT_MS, _register_sqlite_pragmas
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def sqlite_engine(tmp_path):
|
||||
# WAL needs a real file (it writes a companion -wal/-shm file alongside it) —
|
||||
# ":memory:" wouldn't exercise the same path.
|
||||
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path}/test.db")
|
||||
yield engine
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
async def _pragma(engine, name: str):
|
||||
async with engine.connect() as conn:
|
||||
result = await conn.exec_driver_sql(f"PRAGMA {name}")
|
||||
return result.fetchone()[0]
|
||||
|
||||
|
||||
async def test_register_sqlite_pragmas_enables_wal_and_busy_timeout(sqlite_engine):
|
||||
_register_sqlite_pragmas(sqlite_engine)
|
||||
|
||||
assert (await _pragma(sqlite_engine, "journal_mode")).lower() == "wal"
|
||||
assert await _pragma(sqlite_engine, "busy_timeout") == _SQLITE_BUSY_TIMEOUT_MS
|
||||
assert await _pragma(sqlite_engine, "synchronous") == 1 # NORMAL
|
||||
|
||||
|
||||
async def test_register_sqlite_pragmas_applies_to_every_new_connection(sqlite_engine):
|
||||
"""The pool can open more than one underlying DBAPI connection over the
|
||||
engine's lifetime — the pragmas must be re-applied to each one, not just
|
||||
the first, or a later connection would silently fall back to SQLite's
|
||||
defaults."""
|
||||
_register_sqlite_pragmas(sqlite_engine)
|
||||
|
||||
async with sqlite_engine.connect() as first:
|
||||
await first.exec_driver_sql("PRAGMA journal_mode")
|
||||
|
||||
async with sqlite_engine.connect() as second:
|
||||
result = await second.exec_driver_sql("PRAGMA busy_timeout")
|
||||
assert result.fetchone()[0] == _SQLITE_BUSY_TIMEOUT_MS
|
||||
|
||||
|
||||
def test_register_sqlite_pragmas_is_a_noop_for_other_dialects():
|
||||
"""Must not touch (or crash on) a non-sqlite engine — e.g. a future
|
||||
PostgreSQL DATABASE_URL, which neither needs nor understands these
|
||||
pragmas."""
|
||||
|
||||
class _FakeDialect:
|
||||
name = "postgresql"
|
||||
|
||||
class _FakeEngine:
|
||||
dialect = _FakeDialect()
|
||||
|
||||
_register_sqlite_pragmas(_FakeEngine()) # must not raise
|
||||
@@ -0,0 +1,28 @@
|
||||
"""B-42: Swagger/ReDoc/OpenAPI JSON must not be reachable unless explicitly enabled —
|
||||
they enumerate the whole API surface, admin endpoints included."""
|
||||
|
||||
import importlib
|
||||
|
||||
from app.config import settings
|
||||
|
||||
|
||||
def _reload_main():
|
||||
import app.main
|
||||
|
||||
return importlib.reload(app.main)
|
||||
|
||||
|
||||
def test_docs_disabled_by_default(monkeypatch):
|
||||
monkeypatch.setattr(settings, "enable_api_docs", False)
|
||||
main = _reload_main()
|
||||
assert main.app.docs_url is None
|
||||
assert main.app.redoc_url is None
|
||||
assert main.app.openapi_url is None
|
||||
|
||||
|
||||
def test_docs_enabled_when_configured(monkeypatch):
|
||||
monkeypatch.setattr(settings, "enable_api_docs", True)
|
||||
main = _reload_main()
|
||||
assert main.app.docs_url == "/docs"
|
||||
assert main.app.redoc_url == "/redoc"
|
||||
assert main.app.openapi_url == "/openapi.json"
|
||||
@@ -0,0 +1,18 @@
|
||||
"""B-44: README and docs/running-the-server.md must not document a bare
|
||||
`uvicorn --reload` workflow — the server always runs via Docker, in dev and
|
||||
production alike (CLAUDE.md's "Commands" section), and the two files had
|
||||
drifted back to contradicting that policy."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).parent.parent.parent
|
||||
|
||||
|
||||
def test_readme_has_no_bare_uvicorn_command():
|
||||
readme = (REPO_ROOT / "README.md").read_text()
|
||||
assert "uvicorn app.main:app --reload" not in readme
|
||||
|
||||
|
||||
def test_running_the_server_doc_has_no_bare_uvicorn_command():
|
||||
doc = (REPO_ROOT / "docs" / "running-the-server.md").read_text()
|
||||
assert "uvicorn app.main:app --reload" not in doc
|
||||
@@ -0,0 +1,16 @@
|
||||
"""B-47: raw_tx_hex (a full raw signed transaction hex) and payload_json (an
|
||||
arbitrary audit payload) must stay `Text`, not a bare `String`/`VARCHAR` with
|
||||
no length -- SQLite and PostgreSQL accept that, but other backends (e.g.
|
||||
MySQL) require a length on VARCHAR and would reject it."""
|
||||
|
||||
from sqlalchemy import Text
|
||||
|
||||
from app.db.models import AuditLog, PendingTransaction
|
||||
|
||||
|
||||
def test_pending_transaction_raw_tx_hex_is_text():
|
||||
assert isinstance(PendingTransaction.__table__.c.raw_tx_hex.type, Text)
|
||||
|
||||
|
||||
def test_audit_log_payload_json_is_text():
|
||||
assert isinstance(AuditLog.__table__.c.payload_json.type, Text)
|
||||
@@ -0,0 +1,55 @@
|
||||
"""B-41: own_address_for is the single place tx/confirmation.py and
|
||||
tx/reconcile.py derive a PendingTransaction's own address from — a payout's
|
||||
address must always be the pool's, everything else the actual user's."""
|
||||
|
||||
import pytest
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||
|
||||
from app.config import settings
|
||||
from app.db.base import Base
|
||||
from app.db.models import User
|
||||
from app.tx.pending_address import own_address_for
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def session_factory(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(settings, "master_key_path", str(tmp_path / "master.xprv.enc"))
|
||||
monkeypatch.setattr(
|
||||
settings,
|
||||
"xprv_encryption_key",
|
||||
__import__("cryptography.fernet", fromlist=["Fernet"]).Fernet.generate_key().decode(),
|
||||
)
|
||||
from app.wallet import hd
|
||||
|
||||
hd._account_key = None
|
||||
hd.generate_master_key()
|
||||
|
||||
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
yield async_sessionmaker(engine, expire_on_commit=False)
|
||||
await engine.dispose()
|
||||
hd._account_key = None
|
||||
|
||||
|
||||
async def test_payout_uses_the_pool_address_regardless_of_user_id(session_factory):
|
||||
from app.wallet.hd import derive_pool_address
|
||||
|
||||
async with session_factory() as session:
|
||||
address = await own_address_for(session, "payout", None)
|
||||
|
||||
assert address == derive_pool_address()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("kind", ["bet", "withdrawal"])
|
||||
async def test_bet_and_withdrawal_use_the_users_own_address(session_factory, kind):
|
||||
from app.wallet.hd import derive_user_address
|
||||
|
||||
async with session_factory() as session:
|
||||
user = User(username="alice", password_hash="x", derivation_index=3, address=derive_user_address(3))
|
||||
session.add(user)
|
||||
await session.flush()
|
||||
|
||||
address = await own_address_for(session, kind, user.id)
|
||||
|
||||
assert address == derive_user_address(3)
|
||||
@@ -1,5 +1,10 @@
|
||||
"""Regression tests for B-04 (and the "building" half of B-08): a transaction that
|
||||
never made it onto the chain must give the coins back instead of freezing them."""
|
||||
never made it onto the chain must give the coins back instead of freezing them.
|
||||
|
||||
Also covers B-41: existence/reconciliation checks go through
|
||||
blockchain.scripthash.get_history rather than a verbose blockchain.transaction.get
|
||||
reply, so the fake clients below implement get_history.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from embit import script
|
||||
@@ -7,37 +12,60 @@ from embit.transaction import Transaction, TransactionInput, TransactionOutput
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||
|
||||
from app.config import settings
|
||||
from app.db.base import Base
|
||||
from app.db.models import AuditLog, PendingTransaction, RoundParticipant, User, UtxoEvent, Withdrawal
|
||||
from app.tx.reconcile import reconcile_once
|
||||
|
||||
|
||||
class UnknownTxClient:
|
||||
"""A server that doesn't know any of the txids it's asked about."""
|
||||
"""A server whose history for any address never includes our txid."""
|
||||
|
||||
async def get_transaction(self, txid: str, verbose: bool = False):
|
||||
raise RuntimeError(f"missing transaction {txid}")
|
||||
async def get_history(self, scripthash: str) -> list[dict]:
|
||||
return []
|
||||
|
||||
|
||||
class KnownTxClient:
|
||||
async def get_transaction(self, txid: str, verbose: bool = False):
|
||||
return {"txid": txid, "confirmations": 0}
|
||||
"""A server whose history for the address includes our txid — mined or
|
||||
still in the mempool doesn't matter for existence, only for confirmation
|
||||
(which is tx/confirmation.py's concern, not reconcile.py's)."""
|
||||
|
||||
def __init__(self, txid: str = "betxid"):
|
||||
self._txid = txid
|
||||
|
||||
async def get_history(self, scripthash: str) -> list[dict]:
|
||||
return [{"tx_hash": self._txid, "height": 100}]
|
||||
|
||||
|
||||
class BrokenClient:
|
||||
"""A transport failure — says nothing about whether the tx exists."""
|
||||
|
||||
async def get_transaction(self, txid: str, verbose: bool = False):
|
||||
async def get_history(self, scripthash: str) -> list[dict]:
|
||||
raise ConnectionResetError("connection reset")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def session_factory():
|
||||
async def session_factory(tmp_path, monkeypatch):
|
||||
# own_address_for (B-41) derives each row's address via the HD wallet rather
|
||||
# than trusting the DB's address column, so reconcile_once now needs a real
|
||||
# master key set up — same bootstrap test_broadcast.py uses.
|
||||
monkeypatch.setattr(settings, "master_key_path", str(tmp_path / "master.xprv.enc"))
|
||||
monkeypatch.setattr(
|
||||
settings,
|
||||
"xprv_encryption_key",
|
||||
__import__("cryptography.fernet", fromlist=["Fernet"]).Fernet.generate_key().decode(),
|
||||
)
|
||||
from app.wallet import hd
|
||||
|
||||
hd._account_key = None
|
||||
hd.generate_master_key()
|
||||
|
||||
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
yield async_sessionmaker(engine, expire_on_commit=False)
|
||||
await engine.dispose()
|
||||
hd._account_key = None
|
||||
|
||||
|
||||
# A real (unsigned) transaction spending one input, built rather than hand-written
|
||||
@@ -66,11 +94,19 @@ async def _seed_bet(
|
||||
participant_status: str,
|
||||
age_seconds: int,
|
||||
last_broadcast_age_seconds: int | None = None,
|
||||
derivation_index: int = 0,
|
||||
):
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from app.wallet.hd import derive_user_address
|
||||
|
||||
async with session_factory() as session:
|
||||
user = User(username="u", password_hash="x", derivation_index=0, address="plm1qtest")
|
||||
user = User(
|
||||
username="u",
|
||||
password_hash="x",
|
||||
derivation_index=derivation_index,
|
||||
address=derive_user_address(derivation_index),
|
||||
)
|
||||
session.add(user)
|
||||
await session.flush()
|
||||
session.add(
|
||||
@@ -145,7 +181,7 @@ async def test_promotes_a_building_row_whose_tx_did_reach_the_chain(session_fact
|
||||
session_factory, pending_status="building", participant_status="building", age_seconds=300
|
||||
)
|
||||
|
||||
resolved = await reconcile_once(session_factory, KnownTxClient())
|
||||
resolved = await reconcile_once(session_factory, KnownTxClient("betxid"))
|
||||
assert resolved == 1
|
||||
|
||||
async with session_factory() as session:
|
||||
@@ -219,8 +255,10 @@ async def test_abandoned_withdrawal_is_marked_failed_and_kept(session_factory):
|
||||
they can see it didn't go through."""
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from app.wallet.hd import derive_user_address
|
||||
|
||||
async with session_factory() as session:
|
||||
user = User(username="w", password_hash="x", derivation_index=1, address="plm1qtest2")
|
||||
user = User(username="w", password_hash="x", derivation_index=1, address=derive_user_address(1))
|
||||
session.add(user)
|
||||
await session.flush()
|
||||
session.add(
|
||||
@@ -264,3 +302,42 @@ async def test_abandoned_withdrawal_is_marked_failed_and_kept(session_factory):
|
||||
assert withdrawal.status == "failed"
|
||||
assert withdrawal.txid is None
|
||||
assert (await session.scalars(select(UtxoEvent))).one().spent_txid is None
|
||||
|
||||
|
||||
# --- B-41: existence checks now use get_history and share it across candidates
|
||||
# sharing the same address, instead of a per-tx verbose blockchain.transaction.get. --
|
||||
|
||||
|
||||
async def test_reconcile_once_caches_history_per_scripthash(session_factory):
|
||||
"""Two payout PendingTransaction rows always share the same pool address —
|
||||
fetching its history twice in one pass would be wasteful and, at scale
|
||||
across many candidates on one address, needlessly slow the whole tick."""
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
async with session_factory() as session:
|
||||
old = datetime.now(timezone.utc) - timedelta(hours=7)
|
||||
session.add(
|
||||
PendingTransaction(
|
||||
kind="payout", round_id=1, current_txid="payout-a", fee_rate_sat_vb=1,
|
||||
raw_tx_hex=_RAW_TX, status="pending", broadcast_at=old, last_broadcast_at=old,
|
||||
)
|
||||
)
|
||||
session.add(
|
||||
PendingTransaction(
|
||||
kind="payout", round_id=2, current_txid="payout-b", fee_rate_sat_vb=1,
|
||||
raw_tx_hex=_RAW_TX, status="pending", broadcast_at=old, last_broadcast_at=old,
|
||||
)
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
call_count = {"n": 0}
|
||||
|
||||
class CountingClient:
|
||||
async def get_history(self, scripthash: str) -> list[dict]:
|
||||
call_count["n"] += 1
|
||||
return [{"tx_hash": "payout-a", "height": 100}, {"tx_hash": "payout-b", "height": 100}]
|
||||
|
||||
resolved = await reconcile_once(session_factory, CountingClient())
|
||||
|
||||
assert resolved == 0 # both exist — nothing to abandon or promote (already "pending")
|
||||
assert call_count["n"] == 1 # one call covered both rows sharing the pool address
|
||||
@@ -2,7 +2,7 @@ import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
from app.rounds.events import RoundEventBroadcaster, RoundEventCapacityError
|
||||
from app.rounds.events import EVICTED, RoundEventBroadcaster, RoundEventCapacityError
|
||||
|
||||
|
||||
async def test_publish_wakes_up_subscriber():
|
||||
@@ -60,3 +60,71 @@ async def test_unsubscribe_frees_a_capacity_slot():
|
||||
|
||||
broadcaster.unsubscribe(queue)
|
||||
broadcaster.subscribe() # no longer at capacity
|
||||
|
||||
|
||||
# --- B-38: a single IP must not be able to exhaust the global cap and degrade
|
||||
# every other user to polling. ----------------------------------------------------
|
||||
|
||||
|
||||
async def test_subscribe_evicts_the_same_ips_oldest_connection_past_its_cap():
|
||||
"""Past MAX_SUBSCRIBERS_PER_IP, one more stream from the *same* IP evicts
|
||||
that IP's own oldest connection rather than being refused — bounds one
|
||||
source's footprint without an outright block."""
|
||||
broadcaster = RoundEventBroadcaster(max_subscribers=100, max_per_ip=2)
|
||||
first = broadcaster.subscribe("1.2.3.4")
|
||||
second = broadcaster.subscribe("1.2.3.4")
|
||||
|
||||
third = broadcaster.subscribe("1.2.3.4") # past the per-IP cap of 2
|
||||
|
||||
assert await asyncio.wait_for(first.get(), timeout=1) is EVICTED
|
||||
assert second.empty() # untouched — only the oldest was evicted
|
||||
assert third is not None
|
||||
|
||||
|
||||
async def test_subscribe_does_not_evict_across_different_ips():
|
||||
"""A different IP hitting its own cap must never evict an unrelated IP's
|
||||
connection — that would let one abusive source crowd out real users, which
|
||||
is exactly what the global-only cap used to allow."""
|
||||
broadcaster = RoundEventBroadcaster(max_subscribers=100, max_per_ip=1)
|
||||
other_ip_queue = broadcaster.subscribe("9.9.9.9")
|
||||
|
||||
broadcaster.subscribe("1.2.3.4")
|
||||
broadcaster.subscribe("1.2.3.4") # evicts 1.2.3.4's own oldest, not 9.9.9.9's
|
||||
|
||||
assert other_ip_queue.empty()
|
||||
|
||||
|
||||
async def test_subscribe_still_enforces_the_global_cap_across_many_ips():
|
||||
"""The per-IP cap doesn't replace the global backstop — spreading across
|
||||
enough distinct IPs must still eventually hit MAX_SUBSCRIBERS."""
|
||||
broadcaster = RoundEventBroadcaster(max_subscribers=3, max_per_ip=1)
|
||||
broadcaster.subscribe("1.1.1.1")
|
||||
broadcaster.subscribe("2.2.2.2")
|
||||
broadcaster.subscribe("3.3.3.3")
|
||||
|
||||
with pytest.raises(RoundEventCapacityError):
|
||||
broadcaster.subscribe("4.4.4.4")
|
||||
|
||||
|
||||
async def test_unsubscribe_clears_the_per_ip_tracking_too():
|
||||
"""Regression guard: unsubscribe must forget the queue's IP association, or
|
||||
a churned-through connection would keep counting against that IP's cap
|
||||
forever."""
|
||||
broadcaster = RoundEventBroadcaster(max_subscribers=100, max_per_ip=1)
|
||||
queue = broadcaster.subscribe("1.2.3.4")
|
||||
broadcaster.unsubscribe(queue)
|
||||
|
||||
broadcaster.subscribe("1.2.3.4") # must not evict anything — nothing left to evict
|
||||
assert queue.empty()
|
||||
|
||||
|
||||
async def test_subscribe_defaults_to_a_shared_ip_when_none_given():
|
||||
"""Existing callers (and most tests) that don't care about IP isolation
|
||||
still share one implicit bucket rather than needing every call updated."""
|
||||
broadcaster = RoundEventBroadcaster(max_subscribers=100, max_per_ip=2)
|
||||
first = broadcaster.subscribe()
|
||||
broadcaster.subscribe()
|
||||
|
||||
broadcaster.subscribe() # past the default bucket's cap of 2 — evicts, doesn't raise
|
||||
|
||||
assert await asyncio.wait_for(first.get(), timeout=1) is EVICTED
|
||||
Reference in new issue
Block a user