Files
plm-lottery/app/main.py
T
davideandClaude Opus 5 7c4e9983ea Survive a dropped Electrum connection, and fall back to other servers
A dropped connection used to hang the whole platform permanently, and three
defects composed to do it (BUGS.md B-01):

The read loop's death was invisible. When the socket closed, _read_loop broke out
and finished, but _run_once was blocked on gather() over two notification
consumers waiting on queues nobody would ever fill again — it never returned and
never raised, so the reconnect-with-backoff logic was unreachable.
client.wait_closed() now resolves when the loop ends for any reason, and
_run_once races it against the consumers and a keepalive with
asyncio.wait(FIRST_COMPLETED).

Nothing had a timeout. request() registered a future, wrote to a half-closed
socket (drain() often doesn't raise) and awaited a reply that would never come.
That hung a POST /bets *while holding the per-user lock*, and could stop the
confirmation poller for good. Every request is now bounded at 15s, and a timeout
tears the connection down rather than leaving a server that owes us a reply in
rotation.

There was no keepalive, so on a quiet instance the normal way this connection
dies is an idle-timeout drop by the server (~10 minutes for many). A server.ping
every 60s makes that observable within a minute.

listener.client is also cleared before reconnecting, so callers stop treating a
dead connection as live.

On top of the finding, the listener now rotates over a list of servers:
ELECTRUM_FALLBACK_SERVERS holds comma-separated host:port[:notls] extras, tried
after the primary. Everything the platform does goes through this one connection
— deposit credits, broadcasts, confirmations, the chain tip the draw waits on —
which made a single hardcoded server its biggest point of failure. A failed or
dropped session moves to the next server immediately and only sleeps on the
backoff once every server has had a turn, so one dead server costs one attempt
instead of an outage, while a genuinely offline network still backs off. A
malformed entry fails at startup, not during the outage when the fallback is what
you need.

Also fixes B-19: header handling refuses a height below the current tip and
applies height and hex together, since _wait_for_next_block waits for
tip_height > tip_at_close (a regression silently added a block to the draw's
wait) and that hex is the draw's entropy source, so a mismatched pair would be
worse than a stale one.

Verified in the live deployment: the log shows the endpoint list, then "Electrum
connected to santantonio.sytes.net:50002", and the connection holds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 00:34:59 +02:00

141 lines
5.4 KiB
Python

import asyncio
import logging
from contextlib import asynccontextmanager
from pathlib import Path
from fastapi import FastAPI, Request, status
from fastapi.responses import FileResponse, JSONResponse
from fastapi.staticfiles import StaticFiles
from app.logging_config import setup_logging
setup_logging()
import app.bets.confirmation # noqa: F401 (registers the "bet" confirmation handler)
import app.rounds.confirmation # noqa: F401 (registers the "payout" confirmation handler)
import app.withdrawals.confirmation # noqa: F401 (registers the "withdrawal" confirmation handler)
from app.api.routes.admin import router as admin_router
from app.api.routes.bets import router as bets_router
from app.api.routes.qr import router as qr_router
from app.api.routes.rounds import router as rounds_router
from app.api.routes.users import router as users_router
from app.api.routes.withdrawals import router as withdrawals_router
from app.auth.routes import router as auth_router
from app.api.errors import ApiError, http_error
from app.config import settings, validate_runtime_secrets
from app.db.base import AsyncSessionLocal
from app.electrum.client import ElectrumClient, ElectrumEndpoint, parse_endpoints
from app.electrum.listener import ElectrumListener
from app.rounds.scheduler import RoundScheduler
from app.tx.broadcast import RbfBumper
from app.tx.confirmation import ConfirmationPoller
from app.tx.locks import UserLocks
from app.tx.reconcile import PendingTransactionReconciler
logger = logging.getLogger(__name__)
def _make_electrum_client(endpoint: ElectrumEndpoint) -> ElectrumClient:
return ElectrumClient(endpoint.host, endpoint.port, endpoint.use_ssl)
@asynccontextmanager
async def lifespan(app: FastAPI):
# Refuses to serve rather than starting up half-configured — see B-15 in BUGS.md.
validate_runtime_secrets()
endpoints = parse_endpoints(
settings.electrum_host,
settings.electrum_port,
settings.electrum_use_ssl,
settings.electrum_fallback_servers,
)
logger.info("Electrum endpoints (in rotation order): %s", ", ".join(str(e) for e in endpoints))
listener = ElectrumListener(_make_electrum_client, AsyncSessionLocal, endpoints)
app.state.electrum_listener = listener
app.state.user_locks = UserLocks()
scheduler = RoundScheduler(AsyncSessionLocal, listener)
poller = ConfirmationPoller(AsyncSessionLocal, lambda: listener.client)
bumper = RbfBumper(AsyncSessionLocal, lambda: listener.client)
# Resolves in-flight transactions against the chain — the piece that lets the
# system recover on its own from a broadcast that never confirmed (B-04/B-08).
reconciler = PendingTransactionReconciler(AsyncSessionLocal, lambda: listener.client)
tasks = [
asyncio.create_task(listener.run()),
asyncio.create_task(scheduler.run()),
asyncio.create_task(poller.run()),
asyncio.create_task(bumper.run()),
asyncio.create_task(reconciler.run()),
]
try:
yield
finally:
for task in tasks:
task.cancel()
if listener.client is not None:
await listener.client.close()
app = FastAPI(title="PLM Lottery", lifespan=lifespan)
app.include_router(auth_router)
app.include_router(users_router)
app.include_router(bets_router)
app.include_router(withdrawals_router)
app.include_router(admin_router)
app.include_router(qr_router)
app.include_router(rounds_router)
@app.exception_handler(Exception)
async def log_unhandled_exception(request: Request, exc: Exception) -> JSONResponse:
"""Answers with the same structured `detail` shape as every deliberate failure
(app/api/errors.py) so clients never have to special-case unexpected errors.
The exception itself stays in logs/app.log only — never in the response body."""
logger.exception("Unhandled error on %s %s", request.method, request.url.path)
return JSONResponse(
status_code=500,
content={"detail": ApiError("internal_error", "internal server error").as_detail()},
)
@app.get("/health")
async def health() -> dict[str, str]:
return {"status": "ok"}
_NO_STORE_HEADERS = {"Cache-Control": "no-store"}
@app.get("/", include_in_schema=False)
async def index_page() -> FileResponse:
return FileResponse("app/static/index.html", headers=_NO_STORE_HEADERS)
@app.get("/admin", include_in_schema=False)
async def admin_panel() -> FileResponse:
return FileResponse("app/static/admin.html", headers=_NO_STORE_HEADERS)
_USER_GUIDE_PATH = Path("docs/guida-utente.md")
@app.get("/guida", include_in_schema=False)
async def user_guide() -> FileResponse:
"""Serves docs/guida-utente.md as plain text so it opens inline in the
browser (no markdown rendering — keeps this simple), linked from the
navbar's help button in app/static/index.html.
The Dockerfile must COPY docs/ for this to exist inside the container (it
didn't, which made this a guaranteed 500 in every real deployment — B-16).
Answers a clean 404 if the file is missing rather than an unhandled error."""
if not _USER_GUIDE_PATH.is_file():
logger.error("user guide missing at %s (is docs/ shipped in the image?)", _USER_GUIDE_PATH)
raise http_error(status.HTTP_404_NOT_FOUND, "guide_unavailable", "user guide not available")
return FileResponse(_USER_GUIDE_PATH, media_type="text/plain; charset=utf-8")
app.mount("/", StaticFiles(directory="app/static", html=True), name="static")