From 54dbffb552d0a1fe3abe631172ee223682d8571b Mon Sep 17 00:00:00 2001 From: Davide Grilli Date: Fri, 16 Oct 2026 14:25:35 +0200 Subject: [PATCH] Credit deposits at deposit_confirmations and verify the round's coins with the quorum before paying (B-82, DEP-03, DEP-04, DEP-09, DEP-12, DEP-14, ELC-17, ELC-27, RND-21, RND-22, RND-25, RND-39, DAT-10, DAT-11, DAT-24, DAT-62, DAT-67, DAT-68, AUD-12, AUD-44, AUD-45, API-30, API-31, API-36, ADR-06, ADR-52) A user could bet and then double-spend the 1-conference deposit that funded the bet with a private chain: the bet left the chain but its participant stayed `confirmed`, and the payout failed "insufficient pool UTXOs" for ever with everyone else's money stuck. Coinbase outputs were also credited at 1 confirmation (maturity is 120). Deposits - `round_config.deposit_confirmations` (default 6, 1-100, editable from /admin) is the threshold for a deposit; an output of a platform transaction (any txid of a pending_transactions row, RBF versions included) keeps 1; a coinbase output needs 120, recognised from the raw transaction, which must hash to the txid (`is_coinbase`, cached). `read_deposit_policy` / `classify_new_deposits` run between the read and the write, so no session is open across the lookup. - The quorum must also place the coin deep enough on our own tip (ELC-17). - Confirmed outputs short of their threshold are pending deposits (memory only, replaced at every refresh): shown in `pending_balance_sats` / `has_pending` (also for DEP-13's withdrawal message), never spendable. No scripthash notification announces a maturing deposit, so each new block refreshes only the users that have one, one task per user (DEP-09 now says so). Payout - `_trigger_payout` verifies the round's confirmed participants' bet outputs with the other servers (`verify_pool_outpoints`: strict majority of >= 2 answers per outpoint, a mempool-only coin or a tie gives no verdict) and builds from exactly those outpoints (RND-22), never from whatever else sits at the address. No quorum: `payout_failed`, `pool verification unavailable`, nothing changes. Bets that are gone become `reorged_out`, one `participant_reorged_out` row each, `pool_amount_sats` rewritten (only downward) in the same transaction, and the 70/30 split is computed on what is left. - Winner's own bet gone: nothing is built; `rounds.payout_blocked_at` (Alembic f2c64a8d1e93, with `deposit_confirmations`, plain add_column) is set, `payout_blocked` and the pause with `paused_reason = 'payout_blocked'` are written in one transaction; `POST /admin/resume` clears the mark and the next attempt verifies again. /admin/rounds returns it. - Rounds opened before B-80 (pool = the shared reserve address) keep the old active-server `listunspent` choice, since that address holds no single round's bets. Ambiguities resolved: the winner's row stays `confirmed` while held (AUD-45 says every participant whose bet is gone becomes `reorged_out`, but OPS-30/DAT-68 need the verification to repeat after the resume, which only a `confirmed` row allows); `/`'s deposit hint no longer says "1 confirmation" (7 languages, same keys); guides updated. Tests changed because the spec changed: listener tests that credited height-100 coins at tip 0 now set a tip past every threshold; the payout fixtures seed confirmed participants (the new source of the inputs) instead of an active-server `listunspent`, FakeNetwork answers the verification, and test_round_pool_address asserts the quorum is asked about the round's address. New: thresholds at exactly 6/5, configured value, coinbase at 119/120 and a mismatching or missing raw tx, platform outputs at 1, dust, quorum height, waiting-deposit display and disappearance, block-triggered refresh, resume, verification verdicts, gone bets excluded with the split recomputed, winner gone held and paused, held payouts untouched until the resume, re-verification after it, no quorum, a payout with 1 and with 400 participants against reserve+fee+dust, admin config bounds and migration. Mutation-checked: threshold 1, ignored coinbase, no height bound, no verification and no winner hold each fail tests. Co-Authored-By: Claude Sonnet 5.5 --- BUGS.md | 16 - CLAUDE.md | 18 +- README.md | 2 +- app/api/routes/admin.py | 10 +- app/api/routes/users.py | 9 +- app/db/models.py | 6 + app/deposits/service.py | 86 ++++- app/electrum/listener.py | 152 ++++++++- app/rounds/scheduler.py | 149 +++++++- app/static/admin.html | 2 + app/static/admin.js | 2 + app/static/i18n.js | 14 +- app/static/index.html | 2 +- app/wallet/balance.py | 11 +- app/withdrawals/service.py | 5 +- ...iedono-deposit-confirmations-conferme-6.md | 8 +- ...ioni-catena-contengono-invece-ignorarle.md | 6 +- docs/api.md | 17 +- docs/architettura.md | 5 +- docs/audit-log.md | 30 +- docs/ciclo-di-vita-del-round.md | 30 +- docs/connessione-electrum.md | 30 +- docs/guida-admin.md | 1 + docs/guida-utente.md | 13 +- docs/limiti-noti.md | 56 +-- docs/modello-dati.md | 68 ++-- docs/operativita.md | 16 +- docs/registrazione-e-depositi.md | 34 +- docs/sicurezza.md | 5 +- docs/storico-bug.md | 6 +- ...eposit_confirmations_and_payout_blocked.py | 44 +++ tests/unit/fake_network.py | 16 + tests/unit/test_admin.py | 46 +++ tests/unit/test_confirmation_proof.py | 1 + tests/unit/test_deposit_confirmations.py | 312 +++++++++++++++++ tests/unit/test_electrum_listener.py | 4 + tests/unit/test_migrations.py | 10 + tests/unit/test_pool_verification.py | 319 ++++++++++++++++++ tests/unit/test_round_pool_address.py | 6 +- tests/unit/test_scheduler.py | 44 ++- tests/unit/test_sqlite_transactions.py | 6 +- tests/unit/test_users.py | 13 + 42 files changed, 1360 insertions(+), 270 deletions(-) create mode 100644 migrations/versions/f2c64a8d1e93_deposit_confirmations_and_payout_blocked.py create mode 100644 tests/unit/test_deposit_confirmations.py create mode 100644 tests/unit/test_pool_verification.py diff --git a/BUGS.md b/BUGS.md index a0409cc..edbe9a9 100644 --- a/BUGS.md +++ b/BUGS.md @@ -100,22 +100,6 @@ decisions, see `docs/limiti-noti.md`): - **Contain reorgs instead of pretending they don't happen:** detect them, pause on deep ones, and pay out only what is verifiably on chain. -## Critical - -### B-82 — A deposit reorged out after a bet leaves the round unpayable - -Deposits are credited at 1 confirmation. A user can bet, then (if they lost) -double-spend the deposit the bet spent with a private chain: the bet disappears but -its participant stays `confirmed` and counted, and the payout fails "insufficient -pool UTXOs" forever with every other player's money stuck. Immature coinbase outputs -are also credited at 1 confirmation (maturity is 120). - -Fix: deposits credited at `deposit_confirmations` (round_config, default 6), -coinbase outputs at 120. Before building a payout the round address's UTXOs are -verified with the quorum; participants whose bet outpoint is gone are marked -`reorged_out` and the 70/30 split is computed on what is actually there. If the -winner's own bet is gone the payout is held (`payout_blocked`) and the lottery pauses. - ## High ### B-87 — The RBF bump can leave a dust change output diff --git a/CLAUDE.md b/CLAUDE.md index 85104a3..7cebd28 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,9 +8,9 @@ The user communicates in Italian in chat — reply to them in Italian. Everythin ## Project status -All 10 stages of the original build order are code-complete and unit-tested — 637 tests, all under `tests/unit/` (`tests/integration/` is an empty package). Beyond them: Docker + Caddy deployment, admin dashboard (`/admin`), static test UI (`/`), pending-inclusive balance display, an SSE push channel layered over the original polling, self-service password change + admin password reset, and the reconciliation/corroboration machinery below. +All 10 stages of the original build order are code-complete and unit-tested — 670 tests, all under `tests/unit/` (`tests/integration/` is an empty package). Beyond them: Docker + Caddy deployment, admin dashboard (`/admin`), static test UI (`/`), pending-inclusive balance display, an SSE push channel layered over the original polling, self-service password change + admin password reset, and the reconciliation/corroboration machinery below. -Verified on mainnet with real money: registration + address derivation, deposit crediting (1-conf), a real 10 PLM bet (broadcast → confirmed → change credited back), and one full round cycle (close → draw on a real block hash → 70/30 payout with sat math checked against the broadcast tx → confirmation → close → next round auto-opened). **Withdrawal and the RBF bump path have never been exercised against a live broadcast** — unit-tested only. +Verified on mainnet with real money: registration + address derivation, deposit crediting (at the time, 1-conf; deposits now need `deposit_confirmations`), a real 10 PLM bet (broadcast → confirmed → change credited back), and one full round cycle (close → draw on a real block hash → 70/30 payout with sat math checked against the broadcast tx → confirmation → close → next round auto-opened). **Withdrawal and the RBF bump path have never been exercised against a live broadcast** — unit-tested only. Two full-codebase audits — 2026-07-26 (24 findings, 5 critical) and 2026-07-27 (25 more, B-25 … B-49) — are **all fixed** as of 2026-07-27, each with its own regression test. A third audit (2026-08-03, B-52 … B-72) and a fourth, the 2026-09-28 specification review (B-73 … B-98: draw manipulation, ambiguous broadcasts, single-txid tracking, single-server abandonment, the 64 KiB Electrum reply limit, unbumpable payouts, 1-conf deposit reorgs, ops hazards), live in the tracked [BUGS.md](BUGS.md), which lists only what is still **open** (B-71 … B-98 at the time of writing). The spec in `docs/` already describes the *target* behaviour for all of them — every affected requirement is marked `*Stato:* da implementare (B-nn)` with an `*Oggi:*` line saying what the code does now, so read both before assuming a requirement is implemented; the design decisions and the implementation order are in [docs/piano-correzioni.md](docs/piano-correzioni.md); the full history of every B-nn, open and closed, is in [docs/storico-bug.md](docs/storico-bug.md). Closed findings leave the file, so the ~276 `B-nn` markers left in comments across the code are pointers into git history (`git log --all --grep 'B-nn'` finds the commit that fixed one, and `git show f1a1145:BUGS.md`-style the file as it stood). A closed list is not the same as no bugs: the suite is unit-only (`tests/integration/` is empty), and withdrawal and the RBF bump have never been live-broadcast. "Known gaps" at the end of this file is for limitations accepted **by design** instead. A new finding gets the next B-nn, in its own commit with its own regression test. @@ -49,7 +49,7 @@ PYTHONPATH=. python scripts/decrypt_master_key.py # ops recovery: decrypt+pr PYTHONPATH=. python scripts/encrypt_master_key.py # ops bootstrap: import an externally-generated xprv (--overwrite to replace) PYTHONPATH=. python scripts/electrum_smoke_test.py # manual check: connect, handshake, subscribe to headers, print the tip -python -m pytest # all 637 tests +python -m pytest # all 670 tests python -m pytest tests/unit/test_hd.py # one file python -m pytest tests/unit/test_hd.py::test_derivation_is_deterministic # one test ``` @@ -107,7 +107,7 @@ Source of truth: the `PalladiumWallet` repo — [ChainProfiles.cs](../PalladiumW | Min withdrawal | = current `bet_amount_sats` (no separate field) | `withdrawals/service.py` | | Min deposit | none | — | | Min password length | 8 | `auth/security.py:MIN_PASSWORD_LENGTH` | -| Confirmations, every tx kind | **1** | hardcoded in `tx/confirmation.py` | +| Confirmations | **1** for every platform-built tx (confirmation poller) and for the outputs it creates (bet/withdrawal change, payouts); **`deposit_confirmations` (6)** for a deposit, **120** for a coinbase output (B-82) | tx confirmation hardcoded in `tx/confirmation.py`; deposit threshold in `round_config` (`/admin`), coinbase maturity hardcoded in `deposits/service.py` | | Max inputs per *user* tx (bet, withdrawal) | 50 (`MAX_TX_INPUTS`, B-48) — over it the build fails with `too_many_inputs`, it never spends more | hardcoded in `wallet/psbt_builder.py` | | Max inputs per *payout* | 500 (`MAX_PAYOUT_TX_INPUTS`, B-52) — the pool holds one UTXO per bet, so reusing the user cap made any round past ~50 players unpayable | hardcoded in `wallet/psbt_builder.py` | | Max participants per round | 400 (`MAX_PARTICIPANTS_PER_ROUND`, B-52) — the 401st bet is refused with `round_full` *before* any money moves, so "a round can always be paid out" is an invariant rather than something discovered at payout time | hardcoded in `wallet/psbt_builder.py`, enforced in `bets/service.py` | @@ -116,7 +116,7 @@ Source of truth: the `PalladiumWallet` repo — [ChainProfiles.cs](../PalladiumW **Round cooldown** (`round_cooldown_seconds`, not in the original design): gap after a round closes before the next opens, so players can see the outcome. -**Maintenance pause** (`RoundConfig.paused`): toggled by `POST /admin/pause` / `POST /admin/resume` — a deliberate operator action with its own "Manutenzione" card in `/admin`, audit-logged `lottery_paused`/`lottery_resumed`, not a plain config field. It only stops the *next* round from opening (`rounds/service.py:open_new_round_if_needed`); a round in progress still closes, draws and pays its winner. Exposed as `lottery_paused` so `/` can show a banner, with `paused_reason` (B-78, DAT-69): `operator`, or an automatic reason the system wrote itself (`resurrected_tx` so far, `rounds/pause.py`) — an operator pause never overwrites an automatic one, and resume clears whichever it is. +**Maintenance pause** (`RoundConfig.paused`): toggled by `POST /admin/pause` / `POST /admin/resume` — a deliberate operator action with its own "Manutenzione" card in `/admin`, audit-logged `lottery_paused`/`lottery_resumed`, not a plain config field. It only stops the *next* round from opening (`rounds/service.py:open_new_round_if_needed`); a round in progress still closes, draws and pays its winner. Exposed as `lottery_paused` so `/` can show a banner, with `paused_reason` (B-78, DAT-69): `operator`, or an automatic reason the system wrote itself (`resurrected_tx`, `reorg`, `payout_blocked`, `rounds/pause.py`) — an operator pause never overwrites an automatic one, and resume clears whichever it is. **No `fee_address`, no rounds** (`rounds/service.py:rounds_can_open`, B-66): the payout pays the 30% commission to `fee_address`, which has no column default because an operator must set their own — so until they do, `open_new_round_if_needed` refuses to open a round at all. Otherwise every round took bets, confirmed them and only then discovered it was unpayable, wedging in `paying_out` with money already in the pool and needing manual recovery. Same scope as pausing: a round already in progress still closes, draws and pays out (clearing the address mid-round is exactly the operator slip that must not strand a live round). Surfaced as `lottery_configured` on `GET /rounds/current` — `/` shows a *different* banner from the maintenance one, since "come back later" would be false — and as a warning on `/admin`'s Parametri card, the one screen that can fix it. Anything else that would make a round unpayable belongs in `rounds_can_open` next to it, not discovered at payout time. Since B-79 (RND-40) that includes the amounts: `payout_config_problem(bet_amount_sats, fee_rate_sat_vb)` checks a round of 1 and of `MAX_PARTICIPANTS_PER_ROUND` participants against the winner's network fee, the bump reserve and dust outputs, and `PUT /admin/config` refuses a bet amount or fee rate that fails it with `422 config_not_payable` (the payout reads `fee_rate_sat_vb` live, so raising it mid-round is exactly what could otherwise strand the round in progress). @@ -159,7 +159,7 @@ One connection serves everything — deposit credits, broadcasts, confirmations, - **Bounded requests** (`_REQUEST_TIMEOUT_SECONDS` = 15s); a timeout tears the connection down. Unbounded waits used to hang `POST /bets` *while holding the per-user lock*, and could stall the confirmation poller permanently. - **The drop is observable**: `client.wait_closed()` resolves when the read loop dies, and `_run_once` races it against the notification consumers and a 60s `server.ping`. A server that keeps answering pings but stopped following the chain is caught too (B-86): at every ping, if no header has been *accepted* for 360s (`_STALL_SECONDS`), `_check_not_stalled` asks the other servers for their tip and ends the session with `ServerStalledError` if any is higher, so `run()` rotates; none higher means a quiet chain and the session goes on. Without it the listener sat on queues nobody would ever fill while `listener.client` still looked alive. - **Headers are validated, not trusted** (`_apply_header`, async since B-74): the tip never regresses, a header must meet the difficulty target it claims *and* that target must be no easier than `draw.POW_LIMIT` (PLM's powLimit; `tests/unit/conftest.py` swaps in a regtest limit so tests can mine headers, and the tests of the limit itself put `MAINNET_POW_LIMIT` back), and every advance must be linked to the tip: a single block by its `prev_hash`, a jump by downloading the headers in between (`blockchain.block.headers`, ≤2016 per request) from the active server and checking each. Failure raises `HeaderValidationError`, which ends the session like a dropped connection and rotates away — that header is the draw's only entropy, so a fabricated one picks the winner. The listener keeps the last 10 headers (`_window`, built from the first header plus its 9 predecessors); a header that does not join the tip is a reorg, walked back to its fork point inside the window and followed (`_follow_reorg`), a fork older than the window ends the session. A followed reorg writes `chain_reorg_detected` *before* the window moves (`_record_reorg`, B-85), and a depth of 2 or more pauses the lottery with `paused_reason = 'reorg'` in the same transaction; a header at the tip's own height with a different hash is a reorg too, not ignored. One more header is refused *without* ending the session (B-64): one carrying no `hex` at all (nothing to validate or draw from, and applying the height alone would break the `tip_height`/`tip_header_hex` pairing). `_run_once` separately refuses to publish the client while *no* tip is known, so the ignore-don't-kill choice can't reopen B-63. -- **A quorum corroborates every money-moving decision** (`_corroborate_majority`, 10s per server, asking only the *other* endpoints — never the active one, which is what a MITM controls): `corroborate_header` before a block seeds the draw (B-28), `corroborate_utxo_spent` before a UTXO missing from one `listunspent` is written off as externally spent (B-29), and `corroborate_utxo_credit` before a new outpoint credits a balance — same outpoint, same amount, confirmed (B-59). Balances move in both directions, so both directions need the same quorum — and since B-83 that includes `corroborate_utxo_reinstatement` before a UTXO written off as externally spent is put back (same outpoint, amount, confirmed; `deposits/service.py:find_reinstate_candidates` only reads, `reinstate_reappeared_utxos` writes what the quorum agreed on), and `confirmation_proven` before a tx is marked confirmed: `tx/confirmation.py:poll_once` takes the height the active server reports only as a *claim*, and the row confirms when a strict majority of at least 2 of the *other* servers return a `get_merkle` proof that recomputes to the merkle root of the block's header (`electrum/merkle.py`; the header is the tip we hold, or for an older block the one the quorum agrees on and that meets its own target). With no `network` nothing confirms. The one decision still on the active server's word is the pool-coin listing for a payout (B-82). It needs **at least 2 answers** and a strict majority of those that answered (B-84): fewer → returns **False**, since one voice decides nothing independently and an unreachable network proves nothing. Only `ALLOW_NO_QUORUM=true` keeps the old reduced rule (no other server → True, otherwise a majority of whoever answers). A failed credit corroboration only *delays*: `find_new_credit_candidates` re-offers the outpoint on the next refresh or `DepositReconciler` sweep. +- **A quorum corroborates every money-moving decision** (`_corroborate_majority`, 10s per server, asking only the *other* endpoints — never the active one, which is what a MITM controls): `corroborate_header` before a block seeds the draw (B-28), `corroborate_utxo_spent` before a UTXO missing from one `listunspent` is written off as externally spent (B-29), and `corroborate_utxo_credit` before a new outpoint credits a balance — same outpoint, same amount, confirmed (B-59). Balances move in both directions, so both directions need the same quorum — and since B-83 that includes `corroborate_utxo_reinstatement` before a UTXO written off as externally spent is put back (same outpoint, amount, confirmed; `deposits/service.py:find_reinstate_candidates` only reads, `reinstate_reappeared_utxos` writes what the quorum agreed on), and `confirmation_proven` before a tx is marked confirmed: `tx/confirmation.py:poll_once` takes the height the active server reports only as a *claim*, and the row confirms when a strict majority of at least 2 of the *other* servers return a `get_merkle` proof that recomputes to the merkle root of the block's header (`electrum/merkle.py`; the header is the tip we hold, or for an older block the one the quorum agrees on and that meets its own target). With no `network` nothing confirms. Since B-82 the pool-coin check before a payout is on the quorum too (`verify_pool_outpoints`, below). A quorum needs **at least 2 answers** and a strict majority of those that answered (B-84): fewer → returns **False**, since one voice decides nothing independently and an unreachable network proves nothing. Only `ALLOW_NO_QUORUM=true` keeps the old reduced rule (no other server → True, otherwise a majority of whoever answers). A failed credit corroboration only *delays*: `find_new_credit_candidates` re-offers the outpoint on the next refresh or `DepositReconciler` sweep. On reconnect `_subscribe_all_users` runs as its own task with bounded concurrency (`_RESUBSCRIBE_CONCURRENCY` = 20) rather than inline and serially — otherwise a large user base froze `tip_height`, and with it an in-flight draw, for the whole sweep (B-31); one user's failure is logged and skipped. `address_for_new_user` (called right after registration) is best-effort by design: on failure that address stays unsubscribed until the next reconnect or `DepositReconciler` sweep. @@ -169,7 +169,7 @@ Diagrams: [come-funziona.md](docs/come-funziona.md) (the 5 phases), [ciclo-di-vi **REG** — on signup the server derives a P2WPKH address via BIP84 (`m/84'/coin'/0'/0/index`, one index per user) from the encrypted master xprv. Permanent, and doubles as deposit address, winnings address and withdrawal change address. -**DEP** — the listener subscribes to the user's scripthash; balance is credited after **1 confirmation** and only once the other servers corroborate the outpoint and its amount (B-59), with the 1-conf reorg risk knowingly accepted and no rollback logic. `deposits/service.py` also detects UTXOs that vanished (spent outside the platform — corroborated per B-29 first) and *reinstates* ones that reappear. +**DEP** — the listener subscribes to the user's scripthash; balance is credited once the output reaches its threshold (B-82, DEP-03: **1** if a transaction of the platform created it — any txid of a `pending_transactions` row —, **120** for a coinbase, recognised from the raw tx that must hash to the txid, `ElectrumListener.is_coinbase`, and `round_config.deposit_confirmations`, default 6, for everything else) **and** only once the other servers corroborate the outpoint, its amount and a height that gives it that many confirmations on our tip (B-59, ELC-17). The deposit is the one input a user can double-spend, hence the 6. Outputs short of their threshold are *pending deposits* (`ElectrumListener._pending_deposits`, memory only): shown in `pending_balance_sats` (amber), never spendable, and since no scripthash notification announces a maturing deposit each new block refreshes only the users that have one (`_refresh_waiting_deposits`). A deposit that vanishes later takes the spent-outside path; there is no other rollback logic. `deposits/service.py` also detects UTXOs that vanished (spent outside the platform — corroborated per B-29 first) and *reinstates* ones that reappear. **PLAY** — fixed cost, **at most one active bet per user**, and **at most `MAX_PARTICIPANTS_PER_ROUND` (400) players per round** — past that the bet is refused with `round_full` and the player waits for the next round (B-52: the payout must spend one pool UTXO per bet, so a round is only ever allowed to grow to what a single payout tx can drain). PSBT user-address → pool-address, always with a **change output back to the same user address** of at least `DUST_LIMIT_SATS` — a user's balance must never exactly equal the bet, and since B-62 that's enforced (`balance_leaves_no_change`) rather than assumed. Fee ~1 sat/vB, **deducted from the bet amount**. No confirmation within the timeout → RBF bump and rebroadcast. @@ -177,7 +177,7 @@ Diagrams: [come-funziona.md](docs/come-funziona.md) (the 5 phases), [ciclo-di-vi - *Bet cutoff is the round's own deadline* (`round_deadline` = `opened_at + Round.duration_seconds`, the value snapshotted at open time — B-61), **not** the DB status: `place_bet` calls `rounds/service.round_accepts_bets`, which rejects once the deadline passes even while `status` is still `"open"` (the 5s scheduler tick can lag behind it). Once a round leaves `open`, no new bets either, and no new round opens until this one is fully `closed`. The deadline is checked twice — on arrival and again after the transaction is built — and the participant row is then committed behind a **compare-and-set on the round row** (`UPDATE rounds ... WHERE status = 'open'`, B-53): the scheduler flips `open` → `closing` in a transaction of its own and only counts in-flight bets afterwards, so without the CAS a bet could commit in between, be excluded from the draw (only `confirmed` participants are drawn) and still have its sats land in the pool with no refund path. Its mirror image on the scheduler side is `_close_and_draw` re-counting in-flight bets in the same session it snapshots the participants from. - *"Yellow light":* closing **waits for every already-broadcast bet to confirm** before drawing, so a bet in flight at the boundary isn't lost (`building` counts as in-flight; what bounds the wait is the reconciler eventually abandoning a bet that never confirms). - *Algorithm* (deliberately simple, meant to be replaced; **commit-reveal since B-73**, spec RND-14 … RND-20, RND-34 … RND-37): at open the round gets a 32-byte `server_seed` — stored only **Fernet-encrypted** with the master-xprv key (`server_seed_enc`; the plaintext column `server_seed` stays NULL until the reveal) — and publishes `server_seed_hash = sha256(seed)`. Entering `drawing` fixes, in one transaction with the status change, the confirmed participants **ordered by bet txid ascending** (public data, no ties — never `broadcast_at`), each one's `draw_position`, `participants_hash` (sha256 of `txid:vout:address\n` lines; `bet_vout` is the output paying the round's pool address), and `draw_height = tip_at_close + 1`, and writes `draw_committed`. The draw waits until the chain has `draw_confirmations` blocks from `draw_height` (snapshotted at open from `round_config`, default 3), then asks **every** server, the active one included, for the header *at that exact height* (`ElectrumListener.agreed_block_hash`: valid work, at least 2 other answers, strict majority of all answers) — no agreement means the same height is asked again, never another block, so whoever mines or relays blocks cannot pick the one that seeds the draw (`draw_header_corroboration_failed`, once per distinct hash the active server reported). `seed = sha256(server_seed ‖ block_hash ‖ participants_hash)` as a big-endian integer, `index = seed mod n`, winner = participant with that `draw_position`; equal probability for everyone, regardless of amount. The reveal (`server_seed` column + `winner_drawn` payload) happens in the same transaction that stores the winner (or when a round closes with no participants), and `GET /rounds/{id}/proof` (public) returns everything needed to recompute it. A tip of 0 at closing means *unknown*: the round stays `closing` (`draw_baseline_tip_unknown`, once per round, B-63) because `draw_height` must come from a height we actually knew. `draw_stalled` fires when the chain has not advanced for 360s while waiting for confirmations, or the quorum keeps failing to agree. `_complete_draw` reads everything it needs from the database (`_close_and_draw` enters `drawing` and then calls it, and since B-81 `_tick` calls it again for any round found in `drawing` — after an exception or a restart — so it is idempotent: a round no longer in `drawing`, or already holding a winner, is left alone); a round opened before B-73 (no `server_seed_enc`) is given a secret as it enters `drawing`. -- *Payout* is signed with the round's own pool key — every round has its own pool address `m/84'/746'/0'/2/`, stored in `rounds.pool_address` when the round opens (B-80; `derive_pool_address(round_id)`, `derive_round_pool_key`), and bets pay it, the payout reads and spends only its coins, and a withdrawal to it (or to the reserve address `1/0`, or to `fee_address`) is refused with `withdrawal_to_platform_address`. Rounds opened before B-80 keep the old shared `1/0` address as their pool; it is now the *reserve* address. The payout's **network fee comes out of the winner's 70%**; the platform's 30% pays the **bump reserve** (B-79, RND-38: always a third output, to the reserve address `1/0`, recorded in `rounds.bump_reserve_sats`) and the remainder goes to `fee_address`. The reserve is what makes the payout bumpable — it normally spends exactly the round's bet outputs and so has no change — and `bump_fee` shrinks it (`_find_reducible_output`: the reserve output for a payout, the user's change for a bet/withdrawal), never the winner's or the fee address's output. Same timeout → RBF → rebroadcast pattern. +- *Payout* is signed with the round's own pool key — every round has its own pool address `m/84'/746'/0'/2/`, stored in `rounds.pool_address` when the round opens (B-80; `derive_pool_address(round_id)`, `derive_round_pool_key`), and bets pay it, the payout reads and spends only its coins, and a withdrawal to it (or to the reserve address `1/0`, or to `fee_address`) is refused with `withdrawal_to_platform_address`. Rounds opened before B-80 keep the old shared `1/0` address as their pool; it is now the *reserve* address. The payout's **network fee comes out of the winner's 70%**; the platform's 30% pays the **bump reserve** (B-79, RND-38: always a third output, to the reserve address `1/0`, recorded in `rounds.bump_reserve_sats`) and the remainder goes to `fee_address`. The reserve is what makes the payout bumpable — it normally spends exactly the round's bet outputs and so has no change — and `bump_fee` shrinks it (`_find_reducible_output`: the reserve output for a payout, the user's change for a bet/withdrawal), never the winner's or the fee address's output. Same timeout → RBF → rebroadcast pattern. **Verified before it is built** (B-82, RND-22, RND-39, `_trigger_payout` → `_verify_round_pool` → `ElectrumListener.verify_pool_outpoints`): a round with its own pool address spends exactly its confirmed participants' bet outputs (`bet_txid:bet_vout`), never whatever else sits at the address, and first asks the *other* servers whether each is still there — a strict majority of at least 2 answers; a mempool-only coin or a tie concludes nothing (`payout_failed`, `pool verification unavailable`, retried). A participant whose bet is gone becomes `reorged_out`, `participant_reorged_out` is audit-logged, `pool_amount_sats` is rewritten (only ever downward) in the same transaction and the 70/30 split is computed on what is left. If the **winner's own** bet is gone nothing is built: `payout_blocked_at` is set, `payout_blocked` is logged and the lottery is paused with `paused_reason = 'payout_blocked'`; the winner's row stays `confirmed` so that after `POST /admin/resume` (which clears the mark) the verification really runs again. A round opened before B-80 (pool = the shared reserve address) keeps the old active-server `listunspent` choice. - *UI, two independent layers.* A generic phase box ("Pagamento al vincitore in corso…") shows to **every** viewer for the whole closing/drawing/paying_out span — pure cosmetic text driven by `status`. **Additively**, a personalized "Hai vinto!/Non hai vinto" box appears only where `user_played` is true (computed via `get_optional_user`, since the endpoint is reachable logged-out) — nobody else has anything to reveal. - *Reveal timing.* Delayed by at least `draw_animation_seconds`, anchored to the server's `closes_at` so a reload can't reset the countdown, and decoupled from the real (~block-time) wait for `winner_user_id`. Once revealed it's persisted in `localStorage.plm_persisted_result`, surviving the move to `closed` — at which point `get_active_round` stops returning the round and `winner_user_id` disappears from `GET /rounds/current`. `GET /users/me/last-round-result` is the durable DB-backed backstop for a device that missed the live window entirely. Full logic: `refreshRound`/`checkLastRoundResult` in `app/static/app.js`. @@ -251,7 +251,7 @@ Explicit design choices, not derivable from any single file — respect them: - Keys are generated and held **server-side**: this is **custodial**. The user controls nothing until they withdraw. - The deposit address *is* the winnings address — there is no separate "winner address". -- **1 confirmation** for every tx kind. Don't introduce differing thresholds (3, 6, …) without an explicit decision. +- **1 confirmation** for every transaction the platform builds (bets, withdrawals, payouts, and the outputs they create). Deposits (`deposit_confirmations`, 6) and coinbase outputs (120) are the explicit exceptions of B-82 — don't add other thresholds without an explicit decision. - The draw algorithm is a **replaceable component**, not the final design — don't architect around its current form. - `GET /admin/users/{id}/privkey` exporting a raw WIF is **intentional**, not a vulnerability: the server already holds the master key, so this only exposes via API what an operator could script anyway. Every access writes `admin_privkey_accessed` — don't remove that logging. - **Usernames are case-insensitive** (B-57): one namespace, enforced by a unique index on `lower(username)` (`app/db/models.py`) and matched with `func.lower(...)` on both register and login. The name is still *stored* as typed — that's what `/admin` and the audit log show. The migration refuses to run if two existing accounts differ only by case, rather than guessing which one to rename: both may hold funds. diff --git a/README.md b/README.md index fff36ba..0ff9cb8 100644 --- a/README.md +++ b/README.md @@ -62,7 +62,7 @@ python -m pytest # all tests python -m pytest tests/unit/test_hd.py # one file ``` -637 unit tests cover HD derivation, PSBT building, the Electrum client, bets, +670 unit tests cover HD derivation, PSBT building, the Electrum client, bets, deposits, withdrawals, the round/draw engine, RBF fee-bumping, admin config, the pending-inclusive balance calculation, and the SSE push channel. No automated integration tests against a live Electrum connection — mainnet diff --git a/app/api/routes/admin.py b/app/api/routes/admin.py index 8aafb74..7f2ceb8 100644 --- a/app/api/routes/admin.py +++ b/app/api/routes/admin.py @@ -3,7 +3,7 @@ import secrets from fastapi import APIRouter, Depends, Header, HTTPException, Query, status from pydantic import BaseModel, Field, field_validator -from sqlalchemy import select +from sqlalchemy import select, update from sqlalchemy.ext.asyncio import AsyncSession from app.api.errors import http_error @@ -47,6 +47,7 @@ _CONFIG_FIELDS = ( "fee_rate_sat_vb", "rbf_timeout_seconds", "draw_animation_seconds", + "deposit_confirmations", # B-82 (DAT-62) "draw_confirmations", # B-73 (DAT-62) ) @@ -59,6 +60,7 @@ class RoundConfigResponse(BaseModel): fee_rate_sat_vb: int rbf_timeout_seconds: int draw_animation_seconds: int + deposit_confirmations: int # B-82 (DAT-62) draw_confirmations: int # B-73 (DAT-62) paused: bool paused_reason: str | None = None # B-78 (DAT-69): why, or None when not paused @@ -81,6 +83,7 @@ class RoundConfigUpdate(BaseModel): fee_rate_sat_vb: int | None = Field(default=None, ge=1, le=MAX_FEE_RATE_SAT_VB) rbf_timeout_seconds: int | None = Field(default=None, ge=60, le=7 * 24 * 3600) draw_animation_seconds: int | None = Field(default=None, ge=0, le=600) + deposit_confirmations: int | None = Field(default=None, ge=1, le=100) # B-82 (DAT-62) draw_confirmations: int | None = Field(default=None, ge=1, le=20) # B-73 (DAT-62) @field_validator("fee_address") @@ -186,6 +189,9 @@ async def resume_lottery(session: AsyncSession = Depends(get_session)) -> RoundC lifted = config.paused_reason if config.paused else None config.paused = False config.paused_reason = None + # B-82 (DAT-68, API-32): a payout held because the winner's bet left the chain is released + # for a new attempt, which verifies the coins again, in the same transaction as the resume. + await session.execute(update(Round).where(Round.payout_blocked_at.is_not(None)).values(payout_blocked_at=None)) await write_audit_log(session, "lottery_resumed", {"reason": lifted}) await session.commit() return _config_response(config) @@ -287,6 +293,7 @@ class AdminRoundResponse(BaseModel): server_seed_hash: str | None # B-73 (API-36): the commitment only — never the secret draw_height: int | None # B-73 bump_reserve_sats: int | None # B-79 (API-36) + payout_blocked_at: str | None # B-82 (API-36, DAT-68) @router.get("/rounds", response_model=list[AdminRoundResponse], dependencies=[Depends(require_admin)]) @@ -318,6 +325,7 @@ async def list_rounds( server_seed_hash=r.server_seed_hash, draw_height=r.draw_height, bump_reserve_sats=r.bump_reserve_sats, + payout_blocked_at=isoformat_utc(r.payout_blocked_at), ) for r in rounds ] diff --git a/app/api/routes/users.py b/app/api/routes/users.py index 8a84531..6355cae 100644 --- a/app/api/routes/users.py +++ b/app/api/routes/users.py @@ -1,4 +1,4 @@ -from fastapi import APIRouter, Depends, status +from fastapi import APIRouter, Depends, Request, status from pydantic import BaseModel from sqlalchemy import select from sqlalchemy.ext.asyncio import AsyncSession @@ -31,10 +31,15 @@ class MeResponse(BaseModel): @router.get("/me", response_model=MeResponse) async def me( + request: Request, user: User = Depends(get_current_user), session: AsyncSession = Depends(get_session), ) -> MeResponse: - pending_balance_sats, has_pending = await compute_pending_balance(session, user) + # B-82 (DEP-12 (b)): deposits confirmed but short of their threshold, kept in memory by + # the listener (absent in an app without one, e.g. a test of this route alone). + listener = getattr(request.app.state, "electrum_listener", None) + waiting_sats = listener.pending_deposit_sats(user.id) if listener is not None else 0 + pending_balance_sats, has_pending = await compute_pending_balance(session, user, waiting_sats) return MeResponse( id=user.id, username=user.username, diff --git a/app/db/models.py b/app/db/models.py index a317d26..1e5b872 100644 --- a/app/db/models.py +++ b/app/db/models.py @@ -114,6 +114,9 @@ class Round(Base): # the reserve address and is what an RBF bump shrinks, so this is never rewritten by a bump). bump_reserve_sats: Mapped[int | None] = mapped_column(BigInteger, default=None) payout_txid: Mapped[str | None] = mapped_column(String(64), default=None) + # B-82 (DAT-68): set when the verification before the payout finds the *winner's* bet gone + # from the chain; while it is set no payout is built. Cleared by POST /admin/resume. + payout_blocked_at: Mapped[datetime | None] = mapped_column(default=None) # B-73 (DAT-61, DAT-64, DAT-65): the commit-reveal draw. `server_seed_enc` is the round's # 32-byte secret Fernet-encrypted with the master-xprv key, written at open together with # `server_seed_hash` (sha256 of the secret, published from then on); the plaintext @@ -184,6 +187,9 @@ class RoundConfig(Base): # of that block is read; copied onto each round when it opens (DAT-61), so an edit # applies from the next round. draw_confirmations: Mapped[int] = mapped_column(default=3, server_default="3") + # B-82 (DAT-62, DEP-03): confirmations after which an output paying a user is credited + # as a deposit (platform transactions keep 1, coinbase outputs 120). + deposit_confirmations: Mapped[int] = mapped_column(default=6, server_default="6") fee_rate_sat_vb: Mapped[int] = mapped_column(default=1) rbf_timeout_seconds: Mapped[int] = mapped_column(default=900) # Maintenance switch: when true, the round currently in progress still runs to diff --git a/app/deposits/service.py b/app/deposits/service.py index e34d6e5..2ca337e 100644 --- a/app/deposits/service.py +++ b/app/deposits/service.py @@ -1,17 +1,33 @@ import logging +from collections.abc import Awaitable, Callable +from dataclasses import dataclass, field from sqlalchemy import select from sqlalchemy.ext.asyncio import AsyncSession from app.audit.log import write_audit_log from app.db.base import begin_immediate -from app.db.models import UtxoEvent +from app.db.models import PendingTransaction, PendingTransactionTxid, RoundConfig, UtxoEvent from app.rounds.events import broadcaster from app.wallet.balance import recompute_balance from app.wallet.psbt_builder import DUST_LIMIT_SATS logger = logging.getLogger(__name__) +# B-82 (DEP-03): confirmations a coinbase output needs before it can be spent, PLM's +# COINBASE_MATURITY (palladiumcore/src/consensus/consensus.h). A coinbase paying a user's +# address is credited no earlier than this, whatever deposit_confirmations says. +COINBASE_MATURITY = 120 + +# Confirmations an output created by a transaction of the platform itself — the change of +# a bet or a withdrawal, a payout — needs: the platform built it, nobody else could have +# double-spent its inputs (DEP-03). +PLATFORM_TRANSACTION_CONFIRMATIONS = 1 + +# Default of round_config.deposit_confirmations, used only when the config row does not +# exist yet (get_round_config creates it with the same value). +DEFAULT_DEPOSIT_CONFIRMATIONS = 6 + async def find_new_credit_candidates(session: AsyncSession, user_id: int, entries: list[dict]) -> list[dict]: """The subset of `entries` that would actually credit something: confirmed @@ -41,6 +57,74 @@ async def find_new_credit_candidates(session: AsyncSession, user_id: int, entrie ] +@dataclass +class DepositPolicy: + """What decides how many confirmations an incoming output needs (DEP-03), read from the + database before any network call: the configured deposit threshold and which of the + transactions involved are the platform's own.""" + + deposit_confirmations: int = DEFAULT_DEPOSIT_CONFIRMATIONS + platform_txids: set[str] = field(default_factory=set) + + +async def read_deposit_policy(session: AsyncSession, txids: set[str]) -> DepositPolicy: + """The deposit threshold and, among `txids`, the ones created by a platform transaction: + any txid of a `pending_transactions` row, replaced RBF versions included (B-77).""" + configured = await session.scalar(select(RoundConfig.deposit_confirmations)) + platform: set[str] = set() + if txids: + platform |= set( + (await session.scalars(select(PendingTransactionTxid.txid).where(PendingTransactionTxid.txid.in_(txids)))) + ) + platform |= set( + (await session.scalars(select(PendingTransaction.current_txid).where(PendingTransaction.current_txid.in_(txids)))) + ) + return DepositPolicy( + deposit_confirmations=configured if configured else DEFAULT_DEPOSIT_CONFIRMATIONS, platform_txids=platform + ) + + +async def classify_new_deposits( + new_entries: list[dict], + tip_height: int, + policy: DepositPolicy, + is_coinbase: Callable[[str], Awaitable[bool | None]], +) -> tuple[list[tuple[dict, int]], list[dict]]: + """Split the new, confirmed, non-dust entries of `find_new_credit_candidates` into the + ones that may now be credited — as `(entry, required_confirmations)`, the number the + quorum is then asked to confirm (ELC-17) — and the ones still **waiting** for + confirmations (B-82, DEP-03, DEP-14). + + Confirmations are `tip_height - height + 1`. The threshold is 1 for an output of a + platform transaction, 120 (`COINBASE_MATURITY`) for a coinbase output, and + `deposit_confirmations` for anything else; a coinbase is only looked for once the plain + threshold is met, since below it the output waits anyway. `is_coinbase` answers from the + raw transaction (hash-checked by the caller) and may return None when it cannot tell, in + which case the output waits and is asked again at the next refresh. No database session + is involved: this runs between the read and the write phases, where the network is used. + """ + candidates: list[tuple[dict, int]] = [] + waiting: list[dict] = [] + for entry in new_entries: + confirmations = tip_height - entry["height"] + 1 + if entry["tx_hash"] in policy.platform_txids: + required = PLATFORM_TRANSACTION_CONFIRMATIONS + else: + required = policy.deposit_confirmations + if required <= confirmations < COINBASE_MATURITY: + # Plain threshold met, maturity not: only now is it worth knowing whether + # this is a coinbase output (which would still be immature). + coinbase = await is_coinbase(entry["tx_hash"]) + if coinbase is None or coinbase: + waiting.append(entry) + continue + if confirmations < required: + waiting.append(entry) + continue + candidates.append((entry, required)) + return candidates, waiting + + async def credit_confirmed_utxos(session: AsyncSession, user_id: int, entries: list[dict]) -> int: """Insert utxo_events for newly-confirmed entries from an Electrum `listunspent` response (idempotent on txid+vout), refresh the user's cached diff --git a/app/electrum/listener.py b/app/electrum/listener.py index a21e466..23e8f89 100644 --- a/app/electrum/listener.py +++ b/app/electrum/listener.py @@ -12,11 +12,13 @@ from app.audit.log import write_audit_log from app.db.base import begin_immediate from app.db.models import User from app.deposits.service import ( + classify_new_deposits, credit_confirmed_utxos, find_new_credit_candidates, find_reinstate_candidates, find_utxos_missing_from, mark_utxos_spent_externally, + read_deposit_policy, reinstate_reappeared_utxos, ) from app.config import MIN_QUORUM_RESPONSES @@ -120,6 +122,15 @@ class ElectrumListener: # B-86: when the last header was accepted (monotonic seconds), None until one is. self._last_header_at: float | None = None self.client: ElectrumClient | None = None + # B-82 (DEP-14): per user, the deposits seen at the last refresh that are confirmed + # but still short of their threshold — what the displayed balance shows as pending. + # Memory only: it moves no money and is rebuilt by the first refresh after a restart. + self._pending_deposits: dict[int, list[dict]] = {} + # txid -> is it a coinbase (DEP-03). A transaction never changes, so this is safe to + # keep; bounded so a long-lived process does not grow it without limit. + self._coinbase_cache: dict[str, bool] = {} + # Users with a refresh already running because of a new block (see _consume_headers). + self._block_refreshes: set[int] = set() @property def current_endpoint(self) -> ElectrumEndpoint | None: @@ -819,7 +830,9 @@ class ElectrumListener: return await self._corroborate_majority(_ask, lambda agrees: agrees, f"outpoint {txid}:{vout}") - async def corroborate_utxo_credit(self, scripthash: str, txid: str, vout: int, value: int) -> bool: + async def corroborate_utxo_credit( + self, scripthash: str, txid: str, vout: int, value: int, required_confirmations: int | None = None + ) -> bool: """B-59: the mirror of corroborate_utxo_spent, for money going the other way. A candidate external spend could not reduce a balance without a quorum, but `value` and `height` for a *credit* were taken from the single @@ -841,8 +854,11 @@ class ElectrumListener: only credited once the quorum agrees. """ + # B-82 (ELC-17): the other server must also place the coin deep enough — at a height + # that gives it `required_confirmations` counted on *our* tip, not on its own. + max_height = None if required_confirmations is None else self.tip_height - required_confirmations + 1 return await self._corroborate_unspent_confirmed( - scripthash, txid, vout, value, f"credit of {value} sats at {txid}:{vout}" + scripthash, txid, vout, value, f"credit of {value} sats at {txid}:{vout}", max_height=max_height ) async def corroborate_utxo_reinstatement(self, scripthash: str, txid: str, vout: int, value: int) -> bool: @@ -856,10 +872,11 @@ class ElectrumListener: ) async def _corroborate_unspent_confirmed( - self, scripthash: str, txid: str, vout: int, value: int, description: str + self, scripthash: str, txid: str, vout: int, value: int, description: str, max_height: int | None = None ) -> bool: """Do the other servers list this outpoint as unspent, for the same amount, and - confirmed (height > 0)? Shared by credits (B-59) and reinstatements (B-83).""" + confirmed (height > 0, and no higher than `max_height` when one is given)? Shared by + credits (B-59, B-82) and reinstatements (B-83).""" async def _ask(endpoint: ElectrumEndpoint) -> bool | None: client = self._client_factory(endpoint) @@ -873,6 +890,7 @@ class ElectrumListener: and e.get("tx_pos") == vout and e.get("value") == value and (e.get("height") or 0) > 0 + and (max_height is None or (e.get("height") or 0) <= max_height) for e in entries ) except Exception: @@ -882,6 +900,59 @@ class ElectrumListener: return await self._corroborate_majority(_ask, lambda agrees: agrees, description) + async def verify_pool_outpoints( + self, scripthash: str, outpoints: list[tuple[str, int, int]] + ) -> dict[tuple[str, int], bool] | None: + """Which of a round's bet outputs (`(txid, vout, amount)`) are really there, at the + round's pool address, according to the *other* servers (B-82, ELC-27, RND-39). + + Every other server lists the address' unspent coins; each answer says, per outpoint, + *present* (listed, same amount, confirmed), *absent* (not listed, or listed with + another amount) or neither (listed but unconfirmed — a block that left the chain + may well come back, which is no reason to throw a bet out). An outpoint is present + or absent when a strict majority of the answers say so. The result maps every + outpoint to True (present) or False (absent); it is None — nothing concluded — when + fewer than the quorum minimum (ELC-14) of servers answered or an outpoint has no + majority either way, so a participant is only ever excluded on the quorum's word and + the caller simply tries again later. With ALLOW_NO_QUORUM=true and no other server + configured the active server answers instead (development only).""" + + async def _ask(client: ElectrumClient) -> dict[tuple[str, int], tuple[int, int]] | None: + entries = await client.listunspent(scripthash) + return {(e["tx_hash"], e["tx_pos"]): (e["value"], e.get("height") or 0) for e in entries} + + answers = [a for a in await self._ask_other_servers(_ask) if a is not None] + if not self._other_endpoints() and self._allow_no_quorum: + answers = [a for a in [await self._ask_active_server(_ask)] if a is not None] + elif len(answers) < max(self._minimum_other_answers(), 1): + logger.warning( + "could not verify the pool coins at %s: %s answer(s) from the other servers, %s needed", + scripthash, + len(answers), + max(self._minimum_other_answers(), 1), + ) + return None + if not answers: + return None + + result: dict[tuple[str, int], bool] = {} + for txid, vout, amount in outpoints: + present = absent = 0 + for answer in answers: + listed = answer.get((txid, vout)) + if listed is None or listed[0] != amount: + absent += 1 + elif listed[1] > 0: + present += 1 + if present * 2 > len(answers): + result[(txid, vout)] = True + elif absent * 2 > len(answers): + result[(txid, vout)] = False + else: + logger.warning("the servers do not agree whether %s:%s is still on the chain", txid, vout) + return None + return result + # --- Confirmations proven by merkle branch (B-83, TX-22, ELC-26) --------------------- async def _header_at(self, height: int) -> str | None: @@ -942,6 +1013,63 @@ class ElectrumListener: # (rounds/scheduler.py:_wait_for_next_block) — nudge dashboards to # refetch instead of waiting for their next poll. broadcaster.publish() + self._refresh_waiting_deposits() + + def _refresh_waiting_deposits(self) -> None: + """A deposit that is waiting for confirmations (DEP-14) does not change its address' + status when the next block arrives — the history is the same, only the tip moved — so + no scripthash notification will ever announce that it has matured. Refresh the users + that have one at each new block, one task per user at a time, instead of leaving it to + the 300s sweep (B-82, DEP-09).""" + if self.client is None: + return + scripthash_of = {user_id: scripthash for scripthash, user_id in self._scripthash_to_user.items()} + for user_id, waiting in list(self._pending_deposits.items()): + scripthash = scripthash_of.get(user_id) + if not waiting or scripthash is None or user_id in self._block_refreshes: + continue + self._block_refreshes.add(user_id) + task = asyncio.create_task(self._refresh_after_block(user_id, scripthash)) + self._background_tasks.add(task) + task.add_done_callback(self._background_tasks.discard) + + async def _refresh_after_block(self, user_id: int, scripthash: str) -> None: + try: + await self.refresh_user(user_id, scripthash) + except Exception: + logger.exception("refreshing user_id=%s after a new block failed", user_id) + finally: + self._block_refreshes.discard(user_id) + + def pending_deposit_sats(self, user_id: int) -> int: + """Total of the user's deposits that are confirmed but short of their threshold + (DEP-12 (b), DEP-14); 0 when there are none or nothing has been refreshed yet.""" + return sum(entry["value"] for entry in self._pending_deposits.get(user_id, [])) + + async def is_coinbase(self, txid: str) -> bool | None: + """Is `txid` a coinbase transaction (DEP-03)? Read from the raw transaction, which is + accepted only if it hashes to `txid` — so the server cannot misdescribe it — and + whose single input has the null outpoint. None when it could not be fetched or did + not match; the caller then simply asks again at the next refresh.""" + cached = self._coinbase_cache.get(txid) + if cached is not None: + return cached + if self.client is None: + return None + try: + raw = await self.client.get_transaction(txid, verbose=False) + tx = Transaction.parse(bytes.fromhex(raw)) + except Exception: + logger.warning("could not read transaction %s to tell whether it is a coinbase", txid) + return None + if tx.txid().hex() != txid: + logger.warning("transaction %s: the server returned one that hashes to %s", txid, tx.txid().hex()) + return None + result = len(tx.vin) == 1 and tx.vin[0].txid == bytes(32) and tx.vin[0].vout == 0xFFFFFFFF + if len(self._coinbase_cache) >= 10_000: + self._coinbase_cache.clear() + self._coinbase_cache[txid] = result + return result async def _consume_scripthash(self, queue: asyncio.Queue) -> None: while True: @@ -960,7 +1088,8 @@ class ElectrumListener: entries = await self.client.listunspent(scripthash) async with self._session_factory() as session: - credit_candidates = await find_new_credit_candidates(session, user_id, entries) + new_entries = await find_new_credit_candidates(session, user_id, entries) + policy = await read_deposit_policy(session, {entry["tx_hash"] for entry in new_entries}) # B-83 (DEP-02): candidates only — nothing is written before the quorum agrees. reinstate_candidates = await find_reinstate_candidates(session, user_id, entries) candidates = [ @@ -968,11 +1097,20 @@ class ElectrumListener: for row in await find_utxos_missing_from(session, user_id, entries) ] + # B-82 (DEP-03, DEP-14): which of the new outputs have reached their threshold; the + # others are remembered as pending deposits (memory only) and shown as such. + credit_candidates, waiting_deposits = await classify_new_deposits( + new_entries, self.tip_height, policy, self.is_coinbase + ) + self._pending_deposits[user_id] = waiting_deposits + if not waiting_deposits: + self._pending_deposits.pop(user_id, None) + corroborated_credits = [ entry - for entry in credit_candidates + for entry, required in credit_candidates if await self.corroborate_utxo_credit( - scripthash, entry["tx_hash"], entry["tx_pos"], entry["value"] + scripthash, entry["tx_hash"], entry["tx_pos"], entry["value"], required_confirmations=required ) ] diff --git a/app/rounds/scheduler.py b/app/rounds/scheduler.py index 453792f..80681ee 100644 --- a/app/rounds/scheduler.py +++ b/app/rounds/scheduler.py @@ -22,6 +22,7 @@ from app.rounds.draw import ( server_seed_commitment, ) from app.rounds.events import broadcaster +from app.rounds.pause import pause_automatically from app.rounds.service import open_new_round_if_needed, round_deadline, winner_share from app.tx.outcome import BroadcastOutcome, broadcast_with_verdict from app.tx.versions import record_version @@ -437,6 +438,8 @@ class RoundScheduler: either way it's due immediately. """ async with self._session_factory() as session: + if await session.scalar(select(Round.payout_blocked_at).where(Round.id == round_id)) is not None: + return # B-82 (DAT-68): held until an operator resumes the lottery last_failure_at = await session.scalar( select(AuditLog.created_at) .where(AuditLog.event_type == "payout_failed", AuditLog.round_id == round_id) @@ -477,6 +480,11 @@ class RoundScheduler: logger.warning("round %s payout deferred: database busy", round_id) return round_ = await session.get(Round, round_id) + if round_.payout_blocked_at is not None: + # B-82 (DAT-68): held since the winner's own bet left the chain. Nothing is built + # until an operator resumes the lottery, which clears the mark (and the next + # attempt verifies the coins again). + return already_in_flight = await session.scalar( select(PendingTransaction).where( PendingTransaction.round_id == round_id, @@ -504,6 +512,18 @@ class RoundScheduler: winner_user_id = round_.winner_user_id winner = await session.get(User, winner_user_id) winner_address = winner.address if winner is not None else None + # B-82 (RND-22, RND-39): the bets the payout may spend and must first verify — the + # round's confirmed participants, each with the output that pays the pool. + bet_rows = [ + (p.id, p.user_id, p.bet_txid, p.bet_vout, p.bet_amount_sats) + for p in ( + await session.scalars( + select(RoundParticipant) + .where(RoundParticipant.round_id == round_id, RoundParticipant.status == "confirmed") + .order_by(RoundParticipant.draw_position) + ) + ).all() + ] await session.commit() # get_round_config may have created the row if not fee_address: @@ -517,8 +537,10 @@ class RoundScheduler: await self._log_payout_failure(round_id, winner_user_id, "winner user not found") return - winner_sats = winner_share(pool_amount_sats) - commission_share = pool_amount_sats - winner_sats # remainder from rounding goes to fees + # A round opened before the per-round pool address (B-80) pays from the shared reserve + # address, whose coins are not the bets of one round: it keeps the old way of choosing + # them. Every other round is verified and paid from its own bets (RND-22, RND-39). + legacy_pool = round_pool_address == derive_pool_address() # --- Phase 2: build (network read only, no DB write yet) ----------------- try: @@ -528,12 +550,33 @@ class RoundScheduler: pool_key = derive_round_pool_key(round_id, round_pool_address) pool_script_obj = script.p2wpkh(pool_key.to_public()) pool_address = pool_script_obj.address(network=PLM_MAINNET) - entries = await client.listunspent(address_to_scripthash(pool_address)) - utxos = [ - Utxo(e["tx_hash"], e["tx_pos"], e["value"]) - for e in entries - if e["height"] > 0 and (e["tx_hash"], e["tx_pos"]) not in reserved_outpoints - ] + if not legacy_pool: + # B-82 (RND-39): with the quorum, before anything is built — bets that left the + # chain drop out and the split is computed on what is really there. + verified = await self._verify_round_pool( + round_id, pool_address, winner_user_id, bet_rows, pool_amount_sats + ) + if verified is None: + return # held, unverifiable or database busy — already recorded + bet_rows, pool_amount_sats = verified + winner_sats = winner_share(pool_amount_sats) + commission_share = pool_amount_sats - winner_sats # remainder from rounding goes to fees + if legacy_pool: + entries = await client.listunspent(address_to_scripthash(pool_address)) + utxos = [ + Utxo(e["tx_hash"], e["tx_pos"], e["value"]) + for e in entries + if e["height"] > 0 and (e["tx_hash"], e["tx_pos"]) not in reserved_outpoints + ] + else: + # B-82 (RND-22): exactly the verified bet outputs of this round, nothing else + # that happens to sit at the address, and none already claimed by an unresolved + # payout. + utxos = [ + Utxo(txid, vout, amount) + for _id, _user_id, txid, vout, amount in bet_rows + if (txid, vout) not in reserved_outpoints + ] built = build_payout_transaction( signing_key=pool_key, @@ -634,6 +677,96 @@ class RoundScheduler: logger.info("round %s payout broadcast: txid=%s", round_id, built.txid) + async def _verify_round_pool( + self, + round_id: int, + pool_address: str, + winner_user_id: int | None, + bet_rows: list[tuple[int, int, str, int, int]], + pool_amount_sats: int | None, + ) -> tuple[list[tuple[int, int, str, int, int]], int] | None: + """B-82 (RND-39, DAT-67, DAT-68, ELC-27): before a payout is built, check with the quorum + that each confirmed participant's bet output is still at the round's pool address. + + Returns the participants that are really there with the pool amount recomputed on them + (the same pair when nothing was missing), or None when this attempt must stop: the + quorum could not be reached (`payout_failed`, `pool verification unavailable`), the + database was busy, or the **winner's own bet is gone** — then the payout is held + (`payout_blocked`, the lottery pauses) and only an operator's resume tries again. + + Participants whose bet is gone become `reorged_out` and leave the pool, one + `participant_reorged_out` row each, in one transaction with the rewritten + `pool_amount_sats` (which can only go down). The winner's own row is left `confirmed` + in the held case, precisely so that the verification repeats after the resume.""" + verdicts = await self._listener.verify_pool_outpoints( + address_to_scripthash(pool_address), [(txid, vout, amount) for _i, _u, txid, vout, amount in bet_rows] + ) + if verdicts is None: + logger.error("round %s payout deferred: the pool coins could not be verified with the quorum", round_id) + await self._log_payout_failure(round_id, winner_user_id, "pool verification unavailable") + return None + gone = [row for row in bet_rows if not verdicts[(row[2], row[3])]] + if not gone: + return bet_rows, pool_amount_sats + + winner_gone = next((row for row in gone if row[1] == winner_user_id), None) + async with self._session_factory() as session: + try: + await begin_immediate(session) # B-88 + except OperationalError: + logger.warning("round %s payout deferred: database busy while excluding reorged-out bets", round_id) + return None + round_ = await session.get(Round, round_id) + excluded = [row for row in gone if row is not winner_gone] + for participant_id, user_id, txid, vout, amount in excluded: + participant = await session.get(RoundParticipant, participant_id) + if participant is None or participant.status != "confirmed": + continue + participant.status = "reorged_out" + await session.flush() + remaining = ( + await session.scalars( + select(RoundParticipant).where( + RoundParticipant.round_id == round_id, RoundParticipant.status == "confirmed" + ) + ) + ).all() + new_pool = sum(p.bet_amount_sats for p in remaining) + if pool_amount_sats is not None: + new_pool = min(new_pool, pool_amount_sats) # DAT-67: only ever reduced + round_.pool_amount_sats = new_pool + for _participant_id, user_id, txid, vout, amount in excluded: + await write_audit_log( + session, + "participant_reorged_out", + {"txid": txid, "vout": vout, "amount_sats": amount, "pool_amount_sats": new_pool}, + user_id=user_id, + round_id=round_id, + ) + if winner_gone is not None: + _pid, user_id, txid, vout, amount = winner_gone + round_.payout_blocked_at = datetime.now(timezone.utc) + await write_audit_log( + session, + "payout_blocked", + {"txid": txid, "vout": vout, "amount_sats": amount}, + user_id=user_id, + round_id=round_id, + ) + logger.error( + "round %s payout held: the winner's bet %s:%s is no longer on the chain; lottery paused", + round_id, + txid, + vout, + ) + await pause_automatically(session, "payout_blocked") # same transaction (AUD-44) + await session.commit() + broadcaster.publish() + if winner_gone is not None: + return None + gone_ids = {row[0] for row in gone} + return [row for row in bet_rows if row[0] not in gone_ids], new_pool + async def _log_payout_failure(self, round_id: int, winner_user_id: int | None, reason: str) -> None: """Leaves an operator-visible trace in the audit log for a round stuck in "paying_out" — the logs alone don't show up in /admin. Called from every diff --git a/app/static/admin.html b/app/static/admin.html index f772d98..aa6979d 100644 --- a/app/static/admin.html +++ b/app/static/admin.html @@ -85,6 +85,8 @@ + + diff --git a/app/static/admin.js b/app/static/admin.js index 90b29e0..54c7bb7 100644 --- a/app/static/admin.js +++ b/app/static/admin.js @@ -151,6 +151,7 @@ async function adminLoadConfig() { document.getElementById('admin-round-cooldown').value = data.round_cooldown_seconds; document.getElementById('admin-draw-animation').value = data.draw_animation_seconds; document.getElementById('admin-draw-confirmations').value = data.draw_confirmations; + document.getElementById('admin-deposit-confirmations').value = data.deposit_confirmations; document.getElementById('admin-fee-rate').value = data.fee_rate_sat_vb; document.getElementById('admin-rbf-timeout').value = data.rbf_timeout_seconds; renderMaintenanceState(data.paused); @@ -211,6 +212,7 @@ async function adminSave() { round_cooldown_seconds: parseInt(document.getElementById('admin-round-cooldown').value, 10), draw_animation_seconds: parseInt(document.getElementById('admin-draw-animation').value, 10), draw_confirmations: parseInt(document.getElementById('admin-draw-confirmations').value, 10), + deposit_confirmations: parseInt(document.getElementById('admin-deposit-confirmations').value, 10), fee_rate_sat_vb: parseInt(document.getElementById('admin-fee-rate').value, 10), rbf_timeout_seconds: parseInt(document.getElementById('admin-rbf-timeout').value, 10), }; diff --git a/app/static/i18n.js b/app/static/i18n.js index c9fe181..30c37c4 100644 --- a/app/static/i18n.js +++ b/app/static/i18n.js @@ -87,7 +87,7 @@ const TRANSLATIONS = { 'result.lose': 'Not a win this time.', 'deposit.balanceTitle': 'Internal balance', - 'deposit.balanceHint': 'Updated after 1 network confirmation', + 'deposit.balanceHint': 'Updated once the network has confirmed your deposit (a few blocks)', 'deposit.refreshBtn': 'Refresh', 'deposit.refreshAria': 'Refresh balance', 'deposit.addressTitle': 'Deposit address', @@ -258,7 +258,7 @@ const TRANSLATIONS = { 'result.lose': 'Non hai vinto questa volta.', 'deposit.balanceTitle': 'Saldo interno', - 'deposit.balanceHint': 'Aggiornato dopo 1 conferma sulla rete', + 'deposit.balanceHint': 'Aggiornato quando la rete ha confermato il deposito (alcuni blocchi)', 'deposit.refreshBtn': 'Aggiorna', 'deposit.refreshAria': 'Aggiorna saldo', 'deposit.addressTitle': 'Indirizzo di deposito', @@ -426,7 +426,7 @@ const TRANSLATIONS = { 'result.lose': 'Esta vez no has ganado.', 'deposit.balanceTitle': 'Saldo interno', - 'deposit.balanceHint': 'Actualizado tras 1 confirmación en la red', + 'deposit.balanceHint': 'Se actualiza cuando la red confirma tu depósito (unos bloques)', 'deposit.refreshBtn': 'Actualizar', 'deposit.refreshAria': 'Actualizar saldo', 'deposit.addressTitle': 'Dirección de depósito', @@ -594,7 +594,7 @@ const TRANSLATIONS = { 'result.lose': "Pas de gain cette fois-ci.", 'deposit.balanceTitle': 'Solde interne', - 'deposit.balanceHint': 'Mis à jour après 1 confirmation sur le réseau', + 'deposit.balanceHint': 'Mis à jour une fois le dépôt confirmé par le réseau (quelques blocs)', 'deposit.refreshBtn': 'Actualiser', 'deposit.refreshAria': 'Actualiser le solde', 'deposit.addressTitle': 'Adresse de dépôt', @@ -762,7 +762,7 @@ const TRANSLATIONS = { 'result.lose': 'Diesmal kein Gewinn.', 'deposit.balanceTitle': 'Internes Guthaben', - 'deposit.balanceHint': 'Aktualisiert nach 1 Netzwerkbestätigung', + 'deposit.balanceHint': 'Aktualisiert, sobald das Netzwerk die Einzahlung bestätigt hat (einige Blöcke)', 'deposit.refreshBtn': 'Aktualisieren', 'deposit.refreshAria': 'Guthaben aktualisieren', 'deposit.addressTitle': 'Einzahlungsadresse', @@ -930,7 +930,7 @@ const TRANSLATIONS = { 'result.lose': 'На этот раз без выигрыша.', 'deposit.balanceTitle': 'Внутренний баланс', - 'deposit.balanceHint': 'Обновляется после 1 подтверждения в сети', + 'deposit.balanceHint': 'Обновляется, когда сеть подтвердит депозит (несколько блоков)', 'deposit.refreshBtn': 'Обновить', 'deposit.refreshAria': 'Обновить баланс', 'deposit.addressTitle': 'Адрес для депозита', @@ -1098,7 +1098,7 @@ const TRANSLATIONS = { 'result.lose': '这次没有中奖。', 'deposit.balanceTitle': '内部余额', - 'deposit.balanceHint': '在网络确认 1 次后更新', + 'deposit.balanceHint': '网络确认存款后更新（需要数个区块）', 'deposit.refreshBtn': '刷新', 'deposit.refreshAria': '刷新余额', 'deposit.addressTitle': '存款地址', diff --git a/app/static/index.html b/app/static/index.html index 294e6b5..b4dfa41 100644 --- a/app/static/index.html +++ b/app/static/index.html @@ -176,7 +176,7 @@

Saldo interno

-

Aggiornato dopo 1 conferma sulla rete

+

Aggiornato quando la rete ha confermato il deposito (alcuni blocchi)

— PLM