Validate Electrum headers and corroborate the draw's block (B-28)
A single hostile Electrum server, or a MITM on the one active connection, could fabricate the block header the draw's entropy comes from and so pick the winner of every round: headers were accepted with no proof-of-work check and no link to the previous tip. app/rounds/draw.py adds header_meets_its_own_target (rejects a header whose hash doesn't satisfy the difficulty target it claims) and header_prev_hash. electrum/listener.py's _apply_header now rejects a header failing either check by raising HeaderValidationError, which ends the session the same way a dropped connection would so the listener rotates to the next configured server. ElectrumListener gains corroborate_header: before the draw uses a block, it's independently checked against the other configured servers and needs a majority to agree. rounds/scheduler.py's _wait_for_next_block now calls this and, on failure, logs why and waits for a further block instead of ever using an uncorroborated header. Certificate/hostname verification stays disabled, so this doesn't cover an attacker able to MITM every configured server at once - BUGS.md notes that as not covered. Suite grows from 151 to 165 tests. BUGS.md moves B-28 to Previously fixed.
This commit is contained in:
@@ -1,10 +1,11 @@
|
|||||||
# Known bugs
|
# Known bugs
|
||||||
|
|
||||||
A second full-codebase audit on 2026-07-27 found **25 further issues** (4 critical, 6 high,
|
A second full-codebase audit on 2026-07-27 found **25 further issues** (4 critical, 6 high,
|
||||||
7 medium, 8 low), listed below as B-25 … B-49. B-25, B-26 and B-27 are fixed (see "Previously
|
7 medium, 8 low), listed below as B-29 … B-49. B-25 through B-28 are fixed (see "Previously
|
||||||
fixed" below); the other 22 are open. The 139-test suite was green at the time of the audit, so
|
fixed" below) — no Critical-severity finding remains open; the other 21 are High/Medium/Low.
|
||||||
none of these were caught by existing coverage — every fix lands with a regression test (the
|
The 139-test suite was green at the time of the audit, so none of these were caught by existing
|
||||||
three fixes so far brought the suite from 139 to 151).
|
coverage — every fix lands with a regression test (the four fixes so far brought the suite from
|
||||||
|
139 to 165).
|
||||||
|
|
||||||
The recurring pattern across the open findings is worth stating once: the code is rigorous
|
The recurring pattern across the open findings is worth stating once: the code is rigorous
|
||||||
about the failure modes that have actually been hit, and silent about the ones that have not.
|
about the failure modes that have actually been hit, and silent about the ones that have not.
|
||||||
@@ -18,31 +19,6 @@ single-process assumptions, no user-facing history, etc.), see "Known gaps / TOD
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Critical
|
|
||||||
|
|
||||||
### B-28 — A hostile Electrum server (or a MITM) can choose the winner
|
|
||||||
|
|
||||||
`electrum/listener.py:167-186` accepts any header whose `height >= tip_height`: no
|
|
||||||
proof-of-work check, no linkage to the previous block hash. That header is the **sole source
|
|
||||||
of entropy for the draw** (`scheduler.py:129`).
|
|
||||||
|
|
||||||
In parallel, `electrum/client.py:104-106` sets `check_hostname = False` and
|
|
||||||
`verify_mode = CERT_NONE`. The comment justifies this with "the protocol's trust model is
|
|
||||||
server consensus" — but there is no consensus here: one server at a time, rotated over a list
|
|
||||||
of arbitrary third parties. So a hostile server, or anyone able to MITM a connection that
|
|
||||||
validates no certificate, can fabricate a header and thereby decide who wins every round.
|
|
||||||
|
|
||||||
**Proposed fix, in order of value.** (1) Validate headers before accepting them: check the
|
|
||||||
PoW against the claimed target and that `prev_block` matches the current tip; reject anything
|
|
||||||
that fails. (2) Do not trust one server for the draw — fetch the header for
|
|
||||||
`draw_block_height` from *several* endpoints in the rotation and require agreement before
|
|
||||||
using it as the seed. (3) Pin certificates (or verify hostnames) for the configured servers
|
|
||||||
rather than disabling verification wholesale. Longer term this is the argument for replacing
|
|
||||||
the v1 draw algorithm — CLAUDE.md already calls it a replaceable component — with a scheme
|
|
||||||
that does not depend on a single unauthenticated data source.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## High
|
## High
|
||||||
|
|
||||||
### B-29 — `detect_external_spends` is irreversible and trusts a single response
|
### B-29 — `detect_external_spends` is irreversible and trusts a single response
|
||||||
@@ -321,17 +297,13 @@ already does.
|
|||||||
|
|
||||||
## Previously fixed
|
## Previously fixed
|
||||||
|
|
||||||
B-25 (the payout had no two-phase write, unlike bets and withdrawals — a crash or rejected
|
- **B-25** — the payout had no two-phase write, unlike bets and withdrawals
|
||||||
broadcast could leave money on-chain with no DB record, or leave the round wedged with no way
|
- **B-26** — a payout failure or a process restart could wedge a round in `paying_out` forever
|
||||||
to retry safely), B-26 (a transient failure or a process restart at payout time wedged the
|
- **B-27** — an RBF bump reset the reconciler's own abandon clock, so a repeatedly-bumped tx was never abandoned
|
||||||
round in `paying_out` forever, with most failure paths logging nothing) and B-27 (every RBF
|
- **B-28** — a hostile Electrum server (or a MITM) could single-handedly pick the round's winner
|
||||||
bump reset the reconciler's own abandon clock, so a repeatedly-bumped-but-never-mined
|
|
||||||
transaction was never abandoned) are fixed as of 2026-07-27. Together B-25 and B-26 make
|
See git history for the fix-by-fix breakdown (commits `f13f685`, `50a43ae`, `933760e`, and the
|
||||||
`paying_out` fully recoverable — a payout retry is now safe (persisted before broadcast,
|
B-28 fix). Suite grew from 139 to 165 tests over the four.
|
||||||
guarded against double-spend) and automatic (the scheduler retries a stuck round on its own,
|
|
||||||
throttled, with every failure audit-logged with a reason, including across a process restart).
|
|
||||||
See git history (commits `f13f685`, `50a43ae`, `933760e`) for the fix-by-fix breakdown; the
|
|
||||||
regression suite grew from 139 to 151 tests over the three.
|
|
||||||
|
|
||||||
A full-codebase audit on 2026-07-26 (commit `d4e0974`) found 24 bugs across every Python
|
A full-codebase audit on 2026-07-26 (commit `d4e0974`) found 24 bugs across every Python
|
||||||
module under `app/`, both static frontends, and the Docker/Caddy deployment — 5 critical,
|
module under `app/`, both static frontends, and the Docker/Caddy deployment — 5 critical,
|
||||||
|
|||||||
@@ -9,6 +9,12 @@ from app.db.models import User
|
|||||||
from app.deposits.service import credit_confirmed_utxos, detect_external_spends
|
from app.deposits.service import credit_confirmed_utxos, detect_external_spends
|
||||||
from app.electrum.client import ElectrumClient, ElectrumEndpoint
|
from app.electrum.client import ElectrumClient, ElectrumEndpoint
|
||||||
from app.electrum.scripthash import address_to_scripthash
|
from app.electrum.scripthash import address_to_scripthash
|
||||||
|
from app.rounds.draw import (
|
||||||
|
HeaderValidationError,
|
||||||
|
header_hex_to_block_hash,
|
||||||
|
header_meets_its_own_target,
|
||||||
|
header_prev_hash,
|
||||||
|
)
|
||||||
from app.rounds.events import broadcaster
|
from app.rounds.events import broadcaster
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -18,6 +24,12 @@ logger = logging.getLogger(__name__)
|
|||||||
# difference between noticing the drop in a minute and never noticing it at all.
|
# difference between noticing the drop in a minute and never noticing it at all.
|
||||||
_PING_INTERVAL_SECONDS = 60
|
_PING_INTERVAL_SECONDS = 60
|
||||||
|
|
||||||
|
# B-28: how long to wait for any *one* other server's answer when corroborating the
|
||||||
|
# draw's block header. Shorter than the standard request timeout since this is a
|
||||||
|
# supplementary check across several servers at once — a single slow fallback
|
||||||
|
# shouldn't hold up the others.
|
||||||
|
_CORROBORATION_TIMEOUT_SECONDS = 10
|
||||||
|
|
||||||
|
|
||||||
class ElectrumListener:
|
class ElectrumListener:
|
||||||
"""Long-lived background task: keeps one Electrum connection open, subscribes
|
"""Long-lived background task: keeps one Electrum connection open, subscribes
|
||||||
@@ -173,8 +185,19 @@ class ElectrumListener:
|
|||||||
the draw's wait. height and hex are applied together or not at all —
|
the draw's wait. height and hex are applied together or not at all —
|
||||||
applying a losing header's hex would leave tip_height and tip_header_hex
|
applying a losing header's hex would leave tip_height and tip_header_hex
|
||||||
describing different blocks, and that hex is the draw's entropy source.
|
describing different blocks, and that hex is the draw's entropy source.
|
||||||
|
|
||||||
|
Two validation checks guard against a hostile or MITM'd server simply
|
||||||
|
fabricating a header (B-28), since that header is the draw's sole source of
|
||||||
|
entropy: it must satisfy the difficulty target it claims for itself, and —
|
||||||
|
when it's a direct single-block advance from our own current tip, the only
|
||||||
|
case we can check without a full header chain — it must chain from that
|
||||||
|
tip's hash. Either failure raises HeaderValidationError rather than
|
||||||
|
silently ignoring the header, which (via _consume_headers/_run_once) ends
|
||||||
|
this session the same way a dropped connection would, so run() rotates to
|
||||||
|
the next configured server instead of continuing to trust this one.
|
||||||
"""
|
"""
|
||||||
height = header["height"]
|
height = header["height"]
|
||||||
|
header_hex = header.get("hex")
|
||||||
if height < self.tip_height:
|
if height < self.tip_height:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"ignoring Electrum header at height %s, below the current tip %s (reorg or server switch?)",
|
"ignoring Electrum header at height %s, below the current tip %s (reorg or server switch?)",
|
||||||
@@ -182,8 +205,73 @@ class ElectrumListener:
|
|||||||
self.tip_height,
|
self.tip_height,
|
||||||
)
|
)
|
||||||
return
|
return
|
||||||
|
|
||||||
|
if header_hex:
|
||||||
|
if not header_meets_its_own_target(header_hex):
|
||||||
|
raise HeaderValidationError(
|
||||||
|
f"header at height {height} does not satisfy its own claimed difficulty target"
|
||||||
|
)
|
||||||
|
if (
|
||||||
|
self.tip_header_hex
|
||||||
|
and height == self.tip_height + 1
|
||||||
|
and header_prev_hash(header_hex) != header_hex_to_block_hash(self.tip_header_hex)
|
||||||
|
):
|
||||||
|
raise HeaderValidationError(
|
||||||
|
f"header at height {height} does not chain from the current tip (height {self.tip_height})"
|
||||||
|
)
|
||||||
|
|
||||||
self.tip_height = height
|
self.tip_height = height
|
||||||
self.tip_header_hex = header.get("hex")
|
self.tip_header_hex = header_hex
|
||||||
|
|
||||||
|
async def corroborate_header(self, height: int, expected_hash: str) -> bool:
|
||||||
|
"""B-28: independently ask every *other* configured server for the header
|
||||||
|
at `height` and require a strict majority of the ones that actually answer
|
||||||
|
to agree with `expected_hash` — the hash our own active connection
|
||||||
|
reported — before the draw (rounds/scheduler.py:_wait_for_next_block) treats
|
||||||
|
it as trustworthy entropy. Without this, a single hostile server (or a MITM
|
||||||
|
on the one active connection) can single-handedly decide who wins every
|
||||||
|
round; this raises the bar to controlling a majority of the configured
|
||||||
|
servers.
|
||||||
|
|
||||||
|
Returns True if there are no other servers configured at all — a
|
||||||
|
single-endpoint deployment has nothing to corroborate against, and accepted
|
||||||
|
that risk when ELECTRUM_FALLBACK_SERVERS was left empty (see CLAUDE.md).
|
||||||
|
Also returns False (never silently "passes") if none of the other servers
|
||||||
|
could be reached at all, since an unreachable network answers nothing about
|
||||||
|
whether the header is genuine.
|
||||||
|
"""
|
||||||
|
others = [endpoint for endpoint in self._endpoints if endpoint != self.current_endpoint]
|
||||||
|
if not others:
|
||||||
|
return True
|
||||||
|
|
||||||
|
async def _ask(endpoint: ElectrumEndpoint) -> str | None:
|
||||||
|
client = self._client_factory(endpoint)
|
||||||
|
try:
|
||||||
|
await asyncio.wait_for(client.connect(), timeout=_CORROBORATION_TIMEOUT_SECONDS)
|
||||||
|
result = await asyncio.wait_for(
|
||||||
|
client.request("blockchain.block.header", [height]),
|
||||||
|
timeout=_CORROBORATION_TIMEOUT_SECONDS,
|
||||||
|
)
|
||||||
|
if not isinstance(result, str):
|
||||||
|
return None
|
||||||
|
return header_hex_to_block_hash(result)
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
finally:
|
||||||
|
await client.close()
|
||||||
|
|
||||||
|
results = await asyncio.gather(*(_ask(endpoint) for endpoint in others))
|
||||||
|
responded = [block_hash for block_hash in results if block_hash is not None]
|
||||||
|
if not responded:
|
||||||
|
logger.warning(
|
||||||
|
"could not corroborate block %s header with any of %s other configured server(s)",
|
||||||
|
height,
|
||||||
|
len(others),
|
||||||
|
)
|
||||||
|
return False
|
||||||
|
|
||||||
|
agreements = sum(1 for block_hash in responded if block_hash == expected_hash)
|
||||||
|
return agreements * 2 > len(responded)
|
||||||
|
|
||||||
async def _consume_headers(self, queue: asyncio.Queue) -> None:
|
async def _consume_headers(self, queue: asyncio.Queue) -> None:
|
||||||
while True:
|
while True:
|
||||||
|
|||||||
@@ -1,5 +1,22 @@
|
|||||||
import hashlib
|
import hashlib
|
||||||
|
|
||||||
|
# Byte offsets of a standard 80-byte block header: version(4) + prev_block(32) +
|
||||||
|
# merkle_root(32) + timestamp(4) + bits(4) + nonce(4).
|
||||||
|
_HEADER_LENGTH_BYTES = 80
|
||||||
|
_PREV_BLOCK_OFFSET = 4
|
||||||
|
_PREV_BLOCK_LENGTH = 32
|
||||||
|
_BITS_OFFSET = 72
|
||||||
|
_BITS_LENGTH = 4
|
||||||
|
|
||||||
|
|
||||||
|
class HeaderValidationError(Exception):
|
||||||
|
"""Raised by ElectrumListener._apply_header (B-28) when a header either doesn't
|
||||||
|
satisfy the difficulty target it claims for itself, or doesn't chain from the
|
||||||
|
previously accepted tip. Letting this propagate out of the header-consuming
|
||||||
|
task ends the current Electrum session the same way a dropped connection would
|
||||||
|
(see ElectrumListener._run_once), so the listener rotates to the next
|
||||||
|
configured server instead of trusting a header a server just forged."""
|
||||||
|
|
||||||
|
|
||||||
def header_hex_to_block_hash(header_hex: str) -> str:
|
def header_hex_to_block_hash(header_hex: str) -> str:
|
||||||
"""Block hash from a raw Electrum header: sha256d, byte-reversed, hex.
|
"""Block hash from a raw Electrum header: sha256d, byte-reversed, hex.
|
||||||
@@ -10,6 +27,56 @@ def header_hex_to_block_hash(header_hex: str) -> str:
|
|||||||
return digest[::-1].hex()
|
return digest[::-1].hex()
|
||||||
|
|
||||||
|
|
||||||
|
def header_prev_hash(header_hex: str) -> str:
|
||||||
|
"""The header's `prev_block` field, byte-reversed to the same conventional
|
||||||
|
(display) order as header_hex_to_block_hash's return value, so the two can be
|
||||||
|
compared directly to check that one header actually chains from another."""
|
||||||
|
header_bytes = bytes.fromhex(header_hex)
|
||||||
|
prev = header_bytes[_PREV_BLOCK_OFFSET : _PREV_BLOCK_OFFSET + _PREV_BLOCK_LENGTH]
|
||||||
|
return prev[::-1].hex()
|
||||||
|
|
||||||
|
|
||||||
|
def _target_from_bits(bits: int) -> int:
|
||||||
|
"""Decompress Bitcoin-style compact `nBits` difficulty encoding into the full
|
||||||
|
256-bit target a valid header's hash must be less than or equal to."""
|
||||||
|
exponent = bits >> 24
|
||||||
|
mantissa = bits & 0xFFFFFF
|
||||||
|
if exponent <= 3:
|
||||||
|
return mantissa >> (8 * (3 - exponent))
|
||||||
|
return mantissa << (8 * (exponent - 3))
|
||||||
|
|
||||||
|
|
||||||
|
def header_meets_its_own_target(header_hex: str) -> bool:
|
||||||
|
"""Whether this header's hash satisfies the difficulty target *it claims for
|
||||||
|
itself* (the `bits` field). Rejects a header that was never actually mined —
|
||||||
|
e.g. one fabricated wholesale by a hostile or MITM'd Electrum server (B-28),
|
||||||
|
since satisfying a self-chosen target still requires real proof-of-work.
|
||||||
|
|
||||||
|
This does NOT — and, short of downloading and validating the full header
|
||||||
|
chain's difficulty-retarget history, cannot — catch a header honestly mined at
|
||||||
|
a real but implausibly low self-chosen difficulty: a server could still declare
|
||||||
|
an easy target and grind it out with modest hardware. That residual risk is why
|
||||||
|
the draw additionally requires the winning block's header to be corroborated by
|
||||||
|
the *other* configured servers before using it as the seed (see
|
||||||
|
ElectrumListener.corroborate_header and rounds/scheduler.py:_wait_for_next_block)
|
||||||
|
rather than relying on this check alone.
|
||||||
|
"""
|
||||||
|
header_bytes = bytes.fromhex(header_hex)
|
||||||
|
if len(header_bytes) != _HEADER_LENGTH_BYTES:
|
||||||
|
return False
|
||||||
|
bits = int.from_bytes(header_bytes[_BITS_OFFSET : _BITS_OFFSET + _BITS_LENGTH], "little")
|
||||||
|
target = _target_from_bits(bits)
|
||||||
|
if target <= 0:
|
||||||
|
return False
|
||||||
|
digest = hashlib.sha256(hashlib.sha256(header_bytes).digest()).digest()
|
||||||
|
# The hash as the integer comparable against `target`: this is the same digest
|
||||||
|
# header_hex_to_block_hash reverses into the conventional display hex, so
|
||||||
|
# reading it byte-reversed as a big-endian int is equivalent to reading the
|
||||||
|
# original digest bytes as little-endian — both give the same integer.
|
||||||
|
hash_int = int.from_bytes(digest, "little")
|
||||||
|
return hash_int <= target
|
||||||
|
|
||||||
|
|
||||||
def draw_winner(participants: list[str], block_hash_hex: str) -> str:
|
def draw_winner(participants: list[str], block_hash_hex: str) -> str:
|
||||||
"""v1 draw algorithm (flowchart.mmd, node R): seed = block hash as an integer,
|
"""v1 draw algorithm (flowchart.mmd, node R): seed = block hash as an integer,
|
||||||
index = seed mod participant_count, winner = participants[index]. Anyone can
|
index = seed mod participant_count, winner = participants[index]. Anyone can
|
||||||
|
|||||||
+33
-5
@@ -142,7 +142,7 @@ class RoundScheduler:
|
|||||||
broadcaster.publish()
|
broadcaster.publish()
|
||||||
|
|
||||||
tip_at_close = self._listener.tip_height
|
tip_at_close = self._listener.tip_height
|
||||||
block_height, block_hash = await self._wait_for_next_block(tip_at_close)
|
block_height, block_hash = await self._wait_for_next_block(round_id, tip_at_close)
|
||||||
winner_address = draw_winner(addresses, block_hash)
|
winner_address = draw_winner(addresses, block_hash)
|
||||||
|
|
||||||
async with self._session_factory() as session:
|
async with self._session_factory() as session:
|
||||||
@@ -172,10 +172,38 @@ class RoundScheduler:
|
|||||||
logger.info("round %s: winner=%s pool=%s", round_id, winner_address, pool_amount)
|
logger.info("round %s: winner=%s pool=%s", round_id, winner_address, pool_amount)
|
||||||
await self._trigger_payout(round_id)
|
await self._trigger_payout(round_id)
|
||||||
|
|
||||||
async def _wait_for_next_block(self, tip_at_close: int) -> tuple[int, str]:
|
async def _wait_for_next_block(self, round_id: int, tip_at_close: int) -> tuple[int, str]:
|
||||||
while self._listener.tip_height <= tip_at_close or not self._listener.tip_header_hex:
|
"""Waits for a block after tip_at_close and, before handing it back as the
|
||||||
await asyncio.sleep(_TICK_INTERVAL_SECONDS)
|
draw's entropy source, requires it to be corroborated by the other
|
||||||
return self._listener.tip_height, header_hex_to_block_hash(self._listener.tip_header_hex)
|
configured Electrum servers (B-28) — our own active connection is exactly
|
||||||
|
the thing a hostile server or a MITM would control, so its header alone is
|
||||||
|
not enough to seed a payout. A candidate that fails corroboration is never
|
||||||
|
used: this keeps waiting for a further block and tries corroborating that
|
||||||
|
one instead, logging why every time so a stuck draw is visible in
|
||||||
|
/admin's audit log rather than a silent, unexplained wait."""
|
||||||
|
while True:
|
||||||
|
while self._listener.tip_height <= tip_at_close or not self._listener.tip_header_hex:
|
||||||
|
await asyncio.sleep(_TICK_INTERVAL_SECONDS)
|
||||||
|
height = self._listener.tip_height
|
||||||
|
block_hash = header_hex_to_block_hash(self._listener.tip_header_hex)
|
||||||
|
if await self._listener.corroborate_header(height, block_hash):
|
||||||
|
return height, block_hash
|
||||||
|
logger.error(
|
||||||
|
"round %s: block %s header %s could not be corroborated by other Electrum servers; "
|
||||||
|
"waiting for a further block",
|
||||||
|
round_id,
|
||||||
|
height,
|
||||||
|
block_hash,
|
||||||
|
)
|
||||||
|
async with self._session_factory() as session:
|
||||||
|
await write_audit_log(
|
||||||
|
session,
|
||||||
|
"draw_header_corroboration_failed",
|
||||||
|
{"height": height, "reported_hash": block_hash},
|
||||||
|
round_id=round_id,
|
||||||
|
)
|
||||||
|
await session.commit()
|
||||||
|
tip_at_close = height
|
||||||
|
|
||||||
async def _retry_payout_if_due(self, round_id: int) -> None:
|
async def _retry_payout_if_due(self, round_id: int) -> None:
|
||||||
"""B-26: whether a "paying_out" round is due for another payout attempt.
|
"""B-26: whether a "paying_out" round is due for another payout attempt.
|
||||||
|
|||||||
+41
-10
@@ -1,19 +1,50 @@
|
|||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from app.rounds.draw import draw_winner, header_hex_to_block_hash
|
from app.rounds.draw import (
|
||||||
|
draw_winner,
|
||||||
|
header_hex_to_block_hash,
|
||||||
|
header_meets_its_own_target,
|
||||||
|
header_prev_hash,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Real PLM mainnet block 477486: header from blockchain.block.header, hash
|
||||||
|
# cross-checked against the blockhash reported by blockchain.transaction.get for a
|
||||||
|
# tx confirmed in that block.
|
||||||
|
_REAL_HEADER_HEX = (
|
||||||
|
"0020ed30fbc39ee45200d10214f5107c5f36e7bf753032fd1d3279810c170000000000"
|
||||||
|
"009a4ea723f732be4c538f3a2490837dd6560103d73ece606aa7d578a66700447b28875"
|
||||||
|
"e6a47a61b1ad8012582"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_header_hex_to_block_hash_matches_known_mainnet_block():
|
def test_header_hex_to_block_hash_matches_known_mainnet_block():
|
||||||
# Real PLM mainnet block 477486: header from blockchain.block.header, hash
|
|
||||||
# cross-checked against the blockhash reported by blockchain.transaction.get
|
|
||||||
# for a tx confirmed in that block.
|
|
||||||
header_hex = (
|
|
||||||
"0020ed30fbc39ee45200d10214f5107c5f36e7bf753032fd1d3279810c170000000000"
|
|
||||||
"009a4ea723f732be4c538f3a2490837dd6560103d73ece606aa7d578a66700447b28875"
|
|
||||||
"e6a47a61b1ad8012582"
|
|
||||||
)
|
|
||||||
known_block_hash = "00000000000008788b55ade13b74d54ceffda9e54315b802411be1ca65064e86"
|
known_block_hash = "00000000000008788b55ade13b74d54ceffda9e54315b802411be1ca65064e86"
|
||||||
assert header_hex_to_block_hash(header_hex) == known_block_hash
|
assert header_hex_to_block_hash(_REAL_HEADER_HEX) == known_block_hash
|
||||||
|
|
||||||
|
|
||||||
|
def test_header_meets_its_own_target_accepts_a_real_mined_header():
|
||||||
|
"""B-28: a genuinely mined mainnet header must pass its own self-consistency
|
||||||
|
check — this isn't just a synthetic-header property."""
|
||||||
|
assert header_meets_its_own_target(_REAL_HEADER_HEX) is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_header_meets_its_own_target_rejects_a_tampered_header():
|
||||||
|
"""Flipping a single nonce bit changes the hash completely (avalanche effect)
|
||||||
|
without changing the claimed difficulty, so a tampered-but-otherwise-real
|
||||||
|
header should almost certainly fail — this is what would catch a
|
||||||
|
hostile/MITM'd server replaying a real header with a doctored field."""
|
||||||
|
tampered = bytearray(bytes.fromhex(_REAL_HEADER_HEX))
|
||||||
|
tampered[-1] ^= 0xFF # flip the last byte of the nonce
|
||||||
|
assert header_meets_its_own_target(tampered.hex()) is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_header_meets_its_own_target_rejects_wrong_length():
|
||||||
|
assert header_meets_its_own_target("aa" * 10) is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_header_prev_hash_matches_the_known_previous_block():
|
||||||
|
# Block 477486's predecessor, 477485 — independently known from the same chain.
|
||||||
|
assert header_prev_hash(_REAL_HEADER_HEX) == "000000000000170c8179321dfd323075bfe7365f7c10f51402d10052e49ec3fb"
|
||||||
|
|
||||||
|
|
||||||
def test_draw_winner_is_deterministic_and_within_range():
|
def test_draw_winner_is_deterministic_and_within_range():
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
"""Listener-level behaviour: server rotation on failure (the fallback-servers
|
"""Listener-level behaviour: server rotation on failure (the fallback-servers
|
||||||
feature), and the chain-tip monotonicity guard (B-19).
|
feature), the chain-tip monotonicity guard (B-19), header validation and
|
||||||
|
multi-server corroboration (B-28).
|
||||||
|
|
||||||
The reconnect loop itself (B-01) is covered from the client side in
|
The reconnect loop itself (B-01) is covered from the client side in
|
||||||
test_electrum_client.py — what's asserted here is that the listener *acts* on a
|
test_electrum_client.py — what's asserted here is that the listener *acts* on a
|
||||||
@@ -7,6 +8,7 @@ dead connection by moving to the next server instead of retrying the same one.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
|
import struct
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||||
@@ -14,6 +16,7 @@ from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
|||||||
from app.db.base import Base
|
from app.db.base import Base
|
||||||
from app.electrum.client import ElectrumEndpoint
|
from app.electrum.client import ElectrumEndpoint
|
||||||
from app.electrum.listener import ElectrumListener
|
from app.electrum.listener import ElectrumListener
|
||||||
|
from app.rounds.draw import HeaderValidationError, header_hex_to_block_hash, header_meets_its_own_target
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
@@ -31,6 +34,34 @@ _ENDPOINTS = [
|
|||||||
ElectrumEndpoint("third.example", 50001, False),
|
ElectrumEndpoint("third.example", 50001, False),
|
||||||
]
|
]
|
||||||
|
|
||||||
|
# A regtest-style trivial difficulty target (~50% of hashes satisfy it), so mining
|
||||||
|
# a real, self-consistent test header takes a handful of nonce attempts rather than
|
||||||
|
# needing actual mainnet-grade hashpower. Not a valid PLM mainnet difficulty —
|
||||||
|
# irrelevant here, since header_meets_its_own_target only checks self-consistency.
|
||||||
|
_EASY_BITS = 0x207FFFFF
|
||||||
|
|
||||||
|
|
||||||
|
def _build_header(prev_hash_hex: str, nonce: int, *, bits: int = _EASY_BITS) -> str:
|
||||||
|
return (
|
||||||
|
struct.pack("<I", 1) # version
|
||||||
|
+ bytes.fromhex(prev_hash_hex)[::-1]
|
||||||
|
+ bytes.fromhex("00" * 32) # merkle_root, irrelevant to the checks under test
|
||||||
|
+ struct.pack("<I", 0) # timestamp
|
||||||
|
+ struct.pack("<I", bits)
|
||||||
|
+ struct.pack("<I", nonce)
|
||||||
|
).hex()
|
||||||
|
|
||||||
|
|
||||||
|
def _mine_header(prev_hash_hex: str, *, bits: int = _EASY_BITS) -> str:
|
||||||
|
"""A real header that satisfies its own claimed target — good enough to
|
||||||
|
exercise header_meets_its_own_target/_apply_header for real, without needing
|
||||||
|
genuine PLM-mainnet-grade hashpower."""
|
||||||
|
for nonce in range(100_000):
|
||||||
|
header_hex = _build_header(prev_hash_hex, nonce, bits=bits)
|
||||||
|
if header_meets_its_own_target(header_hex):
|
||||||
|
return header_hex
|
||||||
|
raise RuntimeError("failed to mine a test header within the attempt budget")
|
||||||
|
|
||||||
|
|
||||||
async def test_rotates_to_the_next_server_after_a_failed_session(session_factory):
|
async def test_rotates_to_the_next_server_after_a_failed_session(session_factory):
|
||||||
"""One unreachable server should cost a single attempt, not an outage: every
|
"""One unreachable server should cost a single attempt, not an outage: every
|
||||||
@@ -132,11 +163,144 @@ def test_tip_never_moves_backwards(session_factory):
|
|||||||
a stale one."""
|
a stale one."""
|
||||||
listener = ElectrumListener(lambda endpoint: None, session_factory, _ENDPOINTS)
|
listener = ElectrumListener(lambda endpoint: None, session_factory, _ENDPOINTS)
|
||||||
|
|
||||||
listener._apply_header({"height": 100, "hex": "aa"})
|
header_100 = _mine_header("00" * 32)
|
||||||
assert (listener.tip_height, listener.tip_header_hex) == (100, "aa")
|
listener._apply_header({"height": 100, "hex": header_100})
|
||||||
|
assert (listener.tip_height, listener.tip_header_hex) == (100, header_100)
|
||||||
|
|
||||||
listener._apply_header({"height": 99, "hex": "bb"}) # reorg, or a server switch
|
# A lower height is ignored purely on height, before any header validation even
|
||||||
assert (listener.tip_height, listener.tip_header_hex) == (100, "aa")
|
# runs — reorg or server switch, not a real advance.
|
||||||
|
listener._apply_header({"height": 99, "hex": "bb"})
|
||||||
|
assert (listener.tip_height, listener.tip_header_hex) == (100, header_100)
|
||||||
|
|
||||||
listener._apply_header({"height": 101, "hex": "cc"})
|
header_101 = _mine_header(header_hex_to_block_hash(header_100))
|
||||||
assert (listener.tip_height, listener.tip_header_hex) == (101, "cc")
|
listener._apply_header({"height": 101, "hex": header_101})
|
||||||
|
assert (listener.tip_height, listener.tip_header_hex) == (101, header_101)
|
||||||
|
|
||||||
|
|
||||||
|
# --- B-28: a hostile or MITM'd server can no longer single-handedly decide the
|
||||||
|
# draw's entropy — header self-consistency/linkage checks, and multi-server
|
||||||
|
# corroboration for the block the draw actually uses. ---------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_apply_header_rejects_one_that_fails_its_own_pow_target(session_factory):
|
||||||
|
listener = ElectrumListener(lambda endpoint: None, session_factory, _ENDPOINTS)
|
||||||
|
# Real mainnet-grade difficulty (genesis-era Bitcoin bits): satisfying it by
|
||||||
|
# chance is astronomically unlikely, so this header is self-inconsistent.
|
||||||
|
forged = _build_header("00" * 32, nonce=0, bits=0x1D00FFFF)
|
||||||
|
|
||||||
|
with pytest.raises(HeaderValidationError):
|
||||||
|
listener._apply_header({"height": 100, "hex": forged})
|
||||||
|
assert (listener.tip_height, listener.tip_header_hex) == (0, None) # untouched
|
||||||
|
|
||||||
|
|
||||||
|
def test_apply_header_rejects_one_that_does_not_chain_from_the_tip(session_factory):
|
||||||
|
listener = ElectrumListener(lambda endpoint: None, session_factory, _ENDPOINTS)
|
||||||
|
header_100 = _mine_header("00" * 32)
|
||||||
|
listener._apply_header({"height": 100, "hex": header_100})
|
||||||
|
|
||||||
|
# A single-block advance (101 = 100 + 1) whose prev_block claims an unrelated
|
||||||
|
# chain — well-formed and self-consistently mined, but not actually built on
|
||||||
|
# top of our current tip.
|
||||||
|
disconnected = _mine_header("ff" * 32)
|
||||||
|
|
||||||
|
with pytest.raises(HeaderValidationError):
|
||||||
|
listener._apply_header({"height": 101, "hex": disconnected})
|
||||||
|
assert (listener.tip_height, listener.tip_header_hex) == (100, header_100) # untouched
|
||||||
|
|
||||||
|
|
||||||
|
def test_apply_header_skips_linkage_check_across_a_height_gap(session_factory):
|
||||||
|
"""A reconnect (or the very first header of a session) hands us whatever the
|
||||||
|
server's current tip is — which is legitimately not a single-block advance
|
||||||
|
from whatever we last saw. There's no full header chain to check linkage
|
||||||
|
against in that case, so only self-consistency is enforced."""
|
||||||
|
listener = ElectrumListener(lambda endpoint: None, session_factory, _ENDPOINTS)
|
||||||
|
header_100 = _mine_header("00" * 32)
|
||||||
|
listener._apply_header({"height": 100, "hex": header_100})
|
||||||
|
|
||||||
|
header_150 = _mine_header("ff" * 32) # unrelated prev_block, height jumps by 50
|
||||||
|
listener._apply_header({"height": 150, "hex": header_150}) # must not raise
|
||||||
|
|
||||||
|
assert (listener.tip_height, listener.tip_header_hex) == (150, header_150)
|
||||||
|
|
||||||
|
|
||||||
|
async def _endpoint_client_factory(responses: dict[str, object]):
|
||||||
|
"""Builds a client_factory whose fake clients answer blockchain.block.header
|
||||||
|
per-endpoint according to `responses`: a header hex string to agree/disagree
|
||||||
|
with, `None` to simulate an unreachable server, or an Exception instance to
|
||||||
|
simulate a request failure."""
|
||||||
|
|
||||||
|
class _FakeClient:
|
||||||
|
def __init__(self, answer):
|
||||||
|
self._answer = answer
|
||||||
|
self.closed = False
|
||||||
|
|
||||||
|
async def connect(self):
|
||||||
|
if isinstance(self._answer, Exception):
|
||||||
|
raise self._answer
|
||||||
|
|
||||||
|
async def request(self, method, params):
|
||||||
|
assert method == "blockchain.block.header"
|
||||||
|
if self._answer is None:
|
||||||
|
raise ConnectionRefusedError("unreachable")
|
||||||
|
return self._answer
|
||||||
|
|
||||||
|
async def close(self):
|
||||||
|
self.closed = True
|
||||||
|
|
||||||
|
def factory(endpoint):
|
||||||
|
return _FakeClient(responses[endpoint.host])
|
||||||
|
|
||||||
|
return factory
|
||||||
|
|
||||||
|
|
||||||
|
async def test_corroborate_header_true_with_no_other_servers_configured(session_factory):
|
||||||
|
single = [ElectrumEndpoint("only.example", 50002, True)]
|
||||||
|
listener = ElectrumListener(lambda endpoint: None, session_factory, single)
|
||||||
|
assert await listener.corroborate_header(100, "deadbeef") is True
|
||||||
|
|
||||||
|
|
||||||
|
async def test_corroborate_header_true_when_others_agree(session_factory):
|
||||||
|
header_hex = _mine_header("00" * 32)
|
||||||
|
expected_hash = header_hex_to_block_hash(header_hex)
|
||||||
|
factory = await _endpoint_client_factory(
|
||||||
|
{"first.example": header_hex, "second.example": header_hex, "third.example": header_hex}
|
||||||
|
)
|
||||||
|
listener = ElectrumListener(factory, session_factory, _ENDPOINTS)
|
||||||
|
|
||||||
|
assert await listener.corroborate_header(100, expected_hash) is True
|
||||||
|
|
||||||
|
|
||||||
|
async def test_corroborate_header_never_asks_the_currently_active_endpoint(session_factory):
|
||||||
|
"""The active connection is exactly what a hostile server or a MITM would
|
||||||
|
control — corroborating against it too would defeat the point."""
|
||||||
|
header_hex = _mine_header("00" * 32)
|
||||||
|
expected_hash = header_hex_to_block_hash(header_hex)
|
||||||
|
# first.example (the active endpoint) would raise if ever queried.
|
||||||
|
factory = await _endpoint_client_factory(
|
||||||
|
{"first.example": RuntimeError("must not be called"), "second.example": header_hex, "third.example": header_hex}
|
||||||
|
)
|
||||||
|
listener = ElectrumListener(factory, session_factory, _ENDPOINTS)
|
||||||
|
assert listener.current_endpoint.host == "first.example"
|
||||||
|
|
||||||
|
assert await listener.corroborate_header(100, expected_hash) is True
|
||||||
|
|
||||||
|
|
||||||
|
async def test_corroborate_header_false_when_majority_disagrees(session_factory):
|
||||||
|
header_hex = _mine_header("00" * 32)
|
||||||
|
expected_hash = header_hex_to_block_hash(header_hex)
|
||||||
|
disagreeing_hex = _mine_header("11" * 32)
|
||||||
|
factory = await _endpoint_client_factory(
|
||||||
|
{"first.example": header_hex, "second.example": disagreeing_hex, "third.example": disagreeing_hex}
|
||||||
|
)
|
||||||
|
listener = ElectrumListener(factory, session_factory, _ENDPOINTS)
|
||||||
|
|
||||||
|
assert await listener.corroborate_header(100, expected_hash) is False
|
||||||
|
|
||||||
|
|
||||||
|
async def test_corroborate_header_false_when_nobody_responds(session_factory):
|
||||||
|
factory = await _endpoint_client_factory(
|
||||||
|
{"first.example": "irrelevant", "second.example": None, "third.example": ConnectionRefusedError("down")}
|
||||||
|
)
|
||||||
|
listener = ElectrumListener(factory, session_factory, _ENDPOINTS)
|
||||||
|
|
||||||
|
assert await listener.corroborate_header(100, "deadbeef") is False
|
||||||
|
|||||||
@@ -353,3 +353,53 @@ async def test_tick_retries_once_the_throttle_window_has_elapsed(payout_session_
|
|||||||
async with payout_session_factory() as session:
|
async with payout_session_factory() as session:
|
||||||
pending = (await session.scalars(select(PendingTransaction))).one()
|
pending = (await session.scalars(select(PendingTransaction))).one()
|
||||||
assert pending.status == "pending"
|
assert pending.status == "pending"
|
||||||
|
|
||||||
|
|
||||||
|
# --- B-28: the draw must not seed itself from an uncorroborated header -----------
|
||||||
|
|
||||||
|
|
||||||
|
class CorroboratingListener:
|
||||||
|
"""A fake listener whose tip advances the moment a corroboration attempt
|
||||||
|
fails, simulating a further block arriving — lets tests drive
|
||||||
|
_wait_for_next_block's retry loop deterministically without real sleeps."""
|
||||||
|
|
||||||
|
def __init__(self, *, responses: dict[int, bool], advance_to: dict[int, tuple[int, str]] | None = None):
|
||||||
|
self.tip_height, self.tip_header_hex = next(iter(responses)), "aa"
|
||||||
|
self._responses = dict(responses)
|
||||||
|
self._advance_to = advance_to or {}
|
||||||
|
self.corroboration_calls: list[int] = []
|
||||||
|
|
||||||
|
async def corroborate_header(self, height: int, expected_hash: str) -> bool:
|
||||||
|
self.corroboration_calls.append(height)
|
||||||
|
result = self._responses[height]
|
||||||
|
if not result and height in self._advance_to:
|
||||||
|
self.tip_height, self.tip_header_hex = self._advance_to[height]
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
async def test_wait_for_next_block_accepts_an_immediately_corroborated_block(session_factory):
|
||||||
|
listener = CorroboratingListener(responses={101: True})
|
||||||
|
scheduler = RoundScheduler(session_factory, listener)
|
||||||
|
|
||||||
|
height, block_hash = await scheduler._wait_for_next_block(round_id=1, tip_at_close=100)
|
||||||
|
|
||||||
|
assert height == 101
|
||||||
|
assert listener.corroboration_calls == [101]
|
||||||
|
|
||||||
|
|
||||||
|
async def test_wait_for_next_block_retries_after_a_failed_corroboration(session_factory):
|
||||||
|
"""B-28: an uncorroborated header must never be used — the wait keeps going
|
||||||
|
until a later block's header *is* corroborated, logging why each time."""
|
||||||
|
listener = CorroboratingListener(
|
||||||
|
responses={101: False, 102: True}, advance_to={101: (102, "bb")}
|
||||||
|
)
|
||||||
|
scheduler = RoundScheduler(session_factory, listener)
|
||||||
|
|
||||||
|
height, block_hash = await scheduler._wait_for_next_block(round_id=1, tip_at_close=100)
|
||||||
|
|
||||||
|
assert height == 102
|
||||||
|
assert listener.corroboration_calls == [101, 102]
|
||||||
|
|
||||||
|
async with session_factory() as session:
|
||||||
|
events = [e.event_type for e in (await session.scalars(select(AuditLog))).all()]
|
||||||
|
assert events == ["draw_header_corroboration_failed"]
|
||||||
|
|||||||
Reference in New Issue
Block a user