davide/plm-lottery
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Require real proof of work in headers and link every advance to the tip (B-74, ELC-11, ELC-12, ELC-22, ELC-23)

Browse Source 
A header only had to meet the target it declared for itself, so a fabricated one with
`bits = 0x207fffff` passed half the time, a jump of several blocks was accepted without
being linked to the tip (a fake header at a huge height locked `tip_height` and froze every
draw), and an ordinary one-block reorganisation closed the session on every server.

- draw.py: MAINNET_POW_LIMIT (00000000ffff...ffff); header_meets_its_own_target also
  refuses a claimed target easier than POW_LIMIT.
- ElectrumListener._apply_header is async and takes the active client. A single block must
  chain by prev_hash; a jump downloads the headers in between (blockchain.block.headers,
  at most 2016 per request, ElectrumClient.get_block_headers) and each must carry valid
  work and chain, otherwise HeaderValidationError ends the session.
- The listener keeps the last 10 headers (`_window`), built from the process's first header
  and its 9 predecessors (which must chain into it). A header that does not join the tip is
  walked back to its fork point inside the window and the server's branch is followed
  instead of ending the session; a fork older than the window, or a branch that does not
  end at the announced header, still ends it. The audit row and the pause for deep reorgs
  are B-85; a competing header at the tip's own height is still ignored until then.
- _header_at also serves older heights from the window.

Tests changed because the spec changed: the mainnet powLimit cannot be mined in a test, so
tests/unit/conftest.py swaps in a regtest limit at import time (the new tests put the real
one back); the _apply_header tests are async; the two _run_once tests serve the first
header's 9 predecessors; test_apply_header_skips_linkage_check_across_a_height_gap is
replaced by the jump tests, since that behaviour is exactly what the finding removed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
davideandClaude Sonnet 5.5 committed 2026-10-14 15:45:34 +02:00
1 parent a3c51719bd
commit ca50ef62f3
16 files changed
+494 -159
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

app/electrum/client.py
+6
View File
@@ -287,6 +287,12 @@ class ElectrumClient:
async def get_block_header(self, height: int) -> str:
return await self.request("blockchain.block.header", [height])
async def get_block_headers(self, start_height: int, count: int) -> dict:
"""`count` consecutive headers from `start_height` (B-74, ELC-12), as the server's
{"count", "hex", "max"}: `hex` is the 80-byte headers concatenated, and the server
may return fewer than asked (never more than its `max`)."""
return await self.request("blockchain.block.headers", [start_height, count])
async def _read_loop(self) -> None:
"""Frame the stream into lines by hand rather than with StreamReader.readline:
readline raises on a line over the limit, and that exception used to end this
app/electrum/listener.py
+165 -51
View File
@@ -47,6 +47,14 @@ _CORROBORATION_TIMEOUT_SECONDS = 10
# of simultaneous in-flight requests against the one active connection.
_RESUBSCRIBE_CONCURRENCY = 20
# How many of the latest accepted headers the listener keeps (B-74/B-85, ELC-22): what lets
# it recognise a reorganisation and walk back to its fork point instead of mistaking it for
# a hostile server. Also the deepest fork it will follow.
_HEADER_WINDOW = 10
# `blockchain.block.headers` serves at most this many headers per request (ELC-12).
_MAX_HEADERS_PER_REQUEST = 2016
class ElectrumListener:
"""Long-lived background task: keeps one Electrum connection open, subscribes
@@ -85,6 +93,9 @@ class ElectrumListener:
self._background_tasks: set[asyncio.Task] = set()
self.tip_height: int = 0
self.tip_header_hex: str | None = None
# B-74 (ELC-22): height -> header of the last _HEADER_WINDOW accepted headers, tip
# included. Survives reconnections, not restarts.
self._window: dict[int, str] = {}
self.client: ElectrumClient | None = None
@property
@@ -173,7 +184,7 @@ class ElectrumListener:
try:
header = await client.subscribe_headers()
self._apply_header(header)
await self._apply_header(header, client)
if self.tip_header_hex is None:
# B-64: the header was unusable (no hex) and we have never had a tip,
# so publishing this client would hand every consumer a connection
@@ -202,7 +213,7 @@ class ElectrumListener:
# keepalive are what make the drop observable. Whichever finishes first
# ends the session and sends run() around to the next server.
tasks = [
asyncio.create_task(self._consume_headers(headers_queue)),
asyncio.create_task(self._consume_headers(headers_queue, client)),
asyncio.create_task(self._consume_scripthash(scripthash_queue)),
asyncio.create_task(self._keepalive(client)),
asyncio.create_task(client.wait_closed()),
@@ -277,29 +288,27 @@ class ElectrumListener:
await self.client.subscribe_scripthash(scripthash)
await self.refresh_user(user_id, scripthash)
def _apply_header(self, header: dict) -> None:
"""Record a new chain tip, refusing to move backwards or sideways.
async def _apply_header(self, header: dict, client: ElectrumClient | None = None) -> None:
"""Record a new chain tip after checking it (ELC-08 … ELC-13, ELC-22, ELC-23).
Full reorg handling is out of scope for v1 by explicit design decision, but
the tip must never regress: `_wait_for_next_block` waits for
`tip_height > tip_at_close`, so a lower height would silently add a block to
the draw's wait. height and hex are applied together or not at all —
applying a losing header's hex would leave tip_height and tip_header_hex
describing different blocks, and that hex is the draw's entropy source.
The tip never regresses, and `tip_height`/`tip_header_hex` always describe the
same block: they are applied together or not at all, since that hex is where a
draw's entropy comes from. A header that is ignored (no hex, below the tip, the
tip itself again) changes nothing and ends nothing.
Two validation checks guard against a hostile or MITM'd server simply
fabricating a header (B-28), since that header is the draw's sole source of
entropy: it must satisfy the difficulty target it claims for itself, and —
when it's a direct single-block advance from our own current tip, the only
case we can check without a full header chain — it must chain from that
tip's hash. Either failure raises HeaderValidationError rather than
silently ignoring the header, which (via _consume_headers/_run_once) ends
this session the same way a dropped connection would, so run() rotates to
the next configured server instead of continuing to trust this one.
Everything else must be proven, because a forged header steers a draw (B-28):
it must meet the target it claims *and* that target must be no easier than
PLM's powLimit (B-74, ELC-11), and every advance must be linked to the tip we
hold (ELC-12) — a single block by its `prev_hash`, a jump of several by
downloading the headers in between from `client` and checking each. A header that
fails these raises HeaderValidationError, which (via _consume_headers/_run_once)
ends this session like a dropped connection and rotates to the next server.
Two more headers are refused without ending the session, since neither is
evidence of a hostile server the way the above are (B-64): one carrying no
hex, and one at the height we already hold a header for.
A header that merely does not *join* our tip is not hostile in itself: it is a
reorganisation, and is followed back to its fork point inside the last
_HEADER_WINDOW headers (B-74, ELC-23) instead of ending the session on every
server until the next block arrives. `client` is the active connection the
missing headers are fetched from.
"""
height = header["height"]
header_hex = header.get("hex")
@@ -319,7 +328,7 @@ class ElectrumListener:
)
return
if height < self.tip_height:
if self.tip_header_hex is not None and height < self.tip_height:
logger.warning(
"ignoring Electrum header at height %s, below the current tip %s (reorg or server switch?)",
height,
@@ -327,38 +336,141 @@ class ElectrumListener:
)
return
if height == self.tip_height and self.tip_header_hex:
# B-64: a second header for the height we already hold one for. Either the
# same block re-announced (nothing to do) or a competing one — a reorg at
# the tip, or a server swapping out the very hash a draw may be about to
# use. The linkage check above cannot speak to this case at all, since
# there is no height advance to check. Whichever it is, the hash committed
# to for a height is not replaced under us: if ours turns out to be the
# orphan, corroborate_header (B-28) refuses to seed a draw from it and the
# draw waits for a further block instead.
if header_hex != self.tip_header_hex:
logger.warning(
"ignoring a competing header at the current tip height %s "
"(reorg at the tip, or a server disagreeing with the rest)",
height,
)
if self.tip_header_hex is not None and height == self.tip_height:
if header_hex == self.tip_header_hex:
return # the same block announced again
# B-64: a second header for the height we hold one for — a reorg at the tip, or a
# server disagreeing with the rest. Not replaced under us (B-85 follows it,
# ELC-09); corroborate_header refuses to seed a draw from an orphan anyway.
logger.warning(
"ignoring a competing header at the current tip height %s "
"(reorg at the tip, or a server disagreeing with the rest)",
height,
)
return
if not header_meets_its_own_target(header_hex):
raise HeaderValidationError(
f"header at height {height} does not satisfy its own claimed difficulty target"
)
if (
self.tip_header_hex
and height == self.tip_height + 1
and header_prev_hash(header_hex) != header_hex_to_block_hash(self.tip_header_hex)
):
raise HeaderValidationError(
f"header at height {height} does not chain from the current tip (height {self.tip_height})"
f"header at height {height} is not a valid proof of work "
"(its hash misses the target it claims, or the target is easier than the powLimit)"
)
self.tip_height = height
self.tip_header_hex = header_hex
if self.tip_header_hex is None:
await self._adopt_first_header(height, header_hex, client)
return
tip_hash = header_hex_to_block_hash(self.tip_header_hex)
if height == self.tip_height + 1:
if header_prev_hash(header_hex) == tip_hash:
self._extend_window([(height, header_hex)])
return
await self._follow_reorg(height, header_hex, client)
return
# A jump of several blocks (a reconnect, a server that was ahead of us): link it to
# the tip by fetching what lies in between — an unlinked jump used to be accepted
# on the strength of its own difficulty alone.
if client is None:
raise HeaderValidationError(f"cannot link the header at height {height} to the tip: no connection to fetch from")
between = await self._fetch_headers(client, self.tip_height + 1, height - self.tip_height - 1)
chain = [*between, header_hex]
self._check_chained(chain, header_prev_hash(chain[0])) # internal linkage and work only
if header_prev_hash(chain[0]) != tip_hash:
await self._follow_reorg(height, header_hex, client)
return
self._extend_window([(self.tip_height + 1 + i, h) for i, h in enumerate(chain)])
async def _fetch_headers(self, client: ElectrumClient, start: int, count: int) -> list[str]:
"""`count` headers from `start` (blockchain.block.headers, at most
_MAX_HEADERS_PER_REQUEST per request). A server that returns fewer than it must,
or something that is not whole 80-byte headers, ends the session."""
headers: list[str] = []
while len(headers) < count:
wanted = min(count - len(headers), _MAX_HEADERS_PER_REQUEST)
reply = await client.get_block_headers(start + len(headers), wanted)
got = reply.get("count") if isinstance(reply, dict) else None
raw = reply.get("hex") if isinstance(reply, dict) else None
if not isinstance(got, int) or not isinstance(raw, str) or not 0 < got <= wanted or len(raw) != got * 160:
raise HeaderValidationError(
f"the server returned an unusable reply for headers {start + len(headers)}..+{wanted}"
)
headers.extend(raw[i : i + 160] for i in range(0, len(raw), 160))
return headers
@staticmethod
def _check_chained(headers: list[str], prev_hash: str) -> None:
"""Every header carries valid work and each one builds on the one before it, the
first on `prev_hash` (ELC-11, ELC-12). Raises HeaderValidationError."""
for header_hex in headers:
if not header_meets_its_own_target(header_hex):
raise HeaderValidationError("a downloaded header is not a valid proof of work")
if header_prev_hash(header_hex) != prev_hash:
raise HeaderValidationError("the downloaded headers are not a connected chain")
prev_hash = header_hex_to_block_hash(header_hex)
def _extend_window(self, entries: list[tuple[int, str]]) -> None:
"""Make `entries` (consecutive heights, the last the new tip) part of the window,
replacing anything at or above their first height, and move the tip with it."""
first = entries[0][0]
for stale in [h for h in self._window if h >= first]:
del self._window[stale]
self._window.update(entries)
for old in sorted(self._window)[:-_HEADER_WINDOW]:
del self._window[old]
self.tip_height, self.tip_header_hex = entries[-1]
async def _adopt_first_header(self, height: int, header_hex: str, client: ElectrumClient | None) -> None:
"""The process's first header has nothing to join, so it is taken on its work alone
— and the (up to) 9 headers before it are downloaded and must chain into it,
which is what the window of ELC-22 is made of."""
entries: list[tuple[int, str]] = []
if client is not None and height > 0:
count = min(_HEADER_WINDOW - 1, height)
before = await self._fetch_headers(client, height - count, count)
first_prev = header_prev_hash(before[0])
self._check_chained(before, first_prev)
if header_prev_hash(header_hex) != header_hex_to_block_hash(before[-1]):
raise HeaderValidationError(f"the header at height {height} does not chain from the headers before it")
entries = [(height - count + i, h) for i, h in enumerate(before)]
self._window.clear()
self._extend_window([*entries, (height, header_hex)])
async def _follow_reorg(self, height: int, header_hex: str, client: ElectrumClient | None) -> None:
"""The header at `height` does not build on our tip: find where the server's chain
and ours part (the highest window height at which the server's hash is the same as
ours) and, if that is inside the window, adopt the server's branch from there on
(ELC-23). A fork older than the window, or a branch that is not a connected chain
of valid work ending at the announced header, raises HeaderValidationError."""
if client is None:
raise HeaderValidationError(f"the header at height {height} does not join the tip and there is no connection to follow it")
lowest = min(self._window)
ours = [self._window[h] for h in range(lowest, self.tip_height + 1)]
theirs = await self._fetch_headers(client, lowest, len(ours))
fork_height = None
for offset in range(len(ours) - 1, -1, -1):
if header_hex_to_block_hash(theirs[offset]) == header_hex_to_block_hash(ours[offset]):
fork_height = lowest + offset
break
if fork_height is None:
raise HeaderValidationError(
f"the header at height {height} forks more than {_HEADER_WINDOW} blocks below our tip {self.tip_height}"
)
branch = await self._fetch_headers(client, fork_height + 1, height - fork_height)
if branch[-1] != header_hex:
raise HeaderValidationError(f"the server's branch does not end at the header it announced at height {height}")
self._check_chained(branch, header_hex_to_block_hash(self._window[fork_height]))
old_height, old_hex = self.tip_height, self.tip_header_hex
self._extend_window([(fork_height + 1 + i, h) for i, h in enumerate(branch)])
await self._reorg_followed(old_height, old_hex, fork_height)
async def _reorg_followed(self, old_height: int, old_hex: str, fork_height: int) -> None:
logger.warning(
"followed a chain reorganisation: tip %s -> %s, fork at height %s (depth %s)",
old_height,
self.tip_height,
fork_height,
old_height - fork_height,
)
async def _corroborate_majority(
self,
@@ -701,6 +813,8 @@ class ElectrumListener:
return None
if height == self.tip_height and self.tip_header_hex:
return self.tip_header_hex
if height in self._window:
return self._window[height] # validated as it arrived (ELC-22)
async def _ask(client: ElectrumClient) -> str | None:
header = await client.get_block_header(height)
@@ -738,11 +852,11 @@ class ElectrumListener:
return await self._corroborate_majority(_ask, lambda proven: proven, f"merkle proof of {txid} at height {height}")
async def _consume_headers(self, queue: asyncio.Queue) -> None:
async def _consume_headers(self, queue: asyncio.Queue, client: ElectrumClient | None = None) -> None:
while True:
params = await queue.get()
for header in params:
self._apply_header(header)
await self._apply_header(header, client)
# A new block is exactly what the "drawing" phase is waiting on
# (rounds/scheduler.py:_wait_for_next_block) — nudge dashboards to
# refetch instead of waiting for their next poll.