Track every version of a bumped transaction and write the RBF replacement before broadcasting it (B-77, TX-12, TX-13, TX-19, TX-21, DAT-13, DAT-14, DAT-34, DAT-44, DAT-45, DAT-70, DAT-71)
The poller and the reconciler matched current_txid only. If an earlier version of a bumped transaction confirmed it was never seen, and after 6h the row was abandoned: a confirmed bet's participant deleted, a sent withdrawal marked failed, a paid payout rebuilt. bump_fee also broadcast in phase 2 and persisted in phase 3 unconditionally, so a crash or a `database is locked` in between left a transaction on the network that the DB did not know, and an abandon or a confirmation in the meantime was overwritten. - New table pending_transaction_txids (migration a3f5c2d9b7e1; txid unique, index on the owning row; existing rows backfilled with current_txid + raw and the txid their last bump replaced, raw_hex empty). Written with the row by place_bet, request_withdrawal and _trigger_payout; removed with a bet row that a verified refusal deletes (DAT-38). - bump_fee in four phases: read, build and sign, commit the replacement as a candidate version, broadcast (verdict of B-76, fanned out when accepted; an explicit refusal raises RbfError and leaves the row), then switch the row over with `UPDATE ... WHERE status='pending' AND current_txid=<replaced>`. Zero rows matched is a logged conflict: nothing is overwritten and the candidate stays recorded. An uncertain broadcast adopts the replacement. A retry rebuilds the same candidate without tripping the unique txid. - poll_once and reconcile_once look for every txid of a row. When the version found is not the current one, broadcast.adopt_version makes it current (raw tx included) and retargets spent_txid, bet_txid, withdrawals.txid and payout_txid as a bump does, before the confirmation handler runs. _release_inputs frees inputs claimed by any version of the row. - RbfBumper takes the listener as `network`. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
2b6ddb019b
commit
f936d6660c
23 files changed
+836
-130
No files matched your search
+110
-19
@@ -6,13 +6,23 @@ from embit import script
|
||||
from embit.psbt import PSBT
|
||||
from embit.transaction import Transaction, TransactionInput, TransactionOutput
|
||||
from embit.finalizer import finalize_psbt
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
from app.db.base import begin_immediate
|
||||
from app.db.models import PendingTransaction, Round, RoundParticipant, User, UtxoEvent, Withdrawal
|
||||
from app.db.models import (
|
||||
PendingTransaction,
|
||||
PendingTransactionTxid,
|
||||
Round,
|
||||
RoundParticipant,
|
||||
User,
|
||||
UtxoEvent,
|
||||
Withdrawal,
|
||||
)
|
||||
from app.electrum.client import ElectrumClient
|
||||
from app.rounds.config import get_round_config
|
||||
from app.tx.outcome import BroadcastOutcome, TransactionNetwork, broadcast_with_verdict
|
||||
from app.tx.versions import record_version
|
||||
from app.wallet.hd import derive_round_pool_key, derive_user_key
|
||||
from app.wallet.plm_network import PLM_MAINNET
|
||||
from app.wallet.psbt_builder import MAX_FEE_RATE_SAT_VB, RBF_SEQUENCE, estimate_vsize
|
||||
@@ -93,28 +103,42 @@ def _find_change_output(tx: Transaction, change_address: str) -> int | None:
|
||||
|
||||
|
||||
async def bump_fee(
|
||||
session_factory: async_sessionmaker, client: ElectrumClient, pending_id: int
|
||||
session_factory: async_sessionmaker,
|
||||
client: ElectrumClient,
|
||||
pending_id: int,
|
||||
network: TransactionNetwork | None = None,
|
||||
) -> str | None:
|
||||
"""Rebuild pending_transaction `pending_id`'s transaction with a higher fee
|
||||
(same inputs, same recipient outputs, the extra fee taken from the change
|
||||
output) and rebroadcast. Returns the new txid, or None if there was nothing
|
||||
to do (the row is gone or already left "pending" — a normal race with
|
||||
confirmation, not an error).
|
||||
confirmation, not an error, or a conflict recorded below).
|
||||
|
||||
Three phases, so no DB session is held across the network calls this needs
|
||||
Four phases, so no DB session is held across the network calls this needs
|
||||
(one get_transaction per input, then a broadcast) — the same shape used
|
||||
elsewhere for exactly this reason (B-18, rounds/scheduler.py:_trigger_payout;
|
||||
B-31, electrum/listener.py:refresh_user) and now here too (B-40): read what's
|
||||
needed and close the session, do the chain work, then reopen to persist.
|
||||
|
||||
B-77 (TX-12, TX-13, DAT-34): the replacement is **written before it is broadcast**.
|
||||
It used to be sent in the chain phase and recorded afterwards, unconditionally, so
|
||||
a crash or a `database is locked` in between left a transaction on the network
|
||||
that the database did not know, and an abandon or a confirmation that happened in
|
||||
the meantime was silently overwritten. Now the signed replacement is committed as
|
||||
a *candidate* version (pending_transaction_txids) first; a candidate that is never
|
||||
broadcast is harmless, and one that is — and then confirms — is recognised like any
|
||||
other version. After the broadcast the row is switched over with a conditional
|
||||
UPDATE (status still pending, current_txid still the one replaced); zero rows
|
||||
matched is a conflict: the row is left alone, the candidate stays recorded, and
|
||||
the reconciler resolves it.
|
||||
|
||||
Only handles the common case: exactly one change output paying back to the
|
||||
tx's own sender address, large enough to absorb the increase. If there's no
|
||||
such output (e.g. an exact-amount bet with no change), this raises RbfError —
|
||||
bumping such a tx would require selecting additional inputs, which isn't
|
||||
implemented for the MVP; it needs manual operator intervention. Also raises
|
||||
RbfError, rather than bumping, once the row is already at MAX_FEE_RATE_SAT_VB
|
||||
(B-32) — the reconciler abandons it if it never confirms (B-27), instead of
|
||||
this retrying an ever-higher fee forever.
|
||||
(B-32), and when the network explicitly refuses the replacement (TX-24).
|
||||
"""
|
||||
# --- Phase 1: read what's needed, close the session before any network call ---
|
||||
async with session_factory() as session:
|
||||
@@ -131,9 +155,10 @@ async def bump_fee(
|
||||
kind = pending.kind
|
||||
current_fee_rate = pending.fee_rate_sat_vb
|
||||
raw_tx_hex = pending.raw_tx_hex
|
||||
old_txid = pending.current_txid
|
||||
signing_key, own_script, own_address = await _signing_context(session, kind, pending.user_id, pending.round_id)
|
||||
|
||||
# --- Phase 2: chain reads, signing, and the broadcast — no DB session open ----
|
||||
# --- Phase 2: chain reads and signing — no DB session open, nothing sent yet ---
|
||||
old_tx = Transaction.parse(bytes.fromhex(raw_tx_hex))
|
||||
input_amounts = [await _prevout_amount(client, vin) for vin in old_tx.vin]
|
||||
total_in = sum(input_amounts)
|
||||
@@ -178,27 +203,69 @@ async def bump_fee(
|
||||
|
||||
raw_hex = final_tx.serialize().hex()
|
||||
new_txid = final_tx.txid().hex()
|
||||
await client.broadcast(raw_hex)
|
||||
|
||||
# --- Phase 3: persist the outcome ----------------------------------------------
|
||||
# --- Phase 3: record the replacement as a candidate, *then* broadcast ----------
|
||||
async with session_factory() as session:
|
||||
await begin_immediate(session) # B-88
|
||||
pending = await session.get(PendingTransaction, pending_id)
|
||||
old_txid = pending.current_txid
|
||||
pending.replaced_by_txid = old_txid # points backwards: what current_txid replaced
|
||||
pending.current_txid = new_txid
|
||||
pending.raw_tx_hex = raw_hex
|
||||
if pending is None or pending.status != "pending" or pending.current_txid != old_txid:
|
||||
logger.info("pending_transaction %s changed while its bump was being built; skipping", pending_id)
|
||||
return None
|
||||
await record_version(session, pending, new_txid, raw_hex) # B-77 (TX-12, DAT-71)
|
||||
await session.commit()
|
||||
|
||||
verdict = await broadcast_with_verdict(client, network, raw_hex, new_txid) # B-76: also fans out
|
||||
if verdict.outcome is BroadcastOutcome.REFUSED:
|
||||
# The network explicitly refused the replacement and no server knows it: the row
|
||||
# is unchanged and keeps the version that is on the network (TX-24). The
|
||||
# candidate stays recorded — it was never relayed, so nobody will confirm it.
|
||||
raise RbfError(f"pending_transaction {pending_id}: the network refused the replacement: {verdict.reason}")
|
||||
# ACCEPTED, or UNCERTAIN: the replacement may well be on the network, so the row
|
||||
# follows it (TX-12); the old version stays recorded and is still recognised should
|
||||
# it be the one that confirms.
|
||||
|
||||
# --- Phase 4: switch the row over, conditionally --------------------------------
|
||||
async with session_factory() as session:
|
||||
await begin_immediate(session) # B-88
|
||||
# The *actual* resulting rate, not target_fee_rate: when the BIP125-minimum
|
||||
# floor above raised fee_delta past the naive target, the tx now pays more
|
||||
# than target_fee_rate implied. Recording the true rate keeps the next bump's
|
||||
# arithmetic honest instead of drifting from what's really being paid.
|
||||
pending.fee_rate_sat_vb = (old_fee + fee_delta) // vsize
|
||||
pending.attempt_count += 1
|
||||
# last_broadcast_at, not broadcast_at (B-27): broadcast_at must stay the *first*
|
||||
# broadcast, since reconcile.py's abandon-after-N-hours grace period is measured
|
||||
# from it — overwriting it here used to reset that clock on every bump, so a
|
||||
# repeatedly-bumped-but-never-mined tx was never abandoned.
|
||||
pending.last_broadcast_at = datetime.now(timezone.utc)
|
||||
result = await session.execute(
|
||||
update(PendingTransaction)
|
||||
.where(
|
||||
PendingTransaction.id == pending_id,
|
||||
PendingTransaction.status == "pending",
|
||||
PendingTransaction.current_txid == old_txid,
|
||||
)
|
||||
.values(
|
||||
replaced_by_txid=old_txid, # points backwards: what current_txid replaced
|
||||
current_txid=new_txid,
|
||||
raw_tx_hex=raw_hex,
|
||||
fee_rate_sat_vb=(old_fee + fee_delta) // vsize,
|
||||
attempt_count=PendingTransaction.attempt_count + 1,
|
||||
last_broadcast_at=datetime.now(timezone.utc),
|
||||
)
|
||||
.execution_options(synchronize_session=False)
|
||||
)
|
||||
if result.rowcount != 1:
|
||||
# Confirmed, abandoned or already re-bumped while we were broadcasting. The
|
||||
# row is not overwritten; the candidate is still recorded, and confirmation
|
||||
# and reconciliation look for it like any other version (TX-13).
|
||||
await session.rollback()
|
||||
logger.warning(
|
||||
"bump of pending_transaction %s: the row changed during the broadcast (replaced %s by %s); "
|
||||
"left to the reconciler",
|
||||
pending_id,
|
||||
old_txid,
|
||||
new_txid,
|
||||
)
|
||||
return None
|
||||
pending = await session.get(PendingTransaction, pending_id)
|
||||
await _retarget_txid_references(session, pending, old_txid, new_txid)
|
||||
await session.commit()
|
||||
|
||||
@@ -206,6 +273,29 @@ async def bump_fee(
|
||||
return new_txid
|
||||
|
||||
|
||||
async def adopt_version(session: AsyncSession, pending: PendingTransaction, txid: str) -> None:
|
||||
"""The network confirmed (or holds) a version of this row other than the current one
|
||||
— an earlier version of a bumped transaction, or a candidate whose switch-over never
|
||||
happened (B-77, TX-19, DAT-14). `current_txid` and `raw_tx_hex` become that
|
||||
version's, and the txid is propagated everywhere it is recorded, exactly as after a
|
||||
bump; without it the inputs of a replaced transaction would no longer be
|
||||
recognised as the row's (DAT-13)."""
|
||||
if txid == pending.current_txid:
|
||||
return
|
||||
version = await session.scalar(
|
||||
select(PendingTransactionTxid).where(
|
||||
PendingTransactionTxid.txid == txid, PendingTransactionTxid.pending_transaction_id == pending.id
|
||||
)
|
||||
)
|
||||
old_txid = pending.current_txid
|
||||
pending.replaced_by_txid = old_txid
|
||||
pending.current_txid = txid
|
||||
if version is not None and version.raw_hex:
|
||||
pending.raw_tx_hex = version.raw_hex # empty only for a migration-reconstructed txid
|
||||
await _retarget_txid_references(session, pending, old_txid, txid)
|
||||
logger.warning("pending_transaction %s: version %s found on the network, replacing %s", pending.id, txid, old_txid)
|
||||
|
||||
|
||||
async def _retarget_txid_references(
|
||||
session: AsyncSession, pending: PendingTransaction, old_txid: str, new_txid: str
|
||||
) -> None:
|
||||
@@ -247,9 +337,10 @@ async def _retarget_txid_references(
|
||||
|
||||
|
||||
class RbfBumper:
|
||||
def __init__(self, session_factory: async_sessionmaker, get_client):
|
||||
def __init__(self, session_factory: async_sessionmaker, get_client, network: TransactionNetwork | None = None):
|
||||
self._session_factory = session_factory
|
||||
self._get_client = get_client
|
||||
self._network = network # B-76/B-77: verifies and fans out a replacement
|
||||
|
||||
async def run(self) -> None:
|
||||
while True:
|
||||
@@ -274,7 +365,7 @@ class RbfBumper:
|
||||
|
||||
for pending_id in due_ids:
|
||||
try:
|
||||
await bump_fee(self._session_factory, client, pending_id)
|
||||
await bump_fee(self._session_factory, client, pending_id, network=self._network)
|
||||
except RbfError:
|
||||
logger.exception("could not bump pending_transaction %s", pending_id)
|
||||
except Exception:
|
||||
|
||||
Reference in new issue
Block a user