This changes various tests in minor ways:
1. The "l2" secret key in tests/plugins/channeld_fakenet.c is updated.
2. The decompressed gossip data node id needs changing.
3. The coinmoves order changes in bookkeeper for anchors.
4. Various harcoded gossip constants change.
5. Some hardcoded makesecret results change.
6. zeroconf tests which hardcoded node ids change.
7. Arbitrary rune strings change.
8. A log message which uses node ids changes.
This covers the other corner case, where we crash before actually
signing and sending the PSBT. We can spot this because the channel is
in AWAITING_LOCKIN and we have a PSBT, but it's not signed yet.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: we now re-transmit unseen funding transactions on startup, for more robustness.
One issue we have in CI is reconnection races: if an incoming
connection arrives while an outgoing one is negotiated, we close the
outgoing one and issue a disconnect, which fails any connect attempts.
By sending a "reconnected" message instead of disconnect/connect we
can avoid disturbing in-progress connection attempts which happens in CI
quite a bit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/4873
In particular, we used to get upset when a peer accepts our channel,
if it was too small! We should do reasonable checks first.
We no longer try to send requests to delay for 2017 blocks though,
so remove that test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: trying to create a channel below our own min-capacity-sat will now fail before asking the peer, not with an error blaming the peer when they accept!
Prior to it being compulsory, these daemons would need a default value. Now it's
always required, it's clearer if it's always told.
There's no "default_channel_type" now everyone has to specify channel_type either,
so rename it to "desired_channel_type" and put it in lightningd specifically.
Note that the channel_type can have options added: either option_scid_alias or option_zeroconf.
This results in a slight behavior change: we will get type zeroconf even if we didn't ask for it, if they gave it to us.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: fundchannel / fundchannel_start returned `channel_type` will include option_zeroconf if it was implied by a 0 minimum_depth, even if we didn't explicitly ask for a zero conf channel.
When peer backup is enabled by default, it puts things in the datastore, breaking
this assumption. Narrow the test to examine the specific funder directory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
channeld complains if it sees blocks going backwards, but that's
actually expected: allow those log messages:
```
2025-05-13T10:28:56.3283346Z _______________ ERROR at teardown of test_v2_replay_bookkeeping ________________
...
2025-05-13T10:28:56.3301378Z request.node.has_errors = True
2025-05-13T10:28:56.3301833Z > raise ValueError(str(errors))
2025-05-13T10:28:56.3302268Z E ValueError:
2025-05-13T10:28:56.3303425Z E Node errors:
2025-05-13T10:28:56.3303923Z E - lightningd-1: had BROKEN messages
2025-05-13T10:28:56.3304404Z E Global errors:
...
2025-05-13T10:28:56.4619742Z lightningd-1 2025-05-13T10:24:38.112Z **BROKEN** 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-channeld-chan#2: current blockheight 109 less than last 111
2025-05-13T10:28:56.4620188Z lightningd-2 2025-05-13T10:24:38.112Z DEBUG 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-channeld-chan#2: peer_in WIRE_SHUTDOWN
2025-05-13T10:28:56.4620600Z lightningd-1 2025-05-13T10:24:38.112Z DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-channeld-chan#2: Trying commit
2025-05-13T10:28:56.4621119Z lightningd-2 2025-05-13T10:24:38.112Z INFO 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#2: State changed from CHANNELD_NORMAL to CHANNELD_SHUTTING_DOWN
2025-05-13T10:28:56.4621623Z lightningd-1 2025-05-13T10:24:38.112Z **BROKEN** 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-channeld-chan#2: current blockheight 109 less than last 111
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
```
2025-05-12T04:37:57.4824338Z else:
2025-05-12T04:37:57.4824626Z # It will forget the older one.
2025-05-12T04:37:57.4825508Z > l2.daemon.wait_for_log(r"UNUSUAL {}-chan#1: Forgetting channel: It has been {} blocks without the funding transaction ".format(l1.info['id'], blocks + 1))
```
This fails because l3 can still transmit, so the second channel can confirm in time.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Once we speed things up, this can receive COMMITMENT_SIGNED, and hence succeed. Let's
drop on receive, not send!
```
2025-05-06T06:07:51.5997333Z # Peers try RBF, break on initial COMMITMENT_SIGNED
2025-05-06T06:07:51.5997855Z bump = l1.rpc.openchannel_bump(chan_id, chan_amount, initpsbt['psbt'])
2025-05-06T06:07:51.5998214Z > with pytest.raises(RpcError):
2025-05-06T06:07:51.5998542Z E Failed: DID NOT RAISE <class 'pyln.client.lightning.RpcError'>
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Once we speed things up, we can *receive* commitment_signed before we disconnect
due to sending ours: in this case, we *will* have a scratch tx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
fundwallet generates a block: if l2 sees one block and not the other, it can
consider the time on the first fundchannel to be one block earlier than expected:
```
2025-05-05T21:05:33.6260600Z E TimeoutError: Unable to find "[re.compile('UNUSUAL 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Forgetting channel: It has been 51 blocks without the funding transaction ')]" in logs.
...
2025-05-05T21:05:33.7179461Z lightningd-2 2025-05-05T20:44:26.141Z UNUSUAL 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Forgetting channel: It has been 52 blocks without the funding transaction 5a18d113f53df8b205ab679924babde4068f2d1876d34edc43701bf92b8ff13f getting deeply confirmed. We are fundee and can forget channel without loss of funds.
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need one byte for the number of witness elements. Some callers added it themselves,
but it's always needed. So document and fix the callers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We actually need to mine another block, so that lightningd considers both
channels candidates for forgetting (and forgets the older one).
Reported-by: daywalker90
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
fundwallet() actually mines a block, putting our count out. If we see
both blocks at once, we will say "52" blocks instead of "51":
```
2025-05-02T05:28:40.5315155Z have_forgotten = l2.daemon.is_in_log(
2025-05-02T05:28:40.5315650Z r"UNUSUAL {}-chan#1: Forgetting channel: It has been 51 blocks without the funding transaction ".format(l1.info['id'])
2025-05-02T05:28:40.5316105Z )
2025-05-02T05:28:40.5316263Z
2025-05-02T05:28:40.5316417Z if dopay:
2025-05-02T05:28:40.5316616Z assert not have_forgotten
2025-05-02T05:28:40.5317056Z assert set([c['peer_id'] for c in l2.rpc.listpeerchannels()["channels"]]) == set([l1.info['id'], l3.info['id']])
2025-05-02T05:28:40.5317477Z else:
2025-05-02T05:28:40.5317662Z > assert have_forgotten
2025-05-02T05:28:40.5317887Z E assert None
```
```
0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Forgetting channel: It has been 52 blocks without the funding transaction 0bb0579df6b1d983dda49dad47513afc71696c9d5bea3c8b955ba4b76bb053de getting deeply confirmed. We are fundee and can forget channel without loss of funds.
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Michael of Boltz told me this long ago, that when fees spiked some of their clients' opens got stuck. After two weeks their node forgot them, and when fees came back down the opens still failed. Unfortunately, I can't see an issue for this!
We can give some grace: we don't want to waste too many resources, but 100 channels is nothing.
The test needs to be adjusted to have *two* slow open channels, now.
Changelog-Changed: Protocol: we won't forget still-opening channels after 2016 blocks, unless there are more than 100.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Modified by Rusty:
- Simplify (we only need to suppress l1's txs, since it opens)
- Use synchronize_blockheight instead of log messages to ensure l2 has processed the block
- Use listpeerchannels instead of log messages for balance (should be more robust against changes)
- Open-code assertions for better debugging if they're wrong.
Doesn't always die messily, it seems?
```
@unittest.skipIf(TEST_NETWORK != 'regtest', 'elementsd doesnt yet support PSBT features we need')
@pytest.mark.openchannel('v2')
def test_rbf_reconnect_tx_construct(node_factory, bitcoind, chainparams):
disconnects = ['=WIRE_TX_ADD_INPUT', # Initial funding succeeds
'-WIRE_TX_ADD_INPUT',
'+WIRE_TX_ADD_INPUT',
'-WIRE_TX_ADD_OUTPUT',
'+WIRE_TX_ADD_OUTPUT',
'-WIRE_TX_COMPLETE',
'+WIRE_TX_COMPLETE',
'-WIRE_COMMITMENT_SIGNED',
'+WIRE_COMMITMENT_SIGNED']
l1, l2 = node_factory.get_nodes(2,
opts=[{'disconnect': disconnects,
'may_reconnect': True,
'dev-no-reconnect': None},
{'may_reconnect': True,
'dev-no-reconnect': None,
'broken_log': 'dualopend daemon died before signed PSBT returned'}])
l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
amount = 2**24
chan_amount = 100000
bitcoind.rpc.sendtoaddress(l1.rpc.newaddr()['bech32'], amount / 10**8 + 0.01)
bitcoind.generate_block(1)
# Wait for it to arrive.
wait_for(lambda: len(l1.rpc.listfunds()['outputs']) > 0)
res = l1.rpc.fundchannel(l2.info['id'], chan_amount)
chan_id = res['channel_id']
vins = bitcoind.rpc.decoderawtransaction(res['tx'])['vin']
assert(only_one(vins))
prev_utxos = ["{}:{}".format(vins[0]['txid'], vins[0]['vout'])]
# Check that we're waiting for lockin
l1.daemon.wait_for_log(' to DUALOPEND_AWAITING_LOCKIN')
# rbf the lease with a higher amount
rate = int(find_next_feerate(l1, l2)[:-5])
# We 4x the feerate to beat the min-relay fee
next_feerate = '{}perkw'.format(rate * 4)
# Initiate an RBF
startweight = 42 + 172 # base weight, funding output
initpsbt = l1.rpc.utxopsbt(chan_amount, next_feerate, startweight,
prev_utxos, reservedok=True,
excess_as_change=True)
# Run through TX_ADD wires
for d in disconnects[1:-4]:
l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
with pytest.raises(RpcError):
l1.rpc.openchannel_bump(chan_id, chan_amount, initpsbt['psbt'])
wait_for(lambda: l1.rpc.getpeer(l2.info['id'])['connected'] is False)
# The first TX_COMPLETE breaks
l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
bump = l1.rpc.openchannel_bump(chan_id, chan_amount, initpsbt['psbt'])
with pytest.raises(RpcError):
update = l1.rpc.openchannel_update(chan_id, bump['psbt'])
wait_for(lambda: l1.rpc.getpeer(l2.info['id'])['connected'] is False)
l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
# l1 should remember, l2 has forgotten
# l2 should send tx-abort, to reset
l2.daemon.wait_for_log(r'tx-abort: Sent next_funding_txid .* doesn\'t match ours .*')
l1.daemon.wait_for_log(r'Cleaned up incomplete inflight')
# abort doesn't cause a disconnect
assert l1.rpc.getpeer(l2.info['id'])['connected']
# The next TX_COMPLETE break (both remember) + they break on the
# COMMITMENT_SIGNED during the reconnect
bump = l1.rpc.openchannel_bump(chan_id, chan_amount, initpsbt['psbt'])
with pytest.raises(RpcError):
update = l1.rpc.openchannel_update(chan_id, bump['psbt'])
wait_for(lambda: l1.rpc.getpeer(l2.info['id'])['connected'] is False)
l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
l2.daemon.wait_for_logs([r'Got dualopend reestablish',
r'No commitment, not sending our sigs'])
l1.daemon.wait_for_logs([r'Got dualopend reestablish',
r'No commitment, not sending our sigs',
r'dev_disconnect: -WIRE_COMMITMENT_SIGNED',
'peer_disconnect_done'])
assert not l1.rpc.getpeer(l2.info['id'])['connected']
l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
# COMMITMENT_SIGNED disconnects *during* the reconnect
# We can't bump because the last negotiation is in the wrong state
with pytest.raises(RpcError, match=r'Funding sigs for this channel not secured'):
l1.rpc.openchannel_bump(chan_id, chan_amount, initpsbt['psbt'])
# l2 reconnects, but doesn't have l1's commitment
> l2.daemon.wait_for_logs([r'Got dualopend reestablish',
r'No commitment, not sending our sigs',
# This is a BROKEN log, it's expected!
r'dualopend daemon died before signed PSBT returned'])
tests/test_opening.py:944:
...
> raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E TimeoutError: Unable to find "[re.compile('dualopend daemon died before signed PSBT returned')]" in logs.
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
As the first user of a persistent layer, this tripped tests which
assumed the datastore would be empty!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It only works on BOLT11, and has long been replaced by the more
generic "decode".
Removing it will stop the confusion!
(Note: documentation claims it was introduced in 23.08, but that was
wrong, as it's been in CLN since the beginning).
[ Fixup from: niftynei <niftynei@gmail.com> ]
Fixes: https://github.com/ElementsProject/lightning/issues/6419
Changelog-Deprecated: JSON-RPC: `decodepay`: use `decode`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Document and enforce the --experimental-anchors deprecation, which was somehow missed in v24.02
Changelog-Deprecated: Config: the --experimental-anchors option is ignored (on by default since v24.02).
This is a difficult transition for us: this string appears in channel
types. We make the transition now in the understanding that it will
be more difficult in future.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `listpeers` `features` array string "option_anchors_zero_fee_htlc_tx": use "option_anchors" (spec renamed it).
Changelog-Added: JSON-RPC: `listpeers` `features` array string uses "option_anchors" for feature 22/23, following renaming in BOLT 9.
Changelog-Changed: JSON-RPC: `listclosedchannels`, `listpeerchannels`, `openchannel_update`, `openchannel_init`, `fundchannel`, `fundchannel_start` and `multifundchannel`: `channel_type` array `names` now contains "anchors" instead of "anchors_zero_fee_htlc_tx".
Changelog-Changed: lightningd: `--list-features-only` now lists "option_anchors" instead of "option_anchors_zero_fee_htlc_tx".
Rather than `allow_broken_log`, we have `broken_log` which is a regex
indicating what log lines are expected. This tightens our tests
significantly, as it will catch *unexpected* brokenness.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I was trying to debug test_zeroconf_open and getting very confused.
The reason: l0 is lightning-1, l1 is lightning-2, etc! And there are
two other tests where an l0 has been added at the front: fix them all
to avoid future confusion!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We still want to test non-anchor channels, as we still support them, but
we've made it non-experimental. To test non-anchor channels, we
use dev-force-features: -23.
Changelog-Added: Protocol: `option_anchors_zero_fee_htlc_tx` enabled, no longer experimental.
Changelog-Changed: Config: `experimental-anchors` now does nothing (it's enabled by default).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Header from folded patch 'fixup!_options__make_anchors_enabled_by_default,_ignore_experimental-anchors.patch':
fixup! options: make anchors enabled by default, ignore experimental-anchors.
This commit is a bit messy, but it tries to do the minimal switchover.
Some tests change, so those are included here.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
After this, the invoice (correctly!) gives the zeroconf channel as a routehint,
so this test fails. Simple workaround: make invoice before creating zeroconf
channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
For example, lnprototest got the error 'You gave bad parameters: Did not support channel_type ' which doesn't make it clear that it's rejecting the empty channel type.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Let's tell the caller what channel_type they got!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel`, `multifundchannel`, `fundchannel_start` and `openchannel_init`: new field `channel_type`.
As suggested in https://github.com/lightning/bolts/pull/1092.
We still support channels opened without it, but you can no longer open new ones without it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_gossip_queries` is now required (advertized by all but 16 nodes)
We're about to make static_remotekey compulsory, but we still want to
do tests for pre-existing channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
