Avoids guessing what the timeout should be, use a file trigger. This
is more optimal, and should reduce a flake in test_sql under valgrind.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was added in 24.05, but LND since 0.18.3 no longer ever creates
such onions, and even that version (September 2024) is now a long way
behind.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: we no longer support legacy onions (never sent by LND >= 0.18.3, which was the last)
If we can't decode something, and it decodes as a rune (and all bech32
strings do!), then we would usually just complain it was a malformed
rune. Be a big more useful, when the parameter looks like somthing else.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `decode` is now more informative with malformed strings (won't claim everything is a malformed rune!).
This changes various tests in minor ways:
1. The "l2" secret key in tests/plugins/channeld_fakenet.c is updated.
2. The decompressed gossip data node id needs changing.
3. The coinmoves order changes in bookkeeper for anchors.
4. Various harcoded gossip constants change.
5. Some hardcoded makesecret results change.
6. zeroconf tests which hardcoded node ids change.
7. Arbitrary rune strings change.
8. A log message which uses node ids changes.
These are explicitly written to check that the values don't accidentally change,
which applies to both old and new styles.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means:
1. downgrade changes (we no longer fail due to node biases).
2. various deprecations no longer are
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Seems like sleep(1) isn't always enough. Give in and put a log
message there, and use that:
```
waitfut = executor.submit(l2.rpc.wait, subsystem='forwards', indexname='deleted', nextvalue=1)
time.sleep(1)
l2.rpc.delforward(scid12, 1, 'failed')
waitres = waitfut.result(TIMEOUT)
> assert waitres == {'subsystem': 'forwards',
'deleted': 1,
'forwards': {'in_channel': scid12,
'in_htlc_id': 1,
'status': 'failed'}}
E AssertionError: assert {'subsystem': 'forwards', 'deleted': 1} == {'subsystem': 'forwards', 'deleted': 1, 'forwards': {'in_channel': '103x2x0', 'in_htlc_id': 1, 'status': 'failed'}}
E
E Common items:
E {'deleted': 1, 'subsystem': 'forwards'}
E Right contains 1 more item:
E {'forwards': {'in_channel': '103x2x0', 'in_htlc_id': 1, 'status': 'failed'}}
E
E Full diff:
E {
E 'deleted': 1,
E - 'forwards': {
E - 'in_channel': '103x2x0',
E - 'in_htlc_id': 1,
E - 'status': 'failed',
E - },
E 'subsystem': 'forwards',
E }
tests/test_misc.py:3599: AssertionError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. It was flaky, probably because it didn't wait for the remote update_channel.
2. Rusty applied a fix in 5f664dac77, not clear if it worked.
3. Christian disabled it altogether in 23ce9a947d.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since this was written, we now test if remote side would get into this situation and stop
it from happening, so the test doesn't work any more.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When paying through a direct channel, direct_pay_override() creates a
route bypassing the normal routing path, which skips the CLTV budget
check in payment_getroute(). This allows payments to succeed even when
maxdelay is set below the required min_final_cltv_expiry.
Add a check in direct_pay_override() to verify the required CLTV
doesn't exceed cltv_budget before using the direct channel shortcut.
If it exceeds, skip the direct channel and let normal routing handle
the failure with a proper error message.
Fixes: #8609
Changelog-Fixed: pay: `maxdelay` parameter now enforced for direct channel payments
The keysend plugin previously used `> 1023` as the cutoff for
description length when inserting an invoice. This was
inconsistent with invoice.c, which enforces the BOLT11 description
field limit defined in `common/bolt11.h`.
This patch switches to using `BOLT11_FIELD_BYTE_LIMIT` directly.
As a result, keysend no longer fails on descriptions between
641–1023 bytes, which previously caused unexpected failures.
A new regression test (`test_keysend_description_size_limit`)
exercises boundary cases just below, at, and above the limit.
Changelog-Fixed: Protocol: `keysend` with descriptions of length 640-1023 bytes fixed.
Signed-off-by: Wes Payne <noblepayne@noblepayne.com>
This commit introduces a new field `invoice_msat` to the htlc_accepted
hook. If this field is specified it will replace the amount of the
invoice that belongs to the payment_hash of the HTLC on internal checks.
This is useful in scenarios where we actually expect a smaller amount
than initially specified in an invoice.
Changelog-Changed: Plugins: `htlc_accepted` hook can now override the
expected total amount of the invoice that belongs to the HTLC.
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
If we failed after we register (e.g. channeld not available), we don't
mark it failed. We shouldn't register until we've definitely created
the htlc.
Changelog-Fixed: `xpay` would sometimes leave payment parts status `pending` in failure cases (as seen in listpays or listsendpays).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/8629
In this case, we make an immediately-expiring invoice. This correctly blocks
any successive requests for invoices, as per the spec requirement.
This means we have to handle invoice_requests without reply_path, amounts
or quantity *if* they specify invreq_recurrence_cancel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We added this to the recurrence spec: the offer expiration only applies to the
first request, not subsequent ones.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changes:
* Fields renumbered to their draft values + billion.
* offer_recurrence now comes in compulsory or optional (backwards compat) flavors.
* `proportional_amount` is now inside `offer_recurrence_base` not `offer_recurrence_paywindow`.
* New field `invreq_recurrence_cancel`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Draft specification for recurring offers changed: old recurring offers will no longer work.
Offer_absolute_expiry should be used if you want to require starting at the start.
Changelog-EXPERIMENTAL: Protocol: BOLT 12 recurrence `start_any_period` removed, use expiry if you need to restrict when they can start using the offer.
This is reliable, meaning we should never get replayed events.
We have to reference count to make sure all commands are complete,
before we return. In particular, annotating with descriptions can
involve several calls to list commands. We need to give them the
results *after* this is all complete.
test_bookkeeping_descriptions() relied on log messages from
notifications, which now only happen when a command is called. This
changes the test a bit.
Since we no longer subscribe to the balance_snapshot event, we
need to create the wallet account at initialization, as callers
expect it to exist.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When we don't have our own db, we're going to need to keep this information
in memory (and the datastore). As a first step, simply cache it in memory
and still write through to the db.
This introduces some changes:
1. The account structures are not temporary, but in the hash table (so don't steal them).
2. test_forward_pad_fees_and_cltv assumed ordering, which was a latent bug.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We didn't remove them from the graph if they were disconnected, leading to us trying to send an onion message to it:
```
➜ ~ lightning-cli fetchinvoice lno1pg7y7s69g98zq5rp09hh2arnypnx7u3qvf3nzut68pmkcet2xdm8s7ngw5ux2am4wqekwmtkddkkk7nyv45ryefexum82anr8phpp6qrse80qf0aara4slvcjxrvu6j2rp5ftmjy4yntlsmsutpkvkt6878sy4xtpdc0j42xp72an4cl0unvzm7ckx2e6ltlzlgeekh0hhe0mgs2qgpetp66ufc9pwln9gmyv0gk3ndpqvvtynef8adzm3lxv0astkxunjcqx0fsq8kmx5cz574ft45vcweaf3tffp90fjdfl9gkdh4s6xfxaz7srtmsju9gnr3xerjhjqw4xtsmp55f4jjsxxrch703kx020l6wn2ttcd24w5h8f2lz723lk2kpa8ftkernz7h2qqkd3zecz2jmhucwh555xf0np4w45zm8tlka4ktw6d3ne4l9u678y37d24xjcy3el53faulcu5tzzquxrxvcyrvzgk7tyjeapt8wu858m2msgdznhqxl8fps04lef9dc9c 1sat
{
"code": -1,
"message": "onion msg: unknown next peer 024b9a1fa8e006f1e3937f65f66c408e6da8e1ca728ea43222a7381df1cc449605"
}
```
Fixes: https://github.com/ElementsProject/lightning/issues/8225
Changelog-Fixed: JSON-RPC: `fetchinvoice` is now more reliable.
Make the payment secret field ('s') mandatory for BOLT11 payment requests,
implementing the requirement specified in BOLT11 spec PR #1242
(https://github.com/lightning/bolts/pull/1242).
This security enhancement prevents payment probing attacks by requiring
all invoices to include payment secrets. Changes include:
1. Adding validation in bolt11_decode_nosig() to reject invoices without
the 's' field
2. Adding payment secrets to all test vectors
3. Updating expected encoded values in test cases to include payment secrets
4. Adding a specific test case that verifies proper rejection of invoices
missing the payment secret field
Changelog-Changed: Made payment secret ('s' field) mandatory in BOLT11 payment requests for improved security.
Sangbida and I traced back through ancient history: when the pay plugin was introduced
in 0.9.0 (2019!) it already used the amount_msat parameter (then called `msatoshi`),
so this case effectively "never happens".
But we added a test for it just in case.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need to wait for exiting startup mode *before* closing channel, so it doesn't ignore
the disconnection of l3, otherwise we don't get a disabled update, and timeout waiting
at line 330:
wait_for(lambda: not l1.is_channel_active(chanid2))
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: sendonion: a new paramter total_amount_msat to make MPP payments with sendpay and sendonion compatible.
[ Reordered to put new parameter at the end --RR ]
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
Our CORK logic was wrong, and it's better to use Nagle anyway:
we disable Nagle just before sending timing-critical messages.
Time for 100 (failed) payments:
Before:
148.8573575
After:
10.7356977
Note this revealed a timing problem in test_reject_invalid_payload: we would
miss the cause of the sendonion failure, and waitsendpay would be called
*after* it had failed, so simply returns "Payment failure reason unknown".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: Removed 200ms latency from sending commit/revoke messages.
It makes the schema simpler, and indeed, expressable by GRPC.
Changelog-Added: JSON-RPC: `wait` now has separate `invoices`, `forwards` and `sendpays` objects for each subsystem.
Changelog-Deprecated: JSON-RPC: `wait` reply `details` object: use subsytem specific object instead.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We reconnect from l3->l2, but l2 is also trying to reconnect and
we can race:
```
# Now try when l3 uses scid for entry point of blinded path.
l3.stop()
l3.daemon.opts['dev-invoice-bpath-scid'] = None
l3.start()
> l3.rpc.connect(l2.info['id'], 'localhost', l2.port)
tests/test_pay.py:5667:
...
> raise RpcError(method, payload, resp['error'])
E pyln.client.lightning.RpcError: RPC call failed: method: connect, payload: {'id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 'host': 'localhost', 'port': 33557}, error: {'code': 402, 'message': 'disconnected during connection'}
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
By submitting them all at once, rather than serially, we should *definitely* hit the
ratelimit. We can also reduce the timeout (from 60 seconds) to speed the test up.
```
@pytest.mark.slow_test
def test_onionmessage_ratelimit(node_factory):
l1, l2 = node_factory.line_graph(2, fundchannel=False,
opts={'allow_warning': True})
offer = l2.rpc.call('offer', {'amount': '2msat',
'description': 'simple test'})
# Hopefully we can do this fast enough to reach ratelimit!
> with pytest.raises(RpcError, match="Timeout waiting for response"):
E Failed: DID NOT RAISE <class 'pyln.client.lightning.RpcError'>
tests/test_pay.py:5825: Failed
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Really, any peer without a live channel is a bad prospect.
This requires us to wire the "enabled" flag through listincoming:
fortunately that's an internal, undocumented interface, so we don't
have a schema change.
Changelog-Fixed: Offline peers no longer selected for blinded paths..
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/8127
