This changes various tests in minor ways:
1. The "l2" secret key in tests/plugins/channeld_fakenet.c is updated.
2. The decompressed gossip data node id needs changing.
3. The coinmoves order changes in bookkeeper for anchors.
4. Various harcoded gossip constants change.
5. Some hardcoded makesecret results change.
6. zeroconf tests which hardcoded node ids change.
7. Arbitrary rune strings change.
8. A log message which uses node ids changes.
This means:
1. downgrade changes (we no longer fail due to node biases).
2. various deprecations no longer are
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When installed, the name is `lightning-hsmtool`. We actually copy
`tools/hsmtool` to `tools/lightning-hsmtool` but that's a silly step
which we should get rid of.
So:
1. Make sure our documentation always refers to it as lightning-hsmtool.
2. Make sure our tests invoke it as `lightning-hsmtool`.
3. Rename the C file.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
tests/test_coinmoves.py::test_generate_coinmoves (2,000,000, sqlite3):
Time (from start to end of l2 node): 135 seconds **WAS 227**
Worst latency: 12.1 seconds **WAS 62.4**
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This attempts to solve a problem we have with Phoenix clients:
This payment has been split in two many parts by the sender: 31 parts vs max 6 parts allowed for on-the-fly funding.
The problem is that we don't have any way in bolt11 or bolt12 to
specify the maximum number of HTLCs.
As a workaround, we start by restricting askrene to 6 parts if the
node is not openly reachable, and if it struggles, we remove the
restriction. This would work much better if askrene handled maxparts
more completely!
See-Also: https://github.com/ElementsProject/lightning/issues/8331
Changelog-Fixed: `xpay` will not try to send too many HTLCs through unknown channels (6, as that is Phoenix's limit) unless it has no choice
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This doesn't happen much in real life, but it's certainly possible, so do what pay does here.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/8612
Changelog-Added: `xpay` will now wait if it suspects a payment failure is due to a height disagreement with the final node.
fetchinvoice is still good for detailed diagnostics and handling
recurring invoices and alternate currencies, but this covers the
"throw some sats" case well.
Changelog-Added: JSON-RPC: `xpay` can now pay a simple offer directly, rather than requiring fetchinvoice first.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
All the core notifications changed over to wrapping the notification
fields in an object with the name of the notification, but notifications
from plugins were missed.
Changelog-Added: Plugins: `channel_hint_update`, `pay_failure` and `pay_success` notifications now have objects of the same name containing the expected fields.
Changelog-Deprecated: Plugins: `channel_hint_update`, `pay_failure` and `pay_success` notification fields outside the same-named object.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rather than forcing them to wrap their parameters in a "payload"
sub-object, copy in params directly. We include the "origin" field
one level up, if they care.
The next patch restores compatibility for the one place we currently use
them, which is the pay plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: pyln-client: plugin custom notifications origins and payload (use parameters directly)
We now have to explicitly enable various deprecated commando commands, and now
when deprecations are disabled, we honour missing MPP option in bolt12 invoices.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In 6e4ff6a7d2 ("offers: add a blinded path
if we have no advertized address") we were overzealous, and set blinded
paths not just for offers and invoicerequests, but for invoices themselves.
This has revealed various interop issues (which is great, but not good
for our users!) so we should disable that. It also reduces the reliability
of payments in general.
Changelog-None: fixes previously overzealous addition
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Use larger CLTVs and we see the failure from l3, complaining the CLTV is outside
the amount in the payment_constraint field, like:
```
Failing HTLC because of an invalid payload (TLV 10 pos 104): cltv_expiry 609 > payment_constraint 376
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We can create this scenario by having one path force close. We take
the opportunity to log this, even in non-slow-mode, since it's interesting
(not our bug, but someone just lost money!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was requested by Michael of Boltz; it's mainly useful if you plan to
try failed payments on a *different* node. In that case, there's a
theoretical possibility that slow parts of this payment could combine with
that from a different node and overpay.
We don't allow this from the same node, already.
Changelog-Added: xpay: `xpay-slow-mode` makes xpay wait for all parts of a payment to complete before returning success or failure.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When we merged blinded paths for nodes with no address
(6e4ff6a7d2), this test
broke. We need to prevent that, otherwise:
```
> assert ret['successful_parts'] == 2
E assert 1 == 2
tests/test_xpay.py:677: AssertionError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is deeply annoying, and we may have to support this properly
(using a separate algorithm entirely) if other implementations don't
fix their crap.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: xpay: suppress multi-part payment if invoice doesn't allow it (please, fix your nodes!)
```
____________________ ERROR at teardown of test_xpay_maxfee _____________________
...
# Format a nice list of everything that went wrong and raise an exception
request.node.has_errors = True
> raise ValueError(str(errors))
E ValueError:
E Node errors:
E - lightningd-1: Node exited with return code 1
E Global errors:
```
And:
```
@unittest.skipIf(TEST_NETWORK != 'regtest', 'too dusty on elements')
def test_xpay_maxfee(node_factory, bitcoind, chainparams):
"""Test which shows that we don't excees maxfee"""
outfile = tempfile.NamedTemporaryFile(prefix='gossip-store-')
subprocess.check_output(['devtools/gossmap-compress',
'decompress',
'--node-map=3301=022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59',
'tests/data/gossip-store-2024-09-22.compressed',
outfile.name]).decode('utf-8').splitlines()
AMOUNT = 100_000_000
# l2 will warn l1 about its invalid gossip: ignore.
# We throttle l1's gossip to avoid massive log spam.
> l1, l2 = node_factory.line_graph(2,
# This is in sats, so 1000x amount we send.
fundamount=AMOUNT,
opts=[{'gossip_store_file': outfile.name,
'subdaemon': 'channeld:../tests/plugins/channeld_fakenet',
'allow_warning': True,
'dev-throttle-gossip': None},
{'allow_bad_gossip': True}])
tests/test_xpay.py:509:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-testing/pyln/testing/utils.py:1720: in line_graph
nodes = self.get_nodes(num_nodes, opts=opts)
contrib/pyln-testing/pyln/testing/utils.py:1602: in get_nodes
return [j.result() for j in jobs]
contrib/pyln-testing/pyln/testing/utils.py:1602: in <listcomp>
return [j.result() for j in jobs]
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/concurrent/futures/_base.py:458: in result
return self.__get_result()
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/concurrent/futures/_base.py:403: in __get_result
raise self._exception
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/concurrent/futures/thread.py:58: in run
result = self.fn(*self.args, **self.kwargs)
contrib/pyln-testing/pyln/testing/utils.py:1653: in get_node
node.start(wait_for_bitcoind_sync)
contrib/pyln-testing/pyln/testing/utils.py:1015: in start
self.daemon.start(stderr_redir=stderr_redir)
contrib/pyln-testing/pyln/testing/utils.py:671: in start
self.wait_for_log("Server started with public key")
contrib/pyln-testing/pyln/testing/utils.py:355: in wait_for_log
return self.wait_for_logs([regex], timeout)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pyln.testing.utils.LightningD object at 0x7f27ab586c20>
regexs = ['Server started with public key'], timeout = 180
def wait_for_logs(self, regexs, timeout=TIMEOUT):
"""Look for `regexs` in the logs.
The logs contain tailed stdout of the process. We look for each regex
in `regexs`, starting from `logsearch_start` which normally is the
position of the last found entry of a previous wait-for logs call.
The ordering inside `regexs` doesn't matter.
We fail if the timeout is exceeded or if the underlying process
exits before all the `regexs` were found.
If timeout is None, no time-out is applied.
"""
logging.debug("Waiting for {} in the logs".format(regexs))
exs = [re.compile(r) for r in regexs]
start_time = time.time()
while True:
if self.logsearch_start >= len(self.logs):
if not self.logs_catchup():
time.sleep(0.25)
if timeout is not None and time.time() > start_time + timeout:
print("Time-out: can't find {} in logs".format(exs))
for r in exs:
if self.is_in_log(r):
print("({} was previously in logs!)".format(r))
> raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E TimeoutError: Unable to find "[re.compile('Server started with public key')]" in logs.
```
gossipd (and other plugins) simply take too long to digest the gossmap under valgrind.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cut & paste from the forwarding code, where we don't let onions use the
unannounced scids. Obviously local commands can use them.
Reported-by: @michael1011
Changelog-Fixed: JSON-RPC: xpay now works through unannounced channels.
Note that the slight code reorder changes the JSON order, which is generally
undefined, but our doc checker is very strict!
Changelog-Changed: `xpay` now gives the same JSON success return as documented by `pay` when `xpay-handle-pay` is set.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/7923
maxfeepercent is use by Zeus, so let's make that work.
maxfee is more precise, so it's the only xpay option (maxfee was added
to pay later).
[ Fix to ppm logic by Lagrang3, thanks! --RR ]
Fixes: https://github.com/ElementsProject/lightning/issues/7926
Changelog-Changed: JSON-RPC: With `xpay-handle-pay` set, xpay will now be used even if `pay` uses maxfeeprecent or exemptfee parameters (e.g. Zeus)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means that it gets shown in listsendpays: omitting this broke spark, apparently!
Changelog-Changed: `xpay` now populates more fields, so `listsendpays` and `listpays` show `destination` and `amount_msat` fields for xpay payments.
Fixes: https://github.com/ElementsProject/lightning/issues/7881
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were handing "maxfee" to every getroutes call, even if we had already
used some of the fees.
Reported-by: @daywalker90
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-None: xpay is new this release.
As we can see from the previous test, l3 tells us why it rejected the payment:
```
lightningd-3 2024-11-19T03:56:27.151Z DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: Failing HTLC because of an invalid payload (TLV 10 pos 104): cltv_expiry 136 > payment_constraint 130
```
It turns out, we were not updating the block height in the plugin!
Without this, when we create a (non-dummy) blinded path we set a
too-low CLTV restriction, and it doesn't work after a few blocks!
Note we were actually triggering this error in the xpay tests!
Reported-by: Vincenzo Palazzo
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Offers: Receiving bolt12 payments where we have no public channels would fail a few blocks after startup.
This is required for VLS which wants to know (and potentially decline) invoices
we're trying to pay.
As a nice side effect, our "check" command for xpay now does much more thorough
checking of arguments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
As the first user of a persistent layer, this tripped tests which
assumed the datastore would be empty!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
