- all forwarding types use the same flow
- forwarding callback returns a htlc_key or None
- forwarding info is persisted in lnworker:
- ongoing_forwardings
- downstream to upstream htlc_key
- htlc_key -> error_bytes
- introduce PaymentFeeBudget, which contains limits for fee budget and cltv budget
- when splitting a payment,
- the fee budget is linearly distributed between the parts
- this resolves a FIXME in lnrouter ("FIXME in case of MPP")
- the cltv budget is simply copied
- we could also add other kinds of budgets later, e.g. for the num in-flight htlcs
- resolves TODO in lnworker ("todo: compare to the fee of the actual route we found")
follow-up https://github.com/spesmilo/electrum/pull/8536
This replaces 69336befee, which was insufficient.
#8536 added a new field into the struct, which older versions do not ignore but raise:
opening a wallet file with new code updated the struct to include it,
after which old code could no longer open the wallet file.
i.e. #8536 was an invisible wallet upgrade, breaking compat.
This commit simply formalises the wallet upgrade: old code will now show
an understandable error when trying to open new files.
LnFeatures.supports() is became part of the hot path of
LNPathFinder.find_path_for_payment()
in 6b43eac6fd,
which made find_path_for_payment considerably slower than before.
It used to take around 0.8 sec originally, then after linked commit went to ~9 sec,
and now this takes it down to ~1.1 sec (on my laptop).
follow-up https://github.com/spesmilo/electrum/pull/8536
```
1.52 | E | gui.qt.ElectrumGui |
Traceback (most recent call last):
File "/home/user/wspace/electrum/electrum/gui/qt/__init__.py", line 342, in start_new_window
wallet = self.daemon.load_wallet(path, None)
File "/home/user/wspace/electrum/electrum/daemon.py", line 469, in func_wrapper
return func(self, *args, **kwargs)
File "/home/user/wspace/electrum/electrum/daemon.py", line 479, in load_wallet
wallet = self._load_wallet(path, password, manual_upgrades=manual_upgrades, config=self.config)
File "/home/user/wspace/electrum/electrum/util.py", line 466, in do_profile
o = func(*args, **kw_args)
File "/home/user/wspace/electrum/electrum/daemon.py", line 504, in _load_wallet
db = WalletDB(storage.read(), manual_upgrades=manual_upgrades)
File "/home/user/wspace/electrum/electrum/wallet_db.py", line 117, in __init__
self._after_upgrade_tasks()
File "/home/user/wspace/electrum/electrum/wallet_db.py", line 247, in _after_upgrade_tasks
self._load_transactions()
File "/home/user/wspace/electrum/electrum/util.py", line 466, in do_profile
o = func(*args, **kw_args)
File "/home/user/wspace/electrum/electrum/wallet_db.py", line 1536, in _load_transactions
self.data = StoredDict(self.data, self, [])
File "/home/user/wspace/electrum/electrum/json_db.py", line 117, in __init__
self.__setitem__(k, v)
File "/home/user/wspace/electrum/electrum/json_db.py", line 49, in wrapper
return func(self, *args, **kwargs)
File "/home/user/wspace/electrum/electrum/json_db.py", line 135, in __setitem__
v = self.db._convert_dict(self.path, key, v)
File "/home/user/wspace/electrum/electrum/json_db.py", line 247, in _convert_dict
v = dict((k, constructor(**x)) for k, x in v.items())
File "/home/user/wspace/electrum/electrum/json_db.py", line 247, in <genexpr>
v = dict((k, constructor(**x)) for k, x in v.items())
TypeError: ImportedChannelBackupStorage.__init__() missing 1 required positional argument: 'local_payment_pubkey'
```
scenario:
- user opens a lightning channel and exports an "imported channel backup"
- user closes channel via local-force-close
- local ctx is published, to_local output has user's funds and they are CSV-locked for days
- user restores wallet file from seed and imports channel backup
- new wallet file should be able to sweep coins from to_local output (after CSV expires)
This was not working previously, as the local_payment_basepoint was not included in the
imported channel backups, and the code was interpreting the lack of this as the channel not
having option_static_remotekey enabled. This resulted in lnutil.extract_ctn_from_tx
using an incorrect funder_payment_basepoint, and lnsweep not recognising the ctx due to
the garbage ctn value.
The imported channel backup serialisation format is slightly changed to include the
previously missing field, and its version number is bumped (0->1). We allow importing
both version 0 and version 1 backups, however v0 backups cannot handle the above
described scenario (they can only be used to request a remote-force-close).
Note that we were/are setting the missing local_payment_basepoint to the pubkey of
one of the wallet change addresses, which is bruteforceable if necessary, but I
think it is not worth the complexity to add this bruteforce logic. Also note
that the bruteforcing could only be done after the local-force-close was broadcast.
Ideally people with existing channels and already exported v0 backups should re-export
v1 backups... Not sure how to handle this.
closes https://github.com/spesmilo/electrum/issues/8516
decorators (instead of overloading JsonDB._convert_dict and
_convert_value)
- stored_in for elements of a StoreDict
- stored_as for singletons
- extra register methods are defined for key conversions
This commit was adapted from the jsonpatch branch
closes https://github.com/spesmilo/electrum/issues/8403
> In Python 3.10 that worked fine, however in Python 3.11 large integer check https://github.com/python/cpython/issues/95778, so now this throws an error.
Apparently this change was deemed a security fix and was backported to all supported branches of CPython (going back to 3.7). i.e. it affects ~all versions of python (if sufficiently updated with bugfix patches), not just 3.11
> Some offending node aliases:
> ```
> ergvein-fiatchannels
> test-mainnet
> arakis
> ```
The features bits set by some of these nodes:
```
(1, 7, 8, 11, 13, 14, 17, 19, 23, 27, 45, 32973, 52973)
(1, 7, 8, 11, 13, 14, 17, 19, 23, 27, 39, 45, 55, 32973, 52973)
```
> P.S. I see there are a lot of nodes with 253 bytes in their feature vectors. Any idea why that could happen?
Note that the valid [merged-into-spec features](50b2df24a2/09-features.md) currently only go as high as ~51.
However the spec does not specify how to choose feature bits for experimental stuff, so I guess some people are using values in the 50k range. The only limit imposed by the spec on the length of the features bitvector is an implicit one due to the max message size: every msg must be smaller than 65KB, and the features bitvector needs to fit inside the init message, hence it can be up to ~524K bits.
(note that the features are not stored in a sparse representation in the init message and in gossip messages, so if many nodes set such high feature bits, that would noticably impact the size of the gossip).
-----
Anyway, our current implementation of LnFeatures is subclassing IntFlag, and it looks like it does not work well for such large integers. I've managed to make IntFlags reasonably in python 3.11 by overriding __str__ and __repr__ (note that in cpython it is apparently only the base2<->base10 conversions that are slow, power-of-2 conversions are fast, so we can e.g. use `hex()`). However in python 3.10 and older, enum.py itself seems really slow for bigints, e.g. enum._decompose in python 3.10.
Try e.g. this script, which is instant in py3.11 but takes minutes in py3.10:
```py
from enum import IntFlag
class c(IntFlag):
known_flag_1 = 1 << 0
known_flag_2 = 1 << 1
known_flag_3 = 1 << 2
if hasattr(IntFlag, "_numeric_repr_"): # python 3.11+
_numeric_repr_ = hex
def __repr__(self):
return f"<{self._name_}: {hex(self._value_)}>"
def __str__(self):
return hex(self._value_)
a = c(2**70000-1)
q1 = repr(a)
q2 = str(a)
```
AFAICT we have two options: either we rewrite LnFeatures so that it does not use IntFlag (and enum.py), or, for the short term as workaround, we could just reject very large feature bits.
For now, I've opted to the latter, rejecting feature bits over 10k.
(note that another option is bumping the min required python to 3.11, in which case with the overrides added in this commit the performance looks perfectly fine)
In the binary serialised format, replace all instances of int16 with uint16.
In particular, this allows port>32767.
Fixes https://github.com/spesmilo/electrum/issues/8264
I think this is backwards compatible, as in, any existing channel backup already out there,
should be properly parsed with the new code. (new code however can serialise cbs that old
code deserialises incorrectly)
```
>>> struct.pack('<h', 258)
b'\x02\x01'
>>> struct.pack('<H', 258)
b'\x02\x01'
```
ShortIDs were originally designed for lightning channels, and are now
understood by some block explorers.
This allows to remove one column in the UTXO tab (height is redundant).
In the transaction dialog, the space saving ensures that all inputs fit
into one line (it was not the case previously with p2wsh addresses).
For clarity and consistency, the ShortID is displayed for both inputs
and outputs in the transaction dialog.
- save remote alias for use in invoices
- derive local alias from wallet xpub
- send channel_type without the option_scid_alias bit
(apparently LND does not like it)
The exceptions are meant to be raised in places where the BOLTs require
the sending of warning or error messages. They are necessary to handle
protocol failures occuring helper functions that check constraints.
* on channel opening we verify that the peer's dust limit is above 354
sat, the limit for unknown segwit versions
* we constrain the allowed scriptpubkey types for channel closing
* we check that the remote's output is above the relay dust limit for
the collaborative close case
Adds liquidity hints for the sending capabilities of routing channels in the
graph. The channel blacklist is incorporated into liquidity hints.
Liquidity hints are updated when a payment fails with a temporary
channel failure or when it succeeds. Liquidity hints are used to give a
penalty in the _edge_cost heuristics used by the pathfinding algorithm.
The base penalty in (_edge_cost) is removed because it is now part of the
liquidity penalty. We don't return early from get_distances, as we want
to explore all channels.
