`suggest_node_channel_open()` did suggest peers with onion hostname,
even if the caller has no proxy enabled. This causes channel openings in
the gui to sometimes just not work and show a `CancelledError()` becaues
it wasn't able to connect to the peer.
Now only clearnet peers will get recommended, as these will always work.
add space
add gossip address field serialization, parsing and tests
fix linter
consolidate tests, fix intendation
refactor test in loops
add gossip address field serialization, parsing and tests
This will be useful if we decide to ship lntransport as a separate
package. It is also a conceptual cleanup.
Notes:
- lntransport still requires crypto.py
- parsing node id from a bolt11 invoice is not supported.
- construct_channel_announcement: return also whether
node ids are in reverse order
- maybe_send_channel_announcement:
return early if signatures have not been received
ChannelDB.load_data() takes ~15 seconds. Previously if the user tried
to close the program while load_data is running, we would block until
load_data() finished. (e.g. consider starting and immediately stopping
Electrum)
Now instead we can abort load_data early.
eclair sends CHANNEL_DISABLED if its peer is offline. E.g. we might be
trying to pay a mobile phone with the app closed. In that case we
should not cache the CHANNEL_DISABLED for too long.
On mobile, it can take a while before channelDB is loaded. If payment is attempted before the DB
is fully loaded, this would result in a payment failure, but also leaves the payment attempt in IN_PROGRESS
state. This patch adds a more specific ChannelDBNotLoaded exception class, so we can handle this case more
gracefully, since we know the payment didn't succeed.
closes https://github.com/spesmilo/electrum/issues/8403
> In Python 3.10 that worked fine, however in Python 3.11 large integer check https://github.com/python/cpython/issues/95778, so now this throws an error.
Apparently this change was deemed a security fix and was backported to all supported branches of CPython (going back to 3.7). i.e. it affects ~all versions of python (if sufficiently updated with bugfix patches), not just 3.11
> Some offending node aliases:
> ```
> ergvein-fiatchannels
> test-mainnet
> arakis
> ```
The features bits set by some of these nodes:
```
(1, 7, 8, 11, 13, 14, 17, 19, 23, 27, 45, 32973, 52973)
(1, 7, 8, 11, 13, 14, 17, 19, 23, 27, 39, 45, 55, 32973, 52973)
```
> P.S. I see there are a lot of nodes with 253 bytes in their feature vectors. Any idea why that could happen?
Note that the valid [merged-into-spec features](50b2df24a2/09-features.md) currently only go as high as ~51.
However the spec does not specify how to choose feature bits for experimental stuff, so I guess some people are using values in the 50k range. The only limit imposed by the spec on the length of the features bitvector is an implicit one due to the max message size: every msg must be smaller than 65KB, and the features bitvector needs to fit inside the init message, hence it can be up to ~524K bits.
(note that the features are not stored in a sparse representation in the init message and in gossip messages, so if many nodes set such high feature bits, that would noticably impact the size of the gossip).
-----
Anyway, our current implementation of LnFeatures is subclassing IntFlag, and it looks like it does not work well for such large integers. I've managed to make IntFlags reasonably in python 3.11 by overriding __str__ and __repr__ (note that in cpython it is apparently only the base2<->base10 conversions that are slow, power-of-2 conversions are fast, so we can e.g. use `hex()`). However in python 3.10 and older, enum.py itself seems really slow for bigints, e.g. enum._decompose in python 3.10.
Try e.g. this script, which is instant in py3.11 but takes minutes in py3.10:
```py
from enum import IntFlag
class c(IntFlag):
known_flag_1 = 1 << 0
known_flag_2 = 1 << 1
known_flag_3 = 1 << 2
if hasattr(IntFlag, "_numeric_repr_"): # python 3.11+
_numeric_repr_ = hex
def __repr__(self):
return f"<{self._name_}: {hex(self._value_)}>"
def __str__(self):
return hex(self._value_)
a = c(2**70000-1)
q1 = repr(a)
q2 = str(a)
```
AFAICT we have two options: either we rewrite LnFeatures so that it does not use IntFlag (and enum.py), or, for the short term as workaround, we could just reject very large feature bits.
For now, I've opted to the latter, rejecting feature bits over 10k.
(note that another option is bumping the min required python to 3.11, in which case with the overrides added in this commit the performance looks perfectly fine)
One usecase is perhaps to save space if using trampoline anyway...
more importantly, if using gossip, LNGossip is heavily filtering what messages we request and get,
and e.g. can missing new NodeAnnouncements, etc,
and this is a quick-and-dirty workaround to force a fresh start.
- use_recoverable_channel is a user setting, available
only in standard wallets with a 'segwit' seed_type
- if enabled, 'lightning_xprv' is derived from seed
- otherwise, wallets use the existing 'lightning_privkey2'
Recoverable channels:
- channel recovery data is added funding tx using an OP_RETURN
- recovery data = 4 magic bytes + node id[0:16]
- recovery data is chacha20 encrypted using funding_address as nonce.
(this will allow to fund multiple channels in the same tx)
GUI:
- whether channels are recoverable is shown in wallet info dialog.
- if the wallet can have recoverable channels but has an old node_id,
users are told to close their channels and restore from seed
to have that feature.
This is re the channel update for the incoming direction of our own channels.
This message can only come from the counterparty itself so maybe the sig check
is redundant... but for sanity I think we should check it anyway.
We pass the private edges to lnrouter, and let it find routes end-to-end.
Previously the edge_cost heuristics didn't apply to the private edges
and we were just randomly picking one of the route hints and use that.
So e.g. cheaper private edges were not preferred, but they are now.
PathEdge now stores both start_node and end_node; not just end_node.
before:
node_id -> set of (host, port, ts)
after:
node_id -> NetAddress -> timestamp
Look at e.g. add_recent_peer; we only want to store
the last connection time, not all of them.
Introduces LNRater, which analyzes the Lightning Network graph for
potential nodes to connect to by taking into account channel capacities,
channel open times and fee policies. A score is constructed to assign a
scalar to each node, which is then used to perform a weighted random
sampling of the nodes.
The gossip db is loaded early when the network is started to save
time when the gui is locked and a wallet not yet loaded. Side effects
of the LNWallet to start peering when a channel db is loaded is
circumvented.
Implements a way to represent the graph (excluding one own's node) in
terms of a dict, which is json encodeable. Address tuples are converted
to NamedTuples to have automatic annotation in the address outputs.
