We tried to delete incoming channels that didn't get funded after
lnutil.CHANNEL_OPENING_TIMEOUT, however an assert prevented this:
```
3.63 | E | lnwatcher.LNWatcher.[default_wallet-LNW] | Exception in check_onchain_situation: AssertionError()
Traceback (most recent call last):
File "/home/user/code/electrum-fork/electrum/util.py", line 1233, in wrapper
return await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/code/electrum-fork/electrum/lnwatcher.py", line 117, in check_onchain_situation
await self.update_channel_state(
...<5 lines>...
keep_watching=keep_watching)
File "/home/user/code/electrum-fork/electrum/lnwatcher.py", line 135, in update_channel_state
chan.update_onchain_state(
~~~~~~~~~~~~~~~~~~~~~~~~~^
funding_txid=funding_txid,
^^^^^^^^^^^^^^^^^^^^^^^^^^
...<2 lines>...
closing_height=closing_height,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
keep_watching=keep_watching)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/code/electrum-fork/electrum/lnchannel.py", line 341, in update_onchain_state
self.update_unfunded_state()
~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/user/code/electrum-fork/electrum/lnchannel.py", line 382, in update_unfunded_state
self.lnworker.remove_channel(self.channel_id)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/user/code/electrum-fork/electrum/lnworker.py", line 3244, in remove_channel
assert chan.can_be_deleted()
~~~~~~~~~~~~~~~~~~~^^
AssertionError
```
Store the channel id instead of the scid in ReceivedMPPHtlc.
The scid can be None, in theory even for multiple channels at the same
time. Using the channel_id which is always available and unique seems
less error prone at the cost of temporarily higher storage requirements
in the db for the duration of the pending htlcs.
Alternatively we could use the local scid alias however using the
channel_id seems less complex and leaves less room for ambiguity.
Allows storing two different payment info of the same payment hash by
including the direction into the db key.
We create and store PaymentInfo for sending attempts and for requests (receiving),
if we try to pay ourself (e.g. through a channel rebalance) the checks
in `save_payment_info` would prevent this and throw an exception.
By storing the PaymentInfos of outgoing and incoming payments separately in
the db this collision is avoided and it makes it easier to reason about
which PaymentInfo belongs where.
I was unable to do a "Max" amount submarine swap because the
`fee_estimate` method used by `LNWallet.num_sats_can_send()` uses a
hardcoded `fee_proportional_millionths` to estimate the fee for the
lightning payment.
When the actual fee determined later is higher
than the estimated fee the payment fails as the channel is unable to add
the htlc sum including the real fees as the amount exceeds the balance of
the channel.
Using the fees the maximum fees user has configured and estimate the
potential fee as inverse of PaymentFeeBudget is more
reliable/conservative as we definitely aren't going to pay more fees
than this amount.
refactor `htlc_switch` to new architecture to make it more robust
against partial settlement of htlc sets and increase maintainability.
Htlcs are now processed in two steps, first the htlcs are collected into
sets from the channels, and potentially failed on their own already.
Then a second loop iterates over the htlc sets and finalizes only on
whole sets.
# Conflicts:
# electrum/lnpeer.py
Enforce that the information used to create a bolt11 invoice using
`get_bolt11_invoice()` is similar to the related instance of PaymentInfo
by requiring a PaymentInfo as argument for `get_bolt11_invoice()`.
This way the invoice cannot differ from the created PaymentInfo.
This allows to use the information in PaymentInfo for validation of
incoming htlcs more reliably.
To cover all required information for the creation of a b11 invoice the
PaymentInfo class has to be extended with a expiry and
min_final_cltv_expiry. This requires a db upgrade.
Renames RecvMPPResolution.ACCEPTED to .COMPLETE as .ACCEPTED is somewhat
misleading. Accepted could imply that the preimage for this set has been
revealed or that the set has been settled, however it only means that we
have received the full set (it is complete), but the set still can be
failed (e.g. through cltv timeout) and has not been claimed yet.
Adds `max_cltv` and `max_fee_msat` parameters to the `lnpay` cli command which allow to
specify the maximum total locktime of the payment and the maximum
absolute fee budget. This is enabled by
constructing a custom `PaymentFeeBudget` object in the lnpay command and
passing it as argument to `LNWallet.pay_invoice()`.
Allowing to specify a `max_cltv` value can be useful for certain
usecases, e.g. see https://github.com/spesmilo/electrum/issues/10056.
Closes#10056
so we don't error out when GUI hinted minimum 2mBTC channel size is used for peers that
constrain max-htlc-in-flight to 45% of channel capacity. (fixes#8712)
channel_db.load_data() is slow, slowing down startup time (when trampoline is disabled).
util.list_enabled_bits() is one of the main contributors to the slowness, called by validate_features().
One could argue that we could even simply *not* call validate_features for gossip messages as part of load_data,
as they have already been validated before storing them in the db. However re-validating them there is a good
clean-up/sanity check IMO. Note that what is considered "valid" can change over time, so just because validate_features
passed when we originally received and stored a gossip message, it might no longer be valid a year later if the bolts change.
This caching decreases the time needed for load_data on two different machines / gossip dbs as below:
47 sec -> 10 sec
18 sec -> 6 sec
If instead of caching, I just rm the validate_features() calls, the benchmarks are almost identical, within noise.
That is, the cache looks really effective.
(the rest of the slowness is mostly due to lnmsg.decode_msg)
```
>>> lnutil.validate_features.cache_info()
CacheInfo(hits=172674, misses=287, maxsize=1000, currsize=277)
```
-----
We could alternatively directly cache util.list_enabled_bits (instead of validate_features).
That would be a bit slower and might end up using a lot more memory in some cases I think, but maybe conceptually would be cleaner.
Also note that if validate_features() raises an exception, that is not cached.
- Wallet.make_unsigned_transaction takes a FeePolicy parameter
- fee sliders act on a FeePolicy instead of config
- different fee policies may be used for different purposes
- do not detect dust outputs in lnsweep, delegate that to lnwatcher
store exception in variable instead of using a bool flag
add default str to routing exceptions
Add separate exception class to handle fee related payment errors
Ideally, given an on-chain backup, after the remote force-closes, we should be able to spend our anchor output,
to CPFP the remote commitment tx (assuming the channel used OPTION_ANCHORS).
To spend the anchor output, we need to be able to sign with the local funding_privkey.
Previously we derived the funding_key from the channel_seed (which comes from os.urandom).
Prior to anchors, there was no use case for signing with the funding_key given a channel backup.
Now with anchors, we should make its derivation deterministic somehow, in a way so that it can
be derived given just an on-chain backup.
- one way would be to put some more data into the existing OP_RETURN
- uses block space
- the OP_RETURNs can be disabled via "use_recoverable_channels"
- only the initiator can use OP_RETURNs (so what if channel is in incoming dir?)
- instead, new scheme for our funding_key:
- we derive the funding_privkey from the lnworker root secret (derived from our bip32 seed)
- for outgoing channels:
- lnworker_root_secret + remote_node_id + funding_tx_nlocktime
- for incoming channels:
- lnworker_root_secret + remote_node_id + remote_funding_pubkey
- a check is added to avoid reusing the same key between channels:
not letting to user open more than one channel with the same peer in a single block
- only the first 16 bytes of the remote_node_id are used, as the onchain backup OP_RETURNs only contain that
- as the funding_privkey cannot be derived from the channel_seed anymore, it is included in the
imported channel backups, which in turn need a new version defined
- a wallet db upgrade is used to update already stored imported cbs
- alternatively we could keep the imported cbs as-is, so no new version, no new funding_privkey field, as it is clearly somewhat redundant given on-chain backups can reconstruct it
- however adding the field seems easier
- otherwise the existing code would try to derive the funding_privkey from the channel_seed
- also note: atm there is no field in the imported backups to distinguish anchor channels vs static-remotekey channels
* sets the weight of htlc transactions to zero, thereby putting a zero
fee for the htlc transactions
* add inputs to htlc-tx for fee bumping
* switches feature flags
* disable anchor test vectors, which are now partially invalid
* add a method for backups to sweep to_remote
* to_remote sweeping needs the payment_basepoint's private key
to sign the sweep transaction
* we restore the private key from our funding multisig pubkey
(pubished with the closing transaction) and a static payment key secret
* to_remote has now an additional csv lock of 1
* anchor outputs are added if to_local/remote outputs are present
* funder balance is reduced to accomodate anchors
* changes the htlc outputs' witness script to have a csv lock of 1
* send signatures for remote ctx with ANYONECANPAY|SINGLE
* refactor htlc weight (useful for zero-fee-htlc)
* in order to be able to sweep to_remote in an onchain backup scenario
we need to retain the private key for the payment_basepoint
* to facilitate the above, we open a channel derived from a static
secret (tied to the wallet seed), the static_payment_key combined with
the funding pubkey (multisig_key), which we can restore from the channel
closing transaction
- Separation between SwapManager and its transport:
Legacy transpport uses http, Nostr uses websockets
- The transport uses a context to open/close connections.
This context is not async, because it needs to be called
from the GUI
- Swapserver fees values are initialized to None instead
of 0, so that any attempt to use them before the swap
manager is initialized will raise an exception.
- Remove swapserver fees disk caching (swap_pairs file)
- Regtests use http transport
- Android uses http transport (until QML is ready)
This will be useful if we decide to ship lntransport as a separate
package. It is also a conceptual cleanup.
Notes:
- lntransport still requires crypto.py
- parsing node id from a bolt11 invoice is not supported.
Add support for key-path-spending taproot utxos into transaction.py.
- no wallet support yet
- add some psbt, and minimal descriptor support
- preliminary work towards script-path spends
get_preimage_script should really have been private API...
looks like everywhere it is used outside of transaction.py, it is actually abused :/
Other existing usages in plugin code I don't dare to touch without lots of manual testing...
Instead of some functions operating with hex strings,
and others using bytes, this consolidates most things to use bytes.
This mainly focuses on bitcoin.py and transaction.py,
and then adapts the API usages in other files.
Notably,
- scripts,
- pubkeys,
- signatures
should be bytes in almost all places now.
SomberNight