Commit Graph

543 Commits

Author SHA1 Message Date
ThomasV
b96cc82333 Make storage a field of db
This comes from the jsonpatch_new branch.
I rather have in master now, because it touches a lot of filese.
2023-08-18 08:08:31 +02:00
Sander van Grieken
f980bd97b5 payment_identifier: factor out bip21 functions to bip21.py to break cyclic dependencies,
parse bolt11 only once, store invoice internally instead of bolt11 string
add is_onchain method to indicate if payment identifier can be paid onchain
2023-07-08 12:18:37 +02:00
ThomasV
15eb765eac payment_identifiers:
- this separates GUI from core handling
 - the PaymentIdentifier class handles network requests
 - the GUI is agnostic about the type of PI
2023-06-28 16:49:28 +02:00
SomberNight
033ad0feb9 lnworker: fix rebalance_channels
fixes https://github.com/spesmilo/electrum/issues/8468
2023-06-04 03:07:06 +00:00
SomberNight
24980feab7 config: introduce ConfigVars
A new config API is introduced, and ~all of the codebase is adapted to it.
The old API is kept but mainly only for dynamic usage where its extra flexibility is needed.

Using examples, the old config API looked this:
```
>>> config.get("request_expiry", 86400)
604800
>>> config.set_key("request_expiry", 86400)
>>>
```

The new config API instead:
```
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS
604800
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS = 86400
>>>
```

The old API operated on arbitrary string keys, the new one uses
a static ~enum-like list of variables.

With the new API:
- there is a single centralised list of config variables, as opposed to
  these being scattered all over
- no more duplication of default values (in the getters)
- there is now some (minimal for now) type-validation/conversion for
  the config values

closes https://github.com/spesmilo/electrum/pull/5640
closes https://github.com/spesmilo/electrum/pull/5649

Note: there is yet a third API added here, for certain niche/abstract use-cases,
where we need a reference to the config variable itself.
It should only be used when needed:
```
>>> var = config.cv.WALLET_PAYREQ_EXPIRY_SECONDS
>>> var
<ConfigVarWithConfig key='request_expiry'>
>>> var.get()
604800
>>> var.set(3600)
>>> var.get_default_value()
86400
>>> var.is_set()
True
>>> var.is_modifiable()
True
```
2023-05-25 17:39:48 +00:00
SomberNight
68fb996d20 wallet_db version 52: break non-homogeneous multisig wallets
- case 1: in version 4.4.1, 4.4.2, the qml GUI wizard allowed creating multisig wallets with an old_mpk as cosigner.
- case 2: in version 4.4.0, 4.4.1, 4.4.2, the qml GUI wizard allowed creating multisig wallets with mixed xpub/Ypub/Zpub.

The corresponding missing input validation was a bug in the wizard, it was unintended behaviour. Validation was added in d2cf21fc2b. Note however that there might be users who created such wallet files.

Re case 1 wallet files: there is no version of Electrum that allows spending from such a wallet. Coins received at addresses are not burned, however it is technically challenging to spend them. (unless the multisig can spend without needing the old_mpk cosigner in the quorum).

Re case 2 wallet files: it is possible to create a corresponding spending wallet for such a multisig, however it is a bit tricky. The script type for the addresses in such a heterogeneous xpub wallet is based on the xpub_type of the first keystore. So e.g. given a wallet file [Yprv1, Zpub2] it will have sh(wsh()) scripts, and the cosigner should create a wallet file [Ypub1, Zprv2] (same order).

Technically case 2 wallet files could be "fixed" automatically by converting the xpub types as part of a wallet_db upgrade. However if the wallet files also contain seeds, those cannot be converted ("standard" vs "segwit" electrum seed).
Case 1 wallet files are not possible to "fix" automatically as the cosigner using the old_mpk is not bip32 based.

It is unclear if there are *any* users out there affected by this. I suspect for case 1 it is very likely there are none (not many people have pre-2.0 electrum seeds which were never supported as part of a multisig who would also now try to create a multisig using them); for case 2 however there might be.

This commit breaks both case 1 and case 2 wallets: these wallet files can no longer be opened in new Electrum, an error message is shown and the crash reporter opens. If any potential users opt to send crash reports, at least we will know they exist and can help them recover.
2023-05-11 14:26:11 +00:00
SomberNight
38ec72527f wallet.get_bolt11_invoice: handle request not having LN part
fixes https://github.com/spesmilo/electrum/issues/8402

To reproduce,
- create wallet from a zpub
- LN is disabled there by default
- create a receive request, which won't have a lightning part
- enable config var "bip21_lightning"
- enable LN
- existing request is now ~breaking receive tab
2023-05-07 00:39:06 +00:00
ThomasV
c0917e65af fix #8391: reintroduce recursion, limited to unconfirmed transactions
It would be better to have topological order, but this issue needs to
be fixed quickly.
2023-05-06 18:10:51 +02:00
SomberNight
1e55042830 wallet: set_frozen_state_of_{addresses,coins} to save_db() to disk
fixes https://github.com/spesmilo/electrum/issues/8389
2023-05-06 13:26:08 +00:00
SomberNight
9c47144418 qt: handle expected errors in DSCancelDialog
closes https://github.com/spesmilo/electrum/issues/8390
2023-05-05 16:06:16 +00:00
SomberNight
56fa832563 wallet.get_tx_parents: explicitly handle missing "from address"
(happened to work even without this)
2023-05-03 15:08:56 +00:00
SomberNight
a1bfea6121 wallet.get_tx_parents: fix: tx should not be its own uncle
```
  3.06 | I | w/wallet.Standard_Wallet.[default_wallet] | get_tx_parents() cp0. entered for txid='407d03126255cce62a1101075db906587bd492f512166119d3f87b8a1b013497'
  3.06 | I | w/wallet.Standard_Wallet.[default_wallet] | get_tx_parents() cp4.1. parents=['e2d915520f6d42273158a6fd08b38d812bd554aa906d3ed45d103757d45af2bb']
  3.06 | I | w/wallet.Standard_Wallet.[default_wallet] | get_tx_parents() cp4.2. uncles=[]
  3.06 | I | w/wallet.Standard_Wallet.[default_wallet] | get_tx_parents() cp2. populating cache... _txid='434808dab0c93715bb8b7ce85f73bffd0bdf7c1ba205fe0f704226646971e555'
  3.06 | I | w/wallet.Standard_Wallet.[default_wallet] | get_tx_parents() cp0. entered for txid='434808dab0c93715bb8b7ce85f73bffd0bdf7c1ba205fe0f704226646971e555'
  3.06 | I | w/wallet.Standard_Wallet.[default_wallet] | get_tx_parents() cp3.0. uncle_txid='434808dab0c93715bb8b7ce85f73bffd0bdf7c1ba205fe0f704226646971e555'
  3.06 | I | w/wallet.Standard_Wallet.[default_wallet] | get_tx_parents() cp3.1. reuse_height=-2 reuse_pos=-1
  3.06 | I | w/wallet.Standard_Wallet.[default_wallet] | get_tx_parents() cp3.2. my_height=1338 my_pos=1
  3.06 | I | w/wallet.Standard_Wallet.[default_wallet] | get_tx_parents() cp4.1. parents=['407d03126255cce62a1101075db906587bd492f512166119d3f87b8a1b013497']
  3.06 | I | w/wallet.Standard_Wallet.[default_wallet] | get_tx_parents() cp4.2. uncles=['434808dab0c93715bb8b7ce85f73bffd0bdf7c1ba205fe0f704226646971e555']
 40.82 | E | gui.qt.exception_window.Exception_Hook | exception caught by crash reporter
Traceback (most recent call last):
  File "/home/user/wspace/electrum/electrum/gui/qt/main_window.py", line 846, in timer_actions
    self.update_wallet()
  File "/home/user/wspace/electrum/electrum/gui/qt/main_window.py", line 1001, in update_wallet
    self.update_tabs()
  File "/home/user/wspace/electrum/electrum/gui/qt/main_window.py", line 1013, in update_tabs
    self.utxo_list.update()
  File "/home/user/wspace/electrum/electrum/util.py", line 470, in <lambda>
    return lambda *args, **kw_args: do_profile(args, kw_args)
  File "/home/user/wspace/electrum/electrum/util.py", line 465, in do_profile
    o = func(*args, **kw_args)
  File "/home/user/wspace/electrum/electrum/gui/qt/utxo_list.py", line 115, in update
    self.refresh_row(name, idx)
  File "/home/user/wspace/electrum/electrum/gui/qt/utxo_list.py", line 137, in refresh_row
    num_parents = self.wallet.get_num_parents(txid)
  File "/home/user/wspace/electrum/electrum/wallet.py", line 897, in get_num_parents
    self._num_parents[txid] = len(self.get_tx_parents(txid))
  File "/home/user/wspace/electrum/electrum/wallet.py", line 910, in get_tx_parents
    self.get_tx_parents(_txid)
  File "/home/user/wspace/electrum/electrum/wallet.py", line 938, in get_tx_parents
    p = self._tx_parents_cache[_txid]
KeyError: '434808dab0c93715bb8b7ce85f73bffd0bdf7c1ba205fe0f704226646971e555'
```
2023-05-03 15:06:37 +00:00
SomberNight
5512c7d905 wallet.get_tx_info: distinguish "future" tx from local in "status" str
closes https://github.com/spesmilo/electrum/issues/8379
2023-05-03 14:15:08 +00:00
ThomasV
8a25d59ba6 wallet: save num_parents of utxos in wallet file.
fixes #8361
2023-05-02 09:27:39 +02:00
ThomasV
6d392a67f9 wallet.get_tx_parents: populate cache regardless of self.is_up_to_date()
Fixes #8315
2023-05-02 09:11:21 +02:00
SomberNight
57ae933582 (trivial) add some type hints to wallet.get_tx_parents 2023-04-26 18:11:55 +00:00
ThomasV
8dca907891 get_tx_parent: populate cache in chronological order 2023-04-26 18:29:46 +02:00
SomberNight
ad5f95cb87 util.profiler: add "min_threshold" arg 2023-04-26 14:49:37 +00:00
SomberNight
bf41675d4c qml: invoice/request list: flip sort order: newest on top
to be consistent with the History, and with other GUIs

(the model is the easiest place to do it. QSortFilterProxyModel/etc looks overkill)
2023-04-24 15:33:10 +00:00
SomberNight
612d3493df fix flake8-bugbear B017
B017 `assertRaises(Exception)` and `pytest.raises(Exception)` should be considered evil. They can lead to your test passing even if the code being tested is never executed due to a typo. Assert for a more specific exception (builtin or custom), or use `assertRaisesRegex` (if using `assertRaises`), or add the `match` keyword argument (if using `pytest.raises`), or use the context manager form with a target.
2023-04-24 12:58:30 +00:00
SomberNight
312f2641e7 don't use bare except
use "except Exception", or if really needed explicitly "except BaseException"
2023-04-24 12:58:01 +00:00
SomberNight
6b0db411ae wallet: save_db immediately after init_lightning() generates keys
Had a crash shortly after enabling lightning and the LN keys were lost...
Though note that opening a channel triggers wallet.save_db(), so I think
nothing of real value is at risk without this change.
2023-04-23 00:06:45 +00:00
SomberNight
dd93ebda4d wallet: _bump_fee_through_decreasing_payment: handle too high fee_rate
fixes https://github.com/spesmilo/electrum/issues/8316
2023-04-19 21:55:07 +00:00
SomberNight
89b75f95d0 wallet.set_broadcasting: fix incorrect type-hint and rename arg 2023-04-19 16:29:54 +00:00
SomberNight
6ac3f84095 wallet/lnworker/etc: add sanity checks to start_network methods
related: https://github.com/spesmilo/electrum/issues/8301
2023-04-17 16:56:57 +00:00
Sander van Grieken
99a78d4d6c wallet: don't restart wallet/network when init_lightning 2023-04-17 15:00:53 +02:00
SomberNight
c9cc56b687 transaction: don't include WIT_UTXO for non-segwit txins
probably regression from d3227d7489

fixes https://github.com/spesmilo/electrum/issues/8305
2023-04-16 21:14:15 +00:00
SomberNight
488dc4871e wallet: is_up_to_date() to return False if taskgroup stopped
If the taskgroup died unexpectedly, this will result in the GUI
showing we are in the "synchronizing" state instead of the green orb.
Being stuck in "synchronizing" provides at least *some* feedback to
the user that something is wrong.

see https://github.com/spesmilo/electrum/issues/8301
2023-04-13 14:42:06 +00:00
SomberNight
e748345be0 addr_sync: change return type of get_address_history to dict from list 2023-04-05 13:09:47 +00:00
ThomasV
03f0d632af wallet.sign_transaction: return tx if signed by swap manager
This fixes bumping swap fee in the qml GUI, because it expects
the value returned by this method to be None if the transaction
could not be signed.
2023-04-05 14:57:58 +02:00
SomberNight
db4943ff86 wallet.get_full_history: more consistent sort order
before:
```
>>> [print(wallet.get_tx_status(txid, wallet.adb.get_tx_height(txid))) for txid in list(wallet.get_full_history())[-10:]]
(7, '2023-04-04 16:13')
(7, '2023-04-04 16:13')
(7, '2023-04-04 16:13')
(7, '2023-04-04 16:13')
(0, 'Unconfirmed [20. sat/b, 0.00 MB]')
(2, 'in 2 blocks')
(3, 'Local [180.4 sat/b]')
(3, 'Local [180.2 sat/b]')
(2, 'in 2016 blocks')
(0, 'Unconfirmed [180. sat/b, 0.00 MB]')
```

after:
```
>>> [print(wallet.get_tx_status(txid, wallet.adb.get_tx_height(txid))) for txid in list(wallet.get_full_history())[-10:]]
(7, '2023-04-04 16:13')
(7, '2023-04-04 16:13')
(7, '2023-04-04 16:13')
(7, '2023-04-04 16:13')
(0, 'Unconfirmed [20. sat/b, 0.00 MB]')
(0, 'Unconfirmed [180. sat/b, 0.00 MB]')
(2, 'in 2016 blocks')
(2, 'in 2 blocks')
(3, 'Local [180.4 sat/b]')
(3, 'Local [180.2 sat/b]')
```
2023-04-04 18:32:53 +00:00
ThomasV
f04e2e2e6f Add an extra state for invoices where our tx has been broadcast
successfully, but it is not in our history yet.

(follow-up 159646fe54)
2023-04-04 19:31:25 +02:00
ThomasV
159646fe54 Set status of onchain invoices to PR_INFLIGHT while tx is being broadcast 2023-04-04 18:22:30 +02:00
ThomasV
56e685feaa invoices: Use the same base method to export invoices and requests.
This fixes an inconsistency where the 'expiration' field was
relative for invoices, and absolute timestamp for requests.

This in turn fixes QML the timer refreshing the request list.

In order to prevent any API using that field from being silently
broken, the 'expiration' field is renamed as 'expiry'.
2023-03-31 14:55:08 +02:00
Sander van Grieken
d99a220c66 qml: add new 'removed_transaction' callback in wallet.py, hook up callback in qewallet and
emit balanceChanged events for add_transaction and remove_transaction
2023-03-31 12:32:02 +02:00
SomberNight
9df5f55a1f password unification: bugfix, now passes test cases
fixes https://github.com/spesmilo/electrum/issues/8259

note that technically this is an API change for
- wallet.check_password
- wallet.update_password
- storage.check_password
2023-03-20 20:07:53 +00:00
SomberNight
11f06d860e tests: add more tests for daemon.update_password_for_directory 2023-03-20 20:05:31 +00:00
SomberNight
b8d4ccd432 wallet: fix get_locktime_for_new_transaction for lagging server
Merchant reported that 0.5% of txs they make are rejected
by the connected server due to the locktime being in the future.

fixes https://github.com/spesmilo/electrum/issues/8245
2023-03-20 15:50:38 +00:00
ThomasV
8cc610298b QML: auto-delete expired requests. Add action to Qt menu 2023-03-18 09:59:18 +01:00
SomberNight
55da7276d3 qt export history/privkeys: put wallet name in path
closes https://github.com/spesmilo/electrum/issues/8255
2023-03-17 15:49:58 +00:00
ThomasV
503776c0de move fiat columns show/hide settings from settings_dialog to tab toolbars 2023-03-12 13:30:11 +01:00
SomberNight
d83863cc52 qt tx dialog: add checkbox "Download input data"
If checked, we download prev (parent) txs from the network, asynchronously.
This allows calculating the fee and showing "input addresses".

We could also SPV-verify the tx, to fill in missing tx_mined_status
(block height, blockhash, timestamp, short ids), but this is not done currently.
Note that there is no clean way to do this with electrum protocol 1.4:
`blockchain.transaction.get_merkle(tx_hash, height)` requires knowledge of the block height.

Loosely based on 6112fe0e51
2023-03-12 00:24:31 +00:00
SomberNight
81772faf6c wallet: add_input_info to no longer do network requests
- wallet.add_input_info() previously had a fallback to download parent
  prev txs from the network (after a lookup in wallet.db failed).
  wallet.add_input_info() is not async, so the network request cannot
  be done cleanly there and was really just a hack.
- tx.add_info_from_wallet() calls wallet.add_input_info() on each txin,
  in which case these network requests were done sequentially, not concurrently
- the network part of wallet.add_input_info() is now split out into new method:
  txin.add_info_from_network()
- in addition to tx.add_info_from_wallet(), there is now also tx.add_info_from_network()
  - callers of old tx.add_info_from_wallet() should now called either
    - tx.add_info_from_wallet(), then tx.add_info_from_network(), preferably in that order
    - tx.add_info_from_wallet() alone is sufficient if the tx is complete,
      or typically when not in a signing context
- callers of wallet.bump_fee and wallet.dscancel are now expected to have already
  called tx.add_info_from_network(), as it cannot be done in a non-async context
  (but for the common case of all-inputs-are-ismine, bump_fee/dscancel should work regardless)
- PartialTxInput.utxo was moved to the baseclass, TxInput.utxo
2023-03-12 00:21:57 +00:00
SomberNight
7746cc8e60 bip32: (trivial) rename method strpath_to_intpath, for symmetry
Required a much higher mental load to parse the name "convert_bip32_path_to_list_of_uint32"
than to parse "convert_bip32_strpath_to_intpath".
And we already have the ~inverse: "convert_bip32_intpath_to_strpath".
2023-03-10 14:23:17 +00:00
SomberNight
7584ba00ce wallet: kill negative conf numbers for TxMinedInfo
fixes https://github.com/spesmilo/electrum/issues/8240

#8240 was triggering an AssertionError in wallet.get_invoice_status,
as code there was assuming conf >= 0. To trigger, force-close
a LN channel, and while the sweep is waiting on the CSV, try to
make a payment in the Send tab to the ismine change address used
for the sweep in the future_tx. (order of events can also be reversed)
2023-03-09 14:59:08 +00:00
ThomasV
364510906f Fix edge case of batch_rbf, where we need to spend outputs from the base tx 2023-03-08 15:03:05 +01:00
ThomasV
3253e4904b Add rbf_merge_txid to PartialTransaction, instead of calling
get_unconfirmed_base_tx_for_batching a second time from GUI.
2023-03-08 12:38:41 +01:00
ThomasV
a5c7cc65ee make_unsigned_transaction: call get_unconfirmed_base_tx_for_batching lazily 2023-03-07 18:13:27 +01:00
ThomasV
74718e9085 confirm_tx_dialog: separate messages from warnings. add warnings about tx batching and spending unconfirmed coins 2023-03-06 19:25:46 +01:00
ThomasV
2f6d60c715 Move transaction related settings into transaction editor.
That way, users can see the effects settings directly on their transaction.
This changes the API of make_tx:
 - get_coins is called inside make_tx, so that inputs can be changed dynamically
 - make_tx takes an optional parameter: unconfirmed_only, passed to get_coins
 - ConfirmTxDialog detects if we can pay by disabling confirmed_only or lowering fee
2023-03-05 10:17:03 +01:00