A new config API is introduced, and ~all of the codebase is adapted to it.
The old API is kept but mainly only for dynamic usage where its extra flexibility is needed.
Using examples, the old config API looked this:
```
>>> config.get("request_expiry", 86400)
604800
>>> config.set_key("request_expiry", 86400)
>>>
```
The new config API instead:
```
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS
604800
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS = 86400
>>>
```
The old API operated on arbitrary string keys, the new one uses
a static ~enum-like list of variables.
With the new API:
- there is a single centralised list of config variables, as opposed to
these being scattered all over
- no more duplication of default values (in the getters)
- there is now some (minimal for now) type-validation/conversion for
the config values
closes https://github.com/spesmilo/electrum/pull/5640
closes https://github.com/spesmilo/electrum/pull/5649
Note: there is yet a third API added here, for certain niche/abstract use-cases,
where we need a reference to the config variable itself.
It should only be used when needed:
```
>>> var = config.cv.WALLET_PAYREQ_EXPIRY_SECONDS
>>> var
<ConfigVarWithConfig key='request_expiry'>
>>> var.get()
604800
>>> var.set(3600)
>>> var.get_default_value()
86400
>>> var.is_set()
True
>>> var.is_modifiable()
True
```
- case 1: in version 4.4.1, 4.4.2, the qml GUI wizard allowed creating multisig wallets with an old_mpk as cosigner.
- case 2: in version 4.4.0, 4.4.1, 4.4.2, the qml GUI wizard allowed creating multisig wallets with mixed xpub/Ypub/Zpub.
The corresponding missing input validation was a bug in the wizard, it was unintended behaviour. Validation was added in d2cf21fc2b. Note however that there might be users who created such wallet files.
Re case 1 wallet files: there is no version of Electrum that allows spending from such a wallet. Coins received at addresses are not burned, however it is technically challenging to spend them. (unless the multisig can spend without needing the old_mpk cosigner in the quorum).
Re case 2 wallet files: it is possible to create a corresponding spending wallet for such a multisig, however it is a bit tricky. The script type for the addresses in such a heterogeneous xpub wallet is based on the xpub_type of the first keystore. So e.g. given a wallet file [Yprv1, Zpub2] it will have sh(wsh()) scripts, and the cosigner should create a wallet file [Ypub1, Zprv2] (same order).
Technically case 2 wallet files could be "fixed" automatically by converting the xpub types as part of a wallet_db upgrade. However if the wallet files also contain seeds, those cannot be converted ("standard" vs "segwit" electrum seed).
Case 1 wallet files are not possible to "fix" automatically as the cosigner using the old_mpk is not bip32 based.
It is unclear if there are *any* users out there affected by this. I suspect for case 1 it is very likely there are none (not many people have pre-2.0 electrum seeds which were never supported as part of a multisig who would also now try to create a multisig using them); for case 2 however there might be.
This commit breaks both case 1 and case 2 wallets: these wallet files can no longer be opened in new Electrum, an error message is shown and the crash reporter opens. If any potential users opt to send crash reports, at least we will know they exist and can help them recover.
The qt, qml, and kivy GUIs have a first-start network-setup screen
that allows the user customising the network settings before creating a wallet.
Previously the daemon used to create the network and start it, before this screen,
before the GUI even starts. If the user changed network settings, those would
be set on the already running network, potentially including restarting the network.
Now it becomes the responsibility of the GUI to start the network, allowing this
first-start customisation to take place before starting the network at all.
The qt and the qml GUIs are adapted to make use of this. Kivy, and the other
prototype GUIs are not adapted and just start the network right away, as before.
If I use apt on ubuntu to install python3-pyqt5 and then remove it, it leaves an empty namespace:
```
$ python3
Python 3.8.10 (default, Mar 15 2022, 12:22:08)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import PyQt5
>>> import PyQt5.QtGui
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'PyQt5.QtGui'
>>> PyQt5
<module 'PyQt5' (namespace)>
```
fixes https://github.com/spesmilo/electrum/issues/7447
Consider this trace for 4.2.0:
```
Traceback (most recent call last):
File "electrum/gui/qt/__init__.py", line 332, in start_new_window
File "electrum/gui/qt/__init__.py", line 363, in _start_wizard_to_select_or_create_wallet
File "electrum/gui/qt/installwizard.py", line 302, in select_storage
File "electrum/util.py", line 504, in get_new_wallet_name
PermissionError: [Errno 1] Operation not permitted: '/Users/admin/Documents/Peach/MS'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "electrum/gui/qt/__init__.py", line 426, in main
File "electrum/gui/qt/__init__.py", line 307, in wrapper
File "electrum/gui/qt/__init__.py", line 349, in start_new_window
File "electrum/util.py", line 504, in get_new_wallet_name
PermissionError: [Errno 1] Operation not permitted: '/Users/admin/Documents/Peach/MS'
```
Note that `get_new_wallet_name` (os.listdir) can raise OSError,
and we were calling that on the main entrypoint codepath without exception-handling.
We were also calling it in the fallback codepath without exception-handling.
i.e. the GUI errored out on every startup for affected users, and without CLI usage
it was not possible to recover.
Some exceptions were just killing the gui silently and not even logged.
E.g.:
```
E | gui.qt.ElectrumGui | error loading wallet (or creating window for it)
Traceback (most recent call last):
File "/opt/electrum/electrum/gui/qt/__init__.py", line 433, in main
if not self.start_new_window(path, self.config.get('url'), app_is_starting=True):
File "/opt/electrum/electrum/gui/qt/__init__.py", line 307, in wrapper
return func(self, *args, **kwargs)
File "/opt/electrum/electrum/gui/qt/__init__.py", line 332, in start_new_window
wallet = self._start_wizard_to_select_or_create_wallet(path)
File "/opt/electrum/electrum/gui/qt/__init__.py", line 377, in _start_wizard_to_select_or_create_wallet
db = WalletDB(storage.read(), manual_upgrades=False)
File "/opt/electrum/electrum/wallet_db.py", line 73, in __init__
self.load_data(raw)
File "/opt/electrum/electrum/wallet_db.py", line 104, in load_data
self._after_upgrade_tasks()
File "/opt/electrum/electrum/wallet_db.py", line 202, in _after_upgrade_tasks
self._load_transactions()
File "/opt/electrum/electrum/util.py", line 439, in <lambda>
return lambda *args, **kw_args: do_profile(args, kw_args)
File "/opt/electrum/electrum/util.py", line 435, in do_profile
o = func(*args, **kw_args)
File "/opt/electrum/electrum/wallet_db.py", line 1310, in _load_transactions
self.data = StoredDict(self.data, self, [])
File "/opt/electrum/electrum/json_db.py", line 79, in __init__
self.__setitem__(k, v)
File "/opt/electrum/electrum/json_db.py", line 44, in wrapper
return func(self, *args, **kwargs)
File "/opt/electrum/electrum/json_db.py", line 97, in __setitem__
v = self.db._convert_dict(self.path, key, v)
File "/opt/electrum/electrum/wallet_db.py", line 1361, in _convert_dict
v = dict((k, SwapData(**x)) for k, x in v.items())
```
If daemon.taskgroup dies
- in GUI mode, show a crash reporter window to the user,
instead of immediately stopping the whole process.
- in daemon mode, log exception and stop process, as before.
related: https://github.com/spesmilo/electrum/issues/7390
```
20210706T091826.513398Z | ERROR | __main__ | daemon.run_gui errored
Traceback (most recent call last):
File "run_electrum", line 407, in handle_cmd
File "electrum\daemon.py", line 584, in run_gui
File "electrum\gui\qt\__init__.py", line 414, in main
File "electrum\gui\qt\__init__.py", line 291, in wrapper
File "electrum\gui\qt\__init__.py", line 316, in start_new_window
File "electrum\gui\qt\__init__.py", line 361, in _start_wizard_to_select_or_create_wallet
File "electrum\wallet_db.py", line 73, in __init__
File "electrum\wallet_db.py", line 106, in load_data
File "electrum\util.py", line 412, in <lambda>
File "electrum\util.py", line 408, in do_profile
File "electrum\wallet_db.py", line 175, in upgrade
File "electrum\wallet_db.py", line 540, in _convert_version_24
ValueError: too many values to unpack (expected 2)
```
This commit ports the work of EchterAgo and cculianu from Electron-Cash,
to implement a new toolchain to scan qr codes.
Previously, on Linux and Win, we have been using zbar to access the camera
and read qrcodes; and on macOS we used CalinsQRReader (an objective-C
project by cculianu).
The new toolchain added here can use QtMultimedia to access the camera,
and then feed that image into zbar. When used this way, zbar needs
fewer dependencies and is easier to compile, in particular it can be
compiled for macOS.
The new toolchain works on all three platforms, with some caveats
(see code comments in related commits) -- so we also keep the end-to-end
zbar toolchain; but at least we can drop CalinsQRReader.
The related changes in Electron-Cash are spread over 50+ commits (several PRs and direct
pushes to master), but see in particular:
https://github.com/Electron-Cash/Electron-Cash/pull/1376
some other interesting links:
b2b737001c163224cf1f3b31e0fcb1eda015908ehttps://github.com/Electron-Cash/Electron-Cash/pull/1545052aa06c23
There is gui.stop() already, which does the same thing (which is shared API with kivy).
Also, the _cleanup_before_exit() call was redundant in close(),
aboutToQuit handles that.
related: https://github.com/spesmilo/electrum/issues/6889
This fixes the case where the user quits by pressing Ctrl+C,
and some other minor things.
There is still another issue that sometimes causes a segfault during shutdown...
It is not realistic to expect Electrum to be used as a watchtower
in GUI mode, and possibly counter-productive (may set wrong
expectations).
A proper watchtower should be configured as a daemon. The
documentation will be updated to reflect this change.
The gossip db is loaded early when the network is started to save
time when the gui is locked and a wallet not yet loaded. Side effects
of the LNWallet to start peering when a channel db is loaded is
circumvented.
Just makes sense in general.
Also, previously, the GUI would freeze if right after startup the user
clicked the hww status bar icon (especially with multiple hww connected).
