Prevents `DeviceMgr.run()` from blocking the `Plugins` `DaemonThread` by
scheduling the hww timeout check instead of awaiting its result on the
`Plugins` thread.
If something in the `_hwd_comms_executor` thread is waiting for user
input, e.g. when setting up a hww in the wizard the user needs to
unlock the hww for `HardwareClientBase.get_xpub()` to return, the
`_hwd_comms_executor` is blocked. If then `DeviceMgr.run()` gets called by
the `Plugins` `DaemonThread` concurrently and tries to check the hww
timeout on the `_hwd_comms_executor` as well the `DaemonThread` is
blocked too until the `_hwd_comms_executor` gets unblocked (and the
`DaemonThread.job_lock` is taken.
Now if something tries to take the `DaemonThread.job_lock` it blocks as
well, so if a user e.g. tries to load a new plugin from the plugins
dialog the whole gui thread will freeze until the hww gets unlocked.
Adds functionality that allows the user to store the plugin authorization pubkey without having to edit files/registry manually.
On Linux systems it spawns the commands in a subprocess with pkexec which will trigger a OS prompt to execute the commands as root.
The user sees the executed commands and can either authorize with the root password or decline.
On windows it uses the windows `ShellExecuteExW` api to edit the registry, this also triggers a OS dialog to accept or decline (UAC dialog).
There is also functionality to reset the key again, which works in the same way.
store all plugin data by plugin name in a root dictionary `plugin_data`
inside the wallet db so that plugin data can get deleted again.
Prunes the data of plugins from the wallet db on wallet stop if the
plugin is not installed anymore.
If the plugin file got already deleted while being in the installation
dialog, trying to delete it again will raise an exception.
This is fixed by catching the exception.
If the user tries to install an external plugin that is already
installed, and then closes the PluginDialog, the PluginsDialog will
get into a bad state, throwing an exeption when opening it. This
happens because the add_plugin_dialog deletes the zipfile if the
user closes or cancels the installation dialog.
This is fixed by checking if the plugin is already existing,
instead of trying to install an already existing plugin.
- both internal and external plugins require GUI install
(except internal HW plugins, which are 'auto-loaded' and hidden)
- remove init_qt hook
- in Qt, reload wallet windows if plugin enabled/disabled
- add 'uninstall' button to PluginDialog
- add 'add plugins' button to wizard hw screen
- add icons to the plugin list
- Allow plugins saved as zipfiles in user data dir
- plugins are authorized with a user chosen password
- pubkey derived from password is saved with admin permissions
```
$ export ELECTRUM_LINTERS=E9,E101,E129,E273,E274,E703,E71,E722,F5,F6,F7,F8,W191,W29,B
$ export ELECTRUM_LINTERS_IGNORE=B007,B009,B010,B019,B036,F541,F841
$ flake8 . --count --select="$ELECTRUM_LINTERS" --ignore="$ELECTRUM_LINTERS_IGNORE" --show-source --statistics --exclude "*_pb2.py,electrum/_vendor/"
./electrum/commands.py:98:1: F811 redefinition of unused 'format_satoshis' from line 48
def format_satoshis(x):
^
./electrum/commands.py:437:9: F811 redefinition of unused 'Mnemonic' from line 62
from .mnemonic import Mnemonic
^
./electrum/gui/qt/wizard/wallet.py:37:5: F811 redefinition of unused 'Daemon' from line 14
from electrum.daemon import Daemon
^
./electrum/lntransport.py:14:1: F811 redefinition of unused 'Optional' from line 12
from typing import NamedTuple, List, Tuple, Mapping, Optional, TYPE_CHECKING, Union, Dict, Set, Sequence
^
./electrum/lntransport.py:14:1: F811 redefinition of unused 'TYPE_CHECKING' from line 12
from typing import NamedTuple, List, Tuple, Mapping, Optional, TYPE_CHECKING, Union, Dict, Set, Sequence
^
./electrum/plugin.py:966:13: F811 redefinition of unused 'hid' from line 593
import hid
^
./electrum/plugin.py:1040:13: F811 redefinition of unused 'hid' from line 593
import hid
^
./electrum/util.py:44:1: F811 redefinition of unused 'json' from line 26
import json
^
./electrum/util.py:46:1: F811 redefinition of unused 'NamedTuple' from line 29
from typing import NamedTuple, Optional
^
./electrum/util.py:46:1: F811 redefinition of unused 'Optional' from line 29
from typing import NamedTuple, Optional
^
./electrum/util.py:1456:56: F811 redefinition of unused 'traceback' from line 34
async def __aexit__(self, exc_type, exc_value, traceback):
^
./electrum/wallet_db.py:536:9: F811 redefinition of unused 'LOCAL' from line 46
LOCAL = 1
^
./electrum/wallet_db.py:537:9: F811 redefinition of unused 'REMOTE' from line 46
REMOTE = -1
^
./tests/test_bitcoin.py:28:1: F811 redefinition of unused 'bitcoin' from line 9
from electrum import crypto, constants, bitcoin
^
./tests/test_txbatcher.py:11:1: F811 redefinition of unused 'Transaction' from line 7
from electrum.transaction import Transaction, PartialTxInput, PartialTxOutput, TxOutpoint
^
./tests/test_wallet_vertical.py:20:1: F811 redefinition of unused 'Transaction' from line 10
from electrum.transaction import Transaction, PartialTxOutput, tx_from_any, Sighash
^
16 F811 redefinition of unused 'format_satoshis' from line 48
16
```
```
./electrum/commands.py:144:9: F824 `global known_commands` is unused: name is never assigned in scope
global known_commands
^
./electrum/commands.py:1916:9: F824 `global known_commands` is unused: name is never assigned in scope
global known_commands
^
./electrum/gui/qt/main_window.py:2405:13: F824 `nonlocal done` is unused: name is never assigned in scope
nonlocal done
^
./electrum/i18n.py:52:5: F824 `global language` is unused: name is never assigned in scope
global language
^
./electrum/plugin.py:189:9: F824 `global _root_permission_cache` is unused: name is never assigned in scope
global _root_permission_cache
^
5 F824 `global known_commands` is unused: name is never assigned in scope
5
```
and restore ability to have different internal ConfigVar name and user-visible "key"
(Keys are hard to change as that breaks compat, but it is nice to be able to change
the internal var name, to reorganise stuff sometimes. After new ConfigVars are added,
sometimes we get better insight into how the older ones should have been named.)
follow-up https://github.com/spesmilo/electrum/pull/9648
When importing a plugin, if it raised an exception in its `__init__` file, we
ignored it, and still loaded the plugin, in a potentially half-broken state.
This is because maybe_load_plugin_init_method only calls exec_module_from_spec
if the plugin is not already in sys.modules, but exec_module_from_spec
will put the plugin into sys.modules even if it errors.
Consider this patch to test with, enable the "labels" plugin:
```patch
diff --git a/electrum/plugins/labels/__init__.py b/electrum/plugins/labels/__init__.py
index b68127df8e..0d6d95abce 100644
--- a/electrum/plugins/labels/__init__.py
+++ b/electrum/plugins/labels/__init__.py
@@ -21,3 +21,5 @@ async def pull(self: 'Commands', plugin: 'LabelsPlugin' = None, wallet=None, for
arg:bool:force:pull all labels
"""
return await plugin.pull_thread(wallet, force=force)
+
+raise Exception("heyheyhey")
```
I would expect we don't load the labels plugin due to the error, but we do:
```
>>> plugins.get_plugin("labels")
<electrum.plugins.labels.qt.Plugin object at 0x7801df30fb50>
```
Log:
```
$ ./run_electrum -v --testnet -o
0.75 | I | simple_config.SimpleConfig | electrum directory /home/user/.electrum/testnet
0.75 | E | p/plugin.Plugins | cannot initialize plugin labels: Error pre-loading electrum.plugins.labels: Exception('heyheyhey')
Traceback (most recent call last):
File "/home/user/wspace/electrum/electrum/plugin.py", line 148, in exec_module_from_spec
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/home/user/wspace/electrum/electrum/plugins/labels/__init__.py", line 25, in <module>
raise Exception("heyheyhey")
Exception: heyheyhey
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/wspace/electrum/electrum/plugin.py", line 167, in load_plugins
self.maybe_load_plugin_init_method(name)
File "/home/user/wspace/electrum/electrum/plugin.py", line 293, in maybe_load_plugin_init_method
module = self.exec_module_from_spec(init_spec, base_name)
File "/home/user/wspace/electrum/electrum/plugin.py", line 150, in exec_module_from_spec
raise Exception(f"Error pre-loading {path}: {repr(e)}") from e
Exception: Error pre-loading electrum.plugins.labels: Exception('heyheyhey')
0.75 | D | util.profiler | Plugins.__init__ 0.0030 sec
0.84 | I | simple_config.SimpleConfig | electrum directory /home/user/.electrum/testnet
0.89 | I | __main__ | get_default_language: detected default as lang='en_UK'
0.89 | I | i18n | setting language to 'en_UK'
0.89 | I | logging | Electrum version: 4.5.8 - https://electrum.org - https://github.com/spesmilo/electrum
0.89 | I | logging | Python version: 3.10.12 (main, Feb 4 2025, 14:57:36) [GCC 11.4.0]. On platform: Linux-6.8.0-52-generic-x86_64-with-glibc2.35
0.89 | I | logging | Logging to file: /home/user/.electrum/testnet/logs/electrum_log_20250319T161247Z_6605.log
0.89 | I | logging | Log filters: verbosity '*', verbosity_shortcuts ''
0.89 | I | exchange_rate.FxThread | using exchange CoinGecko
0.90 | D | util.profiler | Daemon.__init__ 0.0047 sec
0.90 | I | daemon.Daemon | starting taskgroup.
0.90 | I | daemon.CommandsServer | now running and listening. socktype=unix, addr=/home/user/.electrum/testnet/daemon_rpc_socket
0.90 | I | p/plugin.Plugins | registering hardware bitbox02: ['hardware', 'bitbox02', 'BitBox02']
0.90 | I | p/plugin.Plugins | registering hardware coldcard: ['hardware', 'coldcard', 'Coldcard Wallet']
0.90 | I | p/plugin.Plugins | registering hardware digitalbitbox: ['hardware', 'digitalbitbox', 'Digital Bitbox wallet']
0.90 | I | p/plugin.Plugins | could not find manifest.json of plugin hw_wallet, skipping...
0.90 | I | p/plugin.Plugins | registering hardware jade: ['hardware', 'jade', 'Jade wallet']
0.90 | I | p/plugin.Plugins | registering hardware keepkey: ['hardware', 'keepkey', 'KeepKey wallet']
0.90 | I | p/plugin.Plugins | registering hardware ledger: ['hardware', 'ledger', 'Ledger wallet']
0.90 | I | p/plugin.Plugins | registering hardware safe_t: ['hardware', 'safe_t', 'Safe-T mini wallet']
0.90 | I | p/plugin.Plugins | registering hardware trezor: ['hardware', 'trezor', 'Trezor wallet']
0.90 | I | p/plugin.Plugins | registering wallet type ('2fa', 'trustedcoin')
1.01 | I | p/plugin.Plugins | loaded plugin 'labels'. (from thread: 'GUI')
1.01 | D | util.profiler | Plugins.__init__ 0.1183 sec
```
SomberNight