- both internal and external plugins require GUI install
(except internal HW plugins, which are 'auto-loaded' and hidden)
- remove init_qt hook
- in Qt, reload wallet windows if plugin enabled/disabled
- add 'uninstall' button to PluginDialog
- add 'add plugins' button to wizard hw screen
- add icons to the plugin list
- Allow plugins saved as zipfiles in user data dir
- plugins are authorized with a user chosen password
- pubkey derived from password is saved with admin permissions
```
$ export ELECTRUM_LINTERS=E9,E101,E129,E273,E274,E703,E71,E722,F5,F6,F7,F8,W191,W29,B
$ export ELECTRUM_LINTERS_IGNORE=B007,B009,B010,B019,B036,F541,F841
$ flake8 . --count --select="$ELECTRUM_LINTERS" --ignore="$ELECTRUM_LINTERS_IGNORE" --show-source --statistics --exclude "*_pb2.py,electrum/_vendor/"
./electrum/commands.py:98:1: F811 redefinition of unused 'format_satoshis' from line 48
def format_satoshis(x):
^
./electrum/commands.py:437:9: F811 redefinition of unused 'Mnemonic' from line 62
from .mnemonic import Mnemonic
^
./electrum/gui/qt/wizard/wallet.py:37:5: F811 redefinition of unused 'Daemon' from line 14
from electrum.daemon import Daemon
^
./electrum/lntransport.py:14:1: F811 redefinition of unused 'Optional' from line 12
from typing import NamedTuple, List, Tuple, Mapping, Optional, TYPE_CHECKING, Union, Dict, Set, Sequence
^
./electrum/lntransport.py:14:1: F811 redefinition of unused 'TYPE_CHECKING' from line 12
from typing import NamedTuple, List, Tuple, Mapping, Optional, TYPE_CHECKING, Union, Dict, Set, Sequence
^
./electrum/plugin.py:966:13: F811 redefinition of unused 'hid' from line 593
import hid
^
./electrum/plugin.py:1040:13: F811 redefinition of unused 'hid' from line 593
import hid
^
./electrum/util.py:44:1: F811 redefinition of unused 'json' from line 26
import json
^
./electrum/util.py:46:1: F811 redefinition of unused 'NamedTuple' from line 29
from typing import NamedTuple, Optional
^
./electrum/util.py:46:1: F811 redefinition of unused 'Optional' from line 29
from typing import NamedTuple, Optional
^
./electrum/util.py:1456:56: F811 redefinition of unused 'traceback' from line 34
async def __aexit__(self, exc_type, exc_value, traceback):
^
./electrum/wallet_db.py:536:9: F811 redefinition of unused 'LOCAL' from line 46
LOCAL = 1
^
./electrum/wallet_db.py:537:9: F811 redefinition of unused 'REMOTE' from line 46
REMOTE = -1
^
./tests/test_bitcoin.py:28:1: F811 redefinition of unused 'bitcoin' from line 9
from electrum import crypto, constants, bitcoin
^
./tests/test_txbatcher.py:11:1: F811 redefinition of unused 'Transaction' from line 7
from electrum.transaction import Transaction, PartialTxInput, PartialTxOutput, TxOutpoint
^
./tests/test_wallet_vertical.py:20:1: F811 redefinition of unused 'Transaction' from line 10
from electrum.transaction import Transaction, PartialTxOutput, tx_from_any, Sighash
^
16 F811 redefinition of unused 'format_satoshis' from line 48
16
```
```
./electrum/commands.py:144:9: F824 `global known_commands` is unused: name is never assigned in scope
global known_commands
^
./electrum/commands.py:1916:9: F824 `global known_commands` is unused: name is never assigned in scope
global known_commands
^
./electrum/gui/qt/main_window.py:2405:13: F824 `nonlocal done` is unused: name is never assigned in scope
nonlocal done
^
./electrum/i18n.py:52:5: F824 `global language` is unused: name is never assigned in scope
global language
^
./electrum/plugin.py:189:9: F824 `global _root_permission_cache` is unused: name is never assigned in scope
global _root_permission_cache
^
5 F824 `global known_commands` is unused: name is never assigned in scope
5
```
and restore ability to have different internal ConfigVar name and user-visible "key"
(Keys are hard to change as that breaks compat, but it is nice to be able to change
the internal var name, to reorganise stuff sometimes. After new ConfigVars are added,
sometimes we get better insight into how the older ones should have been named.)
follow-up https://github.com/spesmilo/electrum/pull/9648
When importing a plugin, if it raised an exception in its `__init__` file, we
ignored it, and still loaded the plugin, in a potentially half-broken state.
This is because maybe_load_plugin_init_method only calls exec_module_from_spec
if the plugin is not already in sys.modules, but exec_module_from_spec
will put the plugin into sys.modules even if it errors.
Consider this patch to test with, enable the "labels" plugin:
```patch
diff --git a/electrum/plugins/labels/__init__.py b/electrum/plugins/labels/__init__.py
index b68127df8e..0d6d95abce 100644
--- a/electrum/plugins/labels/__init__.py
+++ b/electrum/plugins/labels/__init__.py
@@ -21,3 +21,5 @@ async def pull(self: 'Commands', plugin: 'LabelsPlugin' = None, wallet=None, for
arg:bool:force:pull all labels
"""
return await plugin.pull_thread(wallet, force=force)
+
+raise Exception("heyheyhey")
```
I would expect we don't load the labels plugin due to the error, but we do:
```
>>> plugins.get_plugin("labels")
<electrum.plugins.labels.qt.Plugin object at 0x7801df30fb50>
```
Log:
```
$ ./run_electrum -v --testnet -o
0.75 | I | simple_config.SimpleConfig | electrum directory /home/user/.electrum/testnet
0.75 | E | p/plugin.Plugins | cannot initialize plugin labels: Error pre-loading electrum.plugins.labels: Exception('heyheyhey')
Traceback (most recent call last):
File "/home/user/wspace/electrum/electrum/plugin.py", line 148, in exec_module_from_spec
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/home/user/wspace/electrum/electrum/plugins/labels/__init__.py", line 25, in <module>
raise Exception("heyheyhey")
Exception: heyheyhey
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/wspace/electrum/electrum/plugin.py", line 167, in load_plugins
self.maybe_load_plugin_init_method(name)
File "/home/user/wspace/electrum/electrum/plugin.py", line 293, in maybe_load_plugin_init_method
module = self.exec_module_from_spec(init_spec, base_name)
File "/home/user/wspace/electrum/electrum/plugin.py", line 150, in exec_module_from_spec
raise Exception(f"Error pre-loading {path}: {repr(e)}") from e
Exception: Error pre-loading electrum.plugins.labels: Exception('heyheyhey')
0.75 | D | util.profiler | Plugins.__init__ 0.0030 sec
0.84 | I | simple_config.SimpleConfig | electrum directory /home/user/.electrum/testnet
0.89 | I | __main__ | get_default_language: detected default as lang='en_UK'
0.89 | I | i18n | setting language to 'en_UK'
0.89 | I | logging | Electrum version: 4.5.8 - https://electrum.org - https://github.com/spesmilo/electrum
0.89 | I | logging | Python version: 3.10.12 (main, Feb 4 2025, 14:57:36) [GCC 11.4.0]. On platform: Linux-6.8.0-52-generic-x86_64-with-glibc2.35
0.89 | I | logging | Logging to file: /home/user/.electrum/testnet/logs/electrum_log_20250319T161247Z_6605.log
0.89 | I | logging | Log filters: verbosity '*', verbosity_shortcuts ''
0.89 | I | exchange_rate.FxThread | using exchange CoinGecko
0.90 | D | util.profiler | Daemon.__init__ 0.0047 sec
0.90 | I | daemon.Daemon | starting taskgroup.
0.90 | I | daemon.CommandsServer | now running and listening. socktype=unix, addr=/home/user/.electrum/testnet/daemon_rpc_socket
0.90 | I | p/plugin.Plugins | registering hardware bitbox02: ['hardware', 'bitbox02', 'BitBox02']
0.90 | I | p/plugin.Plugins | registering hardware coldcard: ['hardware', 'coldcard', 'Coldcard Wallet']
0.90 | I | p/plugin.Plugins | registering hardware digitalbitbox: ['hardware', 'digitalbitbox', 'Digital Bitbox wallet']
0.90 | I | p/plugin.Plugins | could not find manifest.json of plugin hw_wallet, skipping...
0.90 | I | p/plugin.Plugins | registering hardware jade: ['hardware', 'jade', 'Jade wallet']
0.90 | I | p/plugin.Plugins | registering hardware keepkey: ['hardware', 'keepkey', 'KeepKey wallet']
0.90 | I | p/plugin.Plugins | registering hardware ledger: ['hardware', 'ledger', 'Ledger wallet']
0.90 | I | p/plugin.Plugins | registering hardware safe_t: ['hardware', 'safe_t', 'Safe-T mini wallet']
0.90 | I | p/plugin.Plugins | registering hardware trezor: ['hardware', 'trezor', 'Trezor wallet']
0.90 | I | p/plugin.Plugins | registering wallet type ('2fa', 'trustedcoin')
1.01 | I | p/plugin.Plugins | loaded plugin 'labels'. (from thread: 'GUI')
1.01 | D | util.profiler | Plugins.__init__ 0.1183 sec
```
Correction to comment in prev commit (and removing it here):
spec.loader.exec_module does not spawn new threads, it simply
executes the module in the current thread.
I got confused but turns out "load_plugin" itself is sometimes
not called from the main thread. Specifically (e.g.), since the recent
wizard rewrite, in the qt gui, the wizard loads the hww plugins
from a new thread.
This now better explains the macos hww crashes: they had started
appearing because we upgraded hidapi (which made it more sensitive
to having to import from main thread) AND scanning(->importing) from
the wizard no longer happened on the main thread after the rewrite.
Plugins should be thread-safe in terms of where they are imported from.
Let's log the importer thread's name (added here), to help recognise
related threading issues.
Ah ok, I give up for now... the prev does not really work.
The prior commit works on Windows but not on macOS.
On Windows, it would package all plugins as code and only as code.
On MacOS however, it would not package any plugins at all. And with this commit,
where I mark the plugins folder to be packaged as *data*, it packages all plugins as *both* code and data.
Not sure why.
Let's just package all plugins as both code and data, and ignore the code instances explicitly...
(=plugin name collision)
This assert is currently failing in pyinstaller builds
(probably have been since it was added in 552bfb589a).
Looks like the pyinstaller binaries include two copies of the hw_wallet and the jade plugins.
This likely has been the case for years, we just never noticed. -- but it started triggering the new assert.
```
2.78 | I | plugin | iter_modules=[
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='audio_modem', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='bitbox02', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='coldcard', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='cosigner_pool', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='digitalbitbox', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='hw_wallet', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='jade', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='keepkey', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='labels', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='ledger', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='payserver', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='revealer', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='safe_t', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='swapserver', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='trezor', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='trustedcoin', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='virtualkeyboard', ispkg=True),
ModuleInfo(module_finder=<pyimod02_importers.FrozenImporter object at 0x015FDFB8>, name='hw_wallet', ispkg=True),
ModuleInfo(module_finder=<pyimod02_importers.FrozenImporter object at 0x015FDFB8>, name='jade', ispkg=True)]
```
Plugins.stop() is now part of the normal shutdown flow (since 90f39bce88),
so this shaves up to 100 ms off that (avg: 50 ms).
It is also waited on in around ~60 unit tests: this saves 3 sec.
This method is often called when there is already an existing paired
client for the keystore, in which case we can avoid scan_devices() -
which would needlessly take several seconds.
Related to prev commit: multiple (compatible) keystores can now match
with the same HardwareClientBase, so we might as well also allow
reusing existing Clients for the wizard.
- the DeviceMgr no longer uses xpubs to keep track of paired hw devices
- instead, introduce keystore.pairing_code(), based on soft_device_id
- xpubs are now only used in a single place when the actual pairing happens
- motivation is to allow pairing a single device with multiple generic
output script descriptors, not just a single account-level xpub
- as a side-effect, we now allow pairing a device with multiple open
windows simultaneously (if keystores have the same root fingerprint
-- was already the case before if keystores had the same xpub)
