Correction to comment in prev commit (and removing it here):
spec.loader.exec_module does not spawn new threads, it simply
executes the module in the current thread.
I got confused but turns out "load_plugin" itself is sometimes
not called from the main thread. Specifically (e.g.), since the recent
wizard rewrite, in the qt gui, the wizard loads the hww plugins
from a new thread.
This now better explains the macos hww crashes: they had started
appearing because we upgraded hidapi (which made it more sensitive
to having to import from main thread) AND scanning(->importing) from
the wizard no longer happened on the main thread after the rewrite.
Plugins should be thread-safe in terms of where they are imported from.
Let's log the importer thread's name (added here), to help recognise
related threading issues.
Ah ok, I give up for now... the prev does not really work.
The prior commit works on Windows but not on macOS.
On Windows, it would package all plugins as code and only as code.
On MacOS however, it would not package any plugins at all. And with this commit,
where I mark the plugins folder to be packaged as *data*, it packages all plugins as *both* code and data.
Not sure why.
Let's just package all plugins as both code and data, and ignore the code instances explicitly...
(=plugin name collision)
This assert is currently failing in pyinstaller builds
(probably have been since it was added in 552bfb589a).
Looks like the pyinstaller binaries include two copies of the hw_wallet and the jade plugins.
This likely has been the case for years, we just never noticed. -- but it started triggering the new assert.
```
2.78 | I | plugin | iter_modules=[
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='audio_modem', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='bitbox02', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='coldcard', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='cosigner_pool', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='digitalbitbox', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='hw_wallet', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='jade', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='keepkey', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='labels', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='ledger', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='payserver', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='revealer', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='safe_t', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='swapserver', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='trezor', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='trustedcoin', ispkg=True),
ModuleInfo(module_finder=FileFinder('C:\\Program Files (x86)\\Electrum\\electrum\\plugins'), name='virtualkeyboard', ispkg=True),
ModuleInfo(module_finder=<pyimod02_importers.FrozenImporter object at 0x015FDFB8>, name='hw_wallet', ispkg=True),
ModuleInfo(module_finder=<pyimod02_importers.FrozenImporter object at 0x015FDFB8>, name='jade', ispkg=True)]
```
Plugins.stop() is now part of the normal shutdown flow (since 90f39bce88),
so this shaves up to 100 ms off that (avg: 50 ms).
It is also waited on in around ~60 unit tests: this saves 3 sec.
This method is often called when there is already an existing paired
client for the keystore, in which case we can avoid scan_devices() -
which would needlessly take several seconds.
Related to prev commit: multiple (compatible) keystores can now match
with the same HardwareClientBase, so we might as well also allow
reusing existing Clients for the wizard.
- the DeviceMgr no longer uses xpubs to keep track of paired hw devices
- instead, introduce keystore.pairing_code(), based on soft_device_id
- xpubs are now only used in a single place when the actual pairing happens
- motivation is to allow pairing a single device with multiple generic
output script descriptors, not just a single account-level xpub
- as a side-effect, we now allow pairing a device with multiple open
windows simultaneously (if keystores have the same root fingerprint
-- was already the case before if keystores had the same xpub)
* typecheck client selected by keystore - check if plugin types match
* add log msg + refactor if branching
* moved type check to own function; moved from 'client_for_keystore' to 'force_pair_xpub' and 'client_by_xpub'
* refactor
Co-authored-by: avirgovi <avirgovi@cisco.com>
ledger now gives an error if querying xpub while device is not (unlocked and in bitcoin app).
we do query the xpub however, to calc root fingerprint to be used as soft device id.
trace:
I | plugin.DeviceMgr | scanning devices...
D | util.profiler | DeviceMgr.scan_devices 3.4463
I | plugin.DeviceMgr | Registering <electrum.plugins.ledger.ledger.Ledger_Client object at 0x0000029DF6B08520>
E | gui.qt.installwizard.InstallWizard |
Traceback (most recent call last):
File "...\electrum\electrum\plugins\ledger\ledger.py", line 225, in checkDevice
self.perform_hw1_preflight()
File "...\electrum\electrum\plugin.py", line 362, in wrapper
return run_in_hwd_thread(partial(func, *args, **kwargs))
File "...\electrum\electrum\plugin.py", line 352, in run_in_hwd_thread
return func()
File "...\electrum\electrum\plugins\ledger\ledger.py", line 219, in perform_hw1_preflight
raise e
File "...\electrum\electrum\plugins\ledger\ledger.py", line 179, in perform_hw1_preflight
firmwareInfo = self.dongleObject.getFirmwareVersion()
File "...\Python38\site-packages\btchip\btchip.py", line 563, in getFirmwareVersion
response = self.dongle.exchange(bytearray(apdu))
File "...\Python38\site-packages\btchip\btchipComm.py", line 127, in exchange
raise BTChipException("Invalid status %04x" % sw, sw)
btchip.btchipException.BTChipException: Exception : Invalid status 6700
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "...\electrum\electrum\base_wizard.py", line 317, in _choose_hw_device
device_infos = devmgr.unpaired_device_infos(None, plugin, devices=scanned_devices,
File "...\electrum\electrum\plugin.py", line 612, in unpaired_device_infos
soft_device_id=client.get_soft_device_id(),
File "...\electrum\electrum\plugin.py", line 362, in wrapper
return run_in_hwd_thread(partial(func, *args, **kwargs))
File "...\electrum\electrum\plugin.py", line 355, in run_in_hwd_thread
return fut.result()
File "...\Python38\lib\concurrent\futures\_base.py", line 439, in result
return self.__get_result()
File "...\Python38\lib\concurrent\futures\_base.py", line 388, in __get_result
raise self._exception
File "...\Python38\lib\concurrent\futures\thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "...\electrum\electrum\plugins\ledger\ledger.py", line 91, in get_soft_device_id
self._soft_device_id = self.request_root_fingerprint_from_device()
File "...\electrum\electrum\plugin.py", line 362, in wrapper
return run_in_hwd_thread(partial(func, *args, **kwargs))
File "...\electrum\electrum\plugin.py", line 352, in run_in_hwd_thread
return func()
File "...\electrum\electrum\plugins\hw_wallet\plugin.py", line 259, in request_root_fingerprint_from_device
child_of_root_xpub = self.get_xpub("m/0'", xtype='standard')
File "...\electrum\electrum\plugin.py", line 362, in wrapper
return run_in_hwd_thread(partial(func, *args, **kwargs))
File "...\electrum\electrum\plugin.py", line 352, in run_in_hwd_thread
return func()
File "...\electrum\electrum\plugins\ledger\ledger.py", line 57, in catch_exception
return func(self, *args, **kwargs)
File "...\electrum\electrum\plugins\ledger\ledger.py", line 111, in get_xpub
self.checkDevice()
File "...\electrum\electrum\plugin.py", line 362, in wrapper
return run_in_hwd_thread(partial(func, *args, **kwargs))
File "...\electrum\electrum\plugin.py", line 352, in run_in_hwd_thread
return func()
File "...\electrum\electrum\plugins\ledger\ledger.py", line 228, in checkDevice
raise UserFacingException(_("Device not in Bitcoin mode")) from e
electrum.util.UserFacingException: Device not in Bitcoin mode
W | gui.qt.installwizard.InstallWizard | error getting device infos for ledger: Device not in Bitcoin mode
shesek reported on IRC:
> the button widget for opening plugins configuration gets cached in `settings_widgets`
> even after the plugin is disabled and re-enabled, which causes it to call `settings_dialog()`
> on the previous plugin instance that got unloaded instead of the new one.
This commit adds support for the BitBox02 hardware wallet.
It supports both single and multisig for the electrum gui wallet.
To use the plugin a local installation of the BitBox02 python library is
required. It can be found on PiPy under the name 'bitbox02' and can be
installed from the bitbox02-firmware repository in the py/bitbox02
directory.
All communication to and from the BitBox02 is noise encrypted, the keys
required for this are stored in the wallet config file under the
bitbox02 key.
The BitBox02 registers a multisig configuration before allowing
transaction signing. This multisig configuration includes the threshold,
cosigner xpubs, keypath, a variable to indicate for mainnet and testnet,
and a name that the user can choose during configuration registration.
The user is asked to register the multisig configuration either during
address verification or during transaction signing.
The check the xpub of the BitBox02 for other hardware wallets, a button
is added in the wallet info dialog.
The wallet encryption key is fetched in a separate api call, requiring a
slightly tweaked override version of the wallet encryption password.
E | gui.qt.main_window.[test_ms_p2wsh_2of3_cc3132_trezort_cc3133] | on_error
Traceback (most recent call last):
File "...\electrum\electrum\gui\qt\util.py", line 794, in run
result = task.task()
File "...\electrum\electrum\plugins\hw_wallet\qt.py", line 232, in trigger_pairings
devices = devmgr.scan_devices()
File "...\electrum\electrum\plugin.py", line 376, in func_wrapper
return func(self, *args, **kwargs)
File "...\electrum\electrum\plugin.py", line 656, in scan_devices
for f in self.enumerate_func:
RuntimeError: Set changed size during iteration
scenario1:
- 2of2 multisig wallet with trezor1 and trezor2 keystores
- only trezor2 connected
- previously we would pair first keystore with connected device and then display error.
now we will pair the device with the correct keystore on the first try
scenario2:
- standard wallet with trezor1 keystore
- trezor2 connected (different device)
- previously we would pair trezor2 with the keystore and then display error.
now we will prompt the user to select which device to pair with (out of one)
related: #5789
Just makes sense in general.
Also, previously, the GUI would freeze if right after startup the user
clicked the hww status bar icon (especially with multiple hww connected).
previously, client.handler was sometimes
- an InstallWizard
- a QtHandlerBase where win was an ElectrumWindow
- a QtHandlerBase where win was an InstallWizard
- a CmdLineHandler
That's just too much dynamic untyped undocumented polymorphism...
Now it will never be an InstallWizard (replaced with QtHandlerBase where win is an InstallWizard),
and now in all cases client.handler is an instance of HardwareHandlerBase, yay.
related: #6063
