We would reject bip21 URIs that contained both an "address=" and a "lightning=" key with a bolt11 invoice,
where the bolt11 invoice did not contain a fallback address.
fixes https://github.com/spesmilo/electrum/issues/7952
see in particular https://github.com/spesmilo/electrum/issues/7952#issuecomment-1227225602
> So the issue is with the aiorpcx monkey patch in util.py, as it
> relies on side-effecting the asyncio.Task, and it patches Task.cancel.
> However, aiohttp also uses Task.cancel for its own timeouting of the
> http request, with the same Task object, and this confuses timeout_after.
> Ultimately FxThread.run exits.
related https://github.com/kyuupichan/aiorpcX/pull/47
---
note: I am not content at all with this monkey-patching approach,
but at the same time I don't see how to properly fix this handling all
edge-cases in aiorpcx.
python 3.11 is finally adding an implementation of TaskGroup [0] and
an async timeout context manager [1] in the asyncio module of the stdlib.
Looking at the implementations, they look unfeasible to backport:
much of the implementation of asyncio.Task had to be changed for them
to work, and TaskGroup in particular relies on the new ExceptionGroups.
Some of these edge cases we are battling with aiorpcx.curio look
difficult to fix without those stdlib changes...
Anyway, when we bump the min python to 3.11, I look forward to switching
to that code instead, and ripping this stuff out.
[0]: https://docs.python.org/3.11/library/asyncio-task.html#task-groups
[1]: https://docs.python.org/3.11/library/asyncio-task.html#asyncio.timeout
If keystore.check_password is called with some pw on a keystore that does not have a password set,
it now raises better exceptions: it should now always raise InvalidPassword, and with a nicer msg.
Previously the exc type would depend on the ks type.
Examples before change:
```
>>> wallet.keystore.check_password("asd")
Traceback (most recent call last):
File "/home/user/wspace/electrum/electrum/keystore.py", line 580, in check_password
xprv = pw_decode(self.xprv, password, version=self.pw_hash_version)
File "/home/user/wspace/electrum/electrum/crypto.py", line 311, in pw_decode
plaintext_bytes = pw_decode_bytes(data, password, version=version)
File "/home/user/wspace/electrum/electrum/crypto.py", line 270, in pw_decode_bytes
data_bytes = bytes(base64.b64decode(data))
File "/usr/lib/python3.10/base64.py", line 87, in b64decode
return binascii.a2b_base64(s)
binascii.Error: Incorrect padding
```
```
>>> wallet.keystore.check_password("asd")
Traceback (most recent call last):
s = aes_decrypt_with_iv(secret, iv, e)
File "/home/user/wspace/electrum/electrum/crypto.py", line 157, in aes_decrypt_with_iv
data = decryptor.update(data) + decryptor.finalize()
File "/usr/lib/python3/dist-packages/cryptography/hazmat/primitives/ciphers/base.py", line 148, in finalize
data = self._ctx.finalize()
File "/usr/lib/python3/dist-packages/cryptography/hazmat/backends/openssl/ciphers.py", line 193, in finalize
raise ValueError(
ValueError: The length of the provided data is not a multiple of the block length.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/wspace/electrum/electrum/gui/qt/console.py", line 254, in exec_command
result = eval(command, self.namespace, self.namespace)
File "<string>", line 1, in <module>
File "/home/user/wspace/electrum/electrum/keystore.py", line 248, in check_password
self.get_private_key(pubkey, password)
File "/home/user/wspace/electrum/electrum/keystore.py", line 267, in get_private_key
sec = pw_decode(self.keypairs[pubkey], password, version=self.pw_hash_version)
File "/home/user/wspace/electrum/electrum/crypto.py", line 311, in pw_decode
plaintext_bytes = pw_decode_bytes(data, password, version=version)
File "/home/user/wspace/electrum/electrum/crypto.py", line 271, in pw_decode_bytes
return _pw_decode_raw(data_bytes, password, version=version)
File "/home/user/wspace/electrum/electrum/crypto.py", line 255, in _pw_decode_raw
raise InvalidPassword() from e
electrum.util.InvalidPassword: Incorrect password
```
-----
Examples after change:
```
>>> wallet.keystore.check_password("asd")
Traceback (most recent call last):
return binascii.a2b_base64(s)
binascii.Error: Incorrect padding
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "...\electrum\keystore.py", line 68, in wrapper
return check_password_fn(self, password)
File "...\electrum\keystore.py", line 605, in check_password
xprv = pw_decode(self.xprv, password, version=self.pw_hash_version)
File "...\electrum\crypto.py", line 311, in pw_decode
plaintext_bytes = pw_decode_bytes(data, password, version=version)
File "...\electrum\crypto.py", line 267, in pw_decode_bytes
raise CiphertextFormatError("ciphertext not valid base64") from e
electrum.crypto.CiphertextFormatError: ciphertext not valid base64
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "...\electrum\gui\qt\console.py", line 254, in exec_command
result = eval(command, self.namespace, self.namespace)
File "<string>", line 1, in <module>
File "...\electrum\keystore.py", line 76, in wrapper
raise InvalidPassword("password given but keystore has no password") from e
electrum.util.InvalidPassword: password given but keystore has no password
```
```
>>> wallet.keystore.check_password("asd")
Traceback (most recent call last):
s = aes_decrypt_with_iv(secret, iv, e)
File "...\electrum\crypto.py", line 158, in aes_decrypt_with_iv
data = cipher.decrypt(data)
File "...\Python310\site-packages\Cryptodome\Cipher\_mode_cbc.py", line 246, in decrypt
raise ValueError("Data must be padded to %d byte boundary in CBC mode" % self.block_size)
ValueError: Data must be padded to 16 byte boundary in CBC mode
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "...\electrum\keystore.py", line 68, in wrapper
return check_password_fn(self, password)
File "...\electrum\keystore.py", line 272, in check_password
self.get_private_key(pubkey, password)
File "...\electrum\keystore.py", line 291, in get_private_key
sec = pw_decode(self.keypairs[pubkey], password, version=self.pw_hash_version)
File "...\electrum\crypto.py", line 311, in pw_decode
plaintext_bytes = pw_decode_bytes(data, password, version=version)
File "...\electrum\crypto.py", line 268, in pw_decode_bytes
return _pw_decode_raw(data_bytes, password, version=version)
File "...\electrum\crypto.py", line 249, in _pw_decode_raw
raise InvalidPassword() from e
electrum.util.InvalidPassword: Incorrect password
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "...\electrum\gui\qt\console.py", line 254, in exec_command
result = eval(command, self.namespace, self.namespace)
File "<string>", line 1, in <module>
File "...\electrum\keystore.py", line 76, in wrapper
raise InvalidPassword("password given but keystore has no password") from e
electrum.util.InvalidPassword: password given but keystore has no password
```
follow-up fe74e4b9c9
Just for sanity -- there could be a collision of class names (defined in separate modules).
```
>>> wallet.is_up_to_date.__qualname__
'Abstract_Wallet.is_up_to_date'
>>> wallet.is_up_to_date.__module__
'electrum.wallet'
```
and use that registration in the iterator.
That way, we do not invoke getattr on arbitrary methods, which
could trigger unwanted code execution (for example, if a method
is decorated with @property, getattr will execute its code).
asyncio.get_event_loop() became deprecated in python3.10. (see https://github.com/python/cpython/issues/83710)
```
.../electrum/electrum/daemon.py:470: DeprecationWarning: There is no current event loop
self.asyncio_loop = asyncio.get_event_loop()
.../electrum/electrum/network.py:276: DeprecationWarning: There is no current event loop
self.asyncio_loop = asyncio.get_event_loop()
```
Also, according to that thread, "set_event_loop() [... is] not deprecated by oversight".
So, we stop using get_event_loop() and set_event_loop() in our own code.
Note that libraries we use (such as the stdlib for python <3.10), might call get_event_loop,
which then relies on us having called set_event_loop e.g. for the GUI thread. To work around
this, a custom event loop policy providing a get_event_loop implementation is used.
Previously, we have been using a single asyncio event loop, created with
util.create_and_start_event_loop, and code in many places got a reference to this loop
using asyncio.get_event_loop().
Now, we still use a single asyncio event loop, but it is now stored as a global in
util._asyncio_event_loop (access with util.get_asyncio_loop()).
I believe these changes also fix https://github.com/spesmilo/electrum/issues/5376
fixes https://github.com/spesmilo/electrum/issues/7447
Consider this trace for 4.2.0:
```
Traceback (most recent call last):
File "electrum/gui/qt/__init__.py", line 332, in start_new_window
File "electrum/gui/qt/__init__.py", line 363, in _start_wizard_to_select_or_create_wallet
File "electrum/gui/qt/installwizard.py", line 302, in select_storage
File "electrum/util.py", line 504, in get_new_wallet_name
PermissionError: [Errno 1] Operation not permitted: '/Users/admin/Documents/Peach/MS'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "electrum/gui/qt/__init__.py", line 426, in main
File "electrum/gui/qt/__init__.py", line 307, in wrapper
File "electrum/gui/qt/__init__.py", line 349, in start_new_window
File "electrum/util.py", line 504, in get_new_wallet_name
PermissionError: [Errno 1] Operation not permitted: '/Users/admin/Documents/Peach/MS'
```
Note that `get_new_wallet_name` (os.listdir) can raise OSError,
and we were calling that on the main entrypoint codepath without exception-handling.
We were also calling it in the fallback codepath without exception-handling.
i.e. the GUI errored out on every startup for affected users, and without CLI usage
it was not possible to recover.
aiorpcx 0.20 changed the behaviour/API of TaskGroups.
When used as a context manager, TaskGroups no longer propagate
exceptions raised by their tasks. Instead, the calling code has
to explicitly check the results of tasks and decide whether to re-raise
any exceptions.
This is a significant change, and so this commit introduces "OldTaskGroup",
which should behave as the TaskGroup class of old aiorpcx. All existing
usages of TaskGroup are replaced with OldTaskGroup.
closes https://github.com/spesmilo/electrum/issues/7446
I think this was originally needed due to incorrect management of group lifecycles,
which our current code is doing better.
also note that if we needed this, in newer aiorpcx, the name of
the field was ~changed from `_closed` to `joined`:
239002689a
Previously, msat precision was leaking through format_satoshis if the
user's base unit was set to "sat". This was a bug.
Some features of format_satoshis did not work well with such values, such
as the "whitespaces" param.
Old code:
>>> util.format_satoshis(Decimal('45831275.748'), decimal_point=2)
'458312.76'
>>> util.format_satoshis(Decimal('45831275.748'), decimal_point=0)
'45831275.748'
New code:
>>> util.format_satoshis(Decimal('45831275.748'), decimal_point=2)
'458312.76'
>>> util.format_satoshis(Decimal('45831275.748'), decimal_point=0)
'45831276.'
