To limit attack surface.
Context:
- both in daemon mode and in GUI mode, we start an RPC server
- the RPC server uses HTTP basic auth, with a random password that is saved in the config file
- read access to the config file implies access to the RPC server
- the traffic is unencrypted
- by default the server listens
- on Windows, on localhost TCP
- all other platform, via unix domain sockets
- if an attacker can listen to localhost TCP traffic, and there was traffic
- they could see the plaintext RPC password and issue their own commands
- e.g. if wireshark was already installed on the system, this might not require root access
- the "ping" and "gui" commands are used by everyday operations that affect most users:
- "ping" is used when trying to launch a second instance of electrum, to contact the first instance and enforce "singleton" behaviour
- "gui" is used for URI handling (`$ xdg-open bitcoin:asdasd`)
- many other sensitive commands, that operate on wallets, require *also* the wallet password
- but note that wallet.unlock can be used by the user to bypass this and store the wallet password in memory (exposed in GUI)
I propose locking down the RPC server when running in GUI mode:
- we still start it, as it is used for "ping" and "gui" RPCs, however we disable all other RPCs
- we could opt-in enable it, using a config var, except that ofc would not help against an attacker that has filesystem write access to the config file
- so I think it's even safer to just "hardcode" disable it: however the functionality is useful for development
- I propose we branch based on `constants.net.TESTNET`
- an alternative we could branch on that is hard to fake is `is_git_clone` in run_electrum
It is much easier to reason about the rpcserver if we don't allow changing its basic settings while it is already running. What does it mean to change the TCP port it is listening on ("rpcport") if it's already running? It is even problematic to change the rpcpassword: care needs to be taken to already update it for the current server.
(ref https://github.com/spesmilo/electrum/issues/6762)
This commit disallows changing all of the "rpc*" config variables if the daemon is already running.
---
Simultaneously, it also ensures rpc_password is always set and auth cannot be disabled.
Previously if there was a daemon running, and the user ran
`$ electrum setconfig rpcpassword ""` that would leave the RPC unauthenticated
for the current session. However next time the daemon restarted, get_rpc_credentials would see
the unset password and generate one.
I think this was the worst of both worlds:
- we did not really allow removing the rpc password, except for the current session, and
- perhaps unexpectedly, we would generate a new password on daemon restart
Instead now we explicitly make sure the RPC server can never get into a state where it does not have a password set.
Based on a report by `Zuzana Kotásková <36777@mail.vsfs.cz>`
Set SimpleConfig.CURRENT_WALLET = None if the wallet gets deleted,
otherwise we try to open it on the next startup which will show an error
message in QML.
When creating a new wallet in a Electrum instance with existing wallets
this change forces the user to reuse a password of any existing wallet
if `SimpleConfig.WALLET_USE_SINGLE_PASSWORD` is True.
This prevents the amount of different passwords from increasing and
guides the user towards a single wallet password (the intended default).
I think _wallet_key_from_path should not raise.
This is probably the sane way to deal with this.
Though all this is assuming that os.path.realpath can be treated as consistent/stateless.
closes https://github.com/spesmilo/electrum/issues/10182
- fix: qml gui errors when trying to open a wallets with only keystore-encryption
- fixes https://github.com/spesmilo/electrum/issues/10171
- qml gui to prompt for password on wallet open even if wallet is not storage-encrypted
Adds a new configvar `WALLET_PARTIAL_WRITES` to enable/disable partial writes for the walletDB.
This is a further restriction on top of the existing restrictions,
e.g. wallet files still need to have file encryption disabled for partial writes.
It defaults to off, so even for unencrypted wallets we disable partial writes for now.
This is used as a stopgap measure until we fix the issues found with the partial writes impl
(see https://github.com/spesmilo/electrum/issues/10000).
Notably verifymessage and decrypt(message) were silently ignoring trailing garbage
or inserted non-base64 characters present in signatures/ciphertext.
(both the CLI commands and in the GUI)
I think it is much cleaner and preferable to treat such signatures/ciphertext as invalid.
In fact I find it surprising that base64.b64decode(validate=False) is the default.
Perhaps we should create a helper function for it that set validate=True and use that.
- CURRENT_WALLET is set when a single wallet is loaded in memory, and it
remains set after Electrum stops running.
- If several wallets are loaded at the same time, CURRENT_WALLET is unset,
and RPCs must specify the wallet explicitly (using --wallet for the CLI)
- The fallback to 'default_wallet' essentially only applies when
creating a new wallet file
The password should not be an empty string.
Traceback (most recent call last):
File "/opt/electrum/electrum/gui/qt/__init__.py", line 377, in start_new_window
wallet = self._start_wizard_to_select_or_create_wallet(path)
File "/opt/electrum/electrum/gui/qt/__init__.py", line 446, in _start_wizard_to_select_or_create_wallet
wallet = self.daemon.load_wallet(wallet_file, d['password'], upgrade=True)
File "/opt/electrum/electrum/daemon.py", line 461, in func_wrapper
return func(self, *args, **kwargs)
File "/opt/electrum/electrum/daemon.py", line 474, in load_wallet
wallet.unlock(password)
File "/opt/electrum/electrum/wallet.py", line 3418, in unlock
self.check_password(password)
File "/opt/electrum/electrum/wallet.py", line 3069, in check_password
raise InvalidPassword("password given but wallet has no password")
electrum.util.InvalidPassword: password given but wallet has no password
Previously, generally, in case of any error, commands would raise a generic "Exception()" and the CLI/RPC would convert that and return it as `str(e)`.
With this change, we now distinguish "user-facing exceptions" (e.g. "Password required" or "wallet not loaded") and "internal errors" (e.g. bugs).
- for "user-facing exceptions", the behaviour is unchanged
- for "internal errors", we now pass around the traceback (e.g. from daemon server to rpc client) and show it to the user (previously, assuming there was a daemon running, the user could only retrieve the exception from the log of that daemon). These errors use a new jsonrpc error code int (code 2).
As the logic only changes for "internal errors", I deem this change not to be compatibility-breaking.
----------
Examples follow.
Consider the following two commands:
```
@command('')
async def errorgood(self):
from electrum.util import UserFacingException
raise UserFacingException("heyheyhey")
@command('')
async def errorbad(self):
raise Exception("heyheyhey")
```
----------
(before change)
CLI with daemon:
```
$ ./run_electrum --testnet daemon -d
starting daemon (PID 9221)
$ ./run_electrum --testnet errorgood
heyheyhey
$ ./run_electrum --testnet errorbad
heyheyhey
$ ./run_electrum --testnet stop
Daemon stopped
```
CLI without daemon:
```
$ ./run_electrum --testnet -o errorgood
heyheyhey
$ ./run_electrum --testnet -o errorbad
heyheyhey
```
RPC:
```
$ curl --data-binary '{"id":"curltext","jsonrpc":"2.0","method":"errorgood","params":[]}' http://user:pass@127.0.0.1:7777
{"id": "curltext", "jsonrpc": "2.0", "error": {"code": 1, "message": "heyheyhey"}}
$ curl --data-binary '{"id":"curltext","jsonrpc":"2.0","method":"errorbad","params":[]}' http://user:pass@127.0.0.1:7777
{"id": "curltext", "jsonrpc": "2.0", "error": {"code": 1, "message": "heyheyhey"}}
```
----------
(after change)
CLI with daemon:
```
$ ./run_electrum --testnet daemon -d
starting daemon (PID 9254)
$ ./run_electrum --testnet errorgood
heyheyhey
$ ./run_electrum --testnet errorbad
(inside daemon): Traceback (most recent call last):
File "/home/user/wspace/electrum/electrum/daemon.py", line 254, in handle
response['result'] = await f(*params)
File "/home/user/wspace/electrum/electrum/daemon.py", line 361, in run_cmdline
result = await func(*args, **kwargs)
File "/home/user/wspace/electrum/electrum/commands.py", line 163, in func_wrapper
return await func(*args, **kwargs)
File "/home/user/wspace/electrum/electrum/commands.py", line 217, in errorbad
raise Exception("heyheyhey")
Exception: heyheyhey
internal error while executing RPC
$ ./run_electrum --testnet stop
Daemon stopped
```
CLI without daemon:
```
$ ./run_electrum --testnet -o errorgood
heyheyhey
$ ./run_electrum --testnet -o errorbad
0.78 | E | __main__ | error running command (without daemon)
Traceback (most recent call last):
File "/home/user/wspace/electrum/./run_electrum", line 534, in handle_cmd
result = fut.result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
return self.__get_result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/home/user/wspace/electrum/./run_electrum", line 255, in run_offline_command
result = await func(*args, **kwargs)
File "/home/user/wspace/electrum/electrum/commands.py", line 163, in func_wrapper
return await func(*args, **kwargs)
File "/home/user/wspace/electrum/electrum/commands.py", line 217, in errorbad
raise Exception("heyheyhey")
Exception: heyheyhey
```
RPC:
```
$ curl --data-binary '{"id":"curltext","jsonrpc":"2.0","method":"errorgood","params":[]}' http://user:pass@127.0.0.1:7777
{"id": "curltext", "jsonrpc": "2.0", "error": {"code": 1, "message": "heyheyhey"}}
$ curl --data-binary '{"id":"curltext","jsonrpc":"2.0","method":"errorbad","params":[]}' http://user:pass@127.0.0.1:7777
{"id": "curltext", "jsonrpc": "2.0", "error": {"code": 2, "message": "internal error while executing RPC", "data": {"exception": "Exception('heyheyhey')", "traceback": "Traceback (most recent call last):\n File \"/home/user/wspace/electrum/electrum/daemon.py\", line 254, in handle\n response['result'] = await f(*params)\n File \"/home/user/wspace/electrum/electrum/commands.py\", line 163, in func_wrapper\n return await func(*args, **kwargs)\n File \"/home/user/wspace/electrum/electrum/commands.py\", line 217, in errorbad\n raise Exception(\"heyheyhey\")\nException: heyheyhey\n"}}}
```
The following exceptions should be expected:
FileNotFoundError: given wallet path does not exist
StorageReadWriteError: given file is not readable/writable or containing folder is not writable
InvalidPassword: wallet requires a password but no password or an invalid password was given
WalletFileException: any internal wallet data issue. specific subclasses can be caught separately:
- WalletRequiresSplit: wallet needs splitting (split_data passed in Exception)
- WalletRequiresUpgrade: wallet needs upgrade, and no upgrade=True was passed to load_wallet
- WalletUnfinished: wallet file contains an action and needs additional information to finalize. (WalletDB passed in exception)
Removed qml/qewalletdb.py
This patch also fixes load_wallet calls in electrum/scripts and adds a qml workaround for dialogs opening and closing so
fast that the dialog opened==true property change is missed (which we need to manage the dialog/page stack)
Plugins.stop was never called, so the Plugins thread only stopped
because of the is_running() check in run(), which triggers too late:
the Plugins thread was stopping after the main thread stopped.
E.g. playing around in the qt wizard with wallet creation for a Trezor,
and closing the wizard (only window):
``` 24.85 | E | p/plugin.Plugins |
Traceback (most recent call last):
File "/home/user/wspace/electrum/electrum/util.py", line 386, in run_jobs
job.run()
File "/home/user/wspace/electrum/electrum/plugin.py", line 430, in run
client.timeout(cutoff)
File "/home/user/wspace/electrum/electrum/plugin.py", line 363, in wrapper
return run_in_hwd_thread(partial(func, *args, **kwargs))
File "/home/user/wspace/electrum/electrum/plugin.py", line 355, in run_in_hwd_thread
fut = _hwd_comms_executor.submit(func)
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 167, in submit
raise RuntimeError('cannot schedule new futures after shutdown')
RuntimeError: cannot schedule new futures after shutdown
```
Without this,
`$ python3 -m unittest electrum.tests.regtest.TestUnixSockets.test_unixsockets`
was failing on my machine but succeeding on CI, due to timing differences.
- standardize_path is made more lenient: it no longer calls os.path.realpath,
as that was causing issues on Windows with some mounted drives
- daemon._wallets is still keyed on the old strict standardize_path, but
filesystem operations in WalletStorage use the new lenient standardize_path.
- daemon._wallets is strict to forbid opening the same logical file twice concurrently
- fs operations however work better on the non-resolved paths, so they use those
closes https://github.com/spesmilo/electrum/issues/8495
A new config API is introduced, and ~all of the codebase is adapted to it.
The old API is kept but mainly only for dynamic usage where its extra flexibility is needed.
Using examples, the old config API looked this:
```
>>> config.get("request_expiry", 86400)
604800
>>> config.set_key("request_expiry", 86400)
>>>
```
The new config API instead:
```
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS
604800
>>> config.WALLET_PAYREQ_EXPIRY_SECONDS = 86400
>>>
```
The old API operated on arbitrary string keys, the new one uses
a static ~enum-like list of variables.
With the new API:
- there is a single centralised list of config variables, as opposed to
these being scattered all over
- no more duplication of default values (in the getters)
- there is now some (minimal for now) type-validation/conversion for
the config values
closes https://github.com/spesmilo/electrum/pull/5640
closes https://github.com/spesmilo/electrum/pull/5649
Note: there is yet a third API added here, for certain niche/abstract use-cases,
where we need a reference to the config variable itself.
It should only be used when needed:
```
>>> var = config.cv.WALLET_PAYREQ_EXPIRY_SECONDS
>>> var
<ConfigVarWithConfig key='request_expiry'>
>>> var.get()
604800
>>> var.set(3600)
>>> var.get_default_value()
86400
>>> var.is_set()
True
>>> var.is_modifiable()
True
```
The qt, qml, and kivy GUIs have a first-start network-setup screen
that allows the user customising the network settings before creating a wallet.
Previously the daemon used to create the network and start it, before this screen,
before the GUI even starts. If the user changed network settings, those would
be set on the already running network, potentially including restarting the network.
Now it becomes the responsibility of the GUI to start the network, allowing this
first-start customisation to take place before starting the network at all.
The qt and the qml GUIs are adapted to make use of this. Kivy, and the other
prototype GUIs are not adapted and just start the network right away, as before.
ghost43