lnutil: cache validate_features() results
channel_db.load_data() is slow, slowing down startup time (when trampoline is disabled). util.list_enabled_bits() is one of the main contributors to the slowness, called by validate_features(). One could argue that we could even simply *not* call validate_features for gossip messages as part of load_data, as they have already been validated before storing them in the db. However re-validating them there is a good clean-up/sanity check IMO. Note that what is considered "valid" can change over time, so just because validate_features passed when we originally received and stored a gossip message, it might no longer be valid a year later if the bolts change. This caching decreases the time needed for load_data on two different machines / gossip dbs as below: 47 sec -> 10 sec 18 sec -> 6 sec If instead of caching, I just rm the validate_features() calls, the benchmarks are almost identical, within noise. That is, the cache looks really effective. (the rest of the slowness is mostly due to lnmsg.decode_msg) ``` >>> lnutil.validate_features.cache_info() CacheInfo(hits=172674, misses=287, maxsize=1000, currsize=277) ``` ----- We could alternatively directly cache util.list_enabled_bits (instead of validate_features). That would be a bit slower and might end up using a lot more memory in some cases I think, but maybe conceptually would be cleaner. Also note that if validate_features() raises an exception, that is not cached.
This commit is contained in:
@@ -7,6 +7,7 @@ from collections import defaultdict
|
||||
from typing import NamedTuple, List, Tuple, Mapping, Optional, TYPE_CHECKING, Union, Dict, Set, Sequence
|
||||
import sys
|
||||
import time
|
||||
from functools import lru_cache
|
||||
|
||||
import electrum_ecc as ecc
|
||||
from electrum_ecc import CURVE_ORDER, ecdsa_sig64_from_der_sig
|
||||
@@ -1801,6 +1802,7 @@ if hasattr(sys, "get_int_max_str_digits"):
|
||||
assert sys.get_int_max_str_digits() >= 4300, f"sys.get_int_max_str_digits() too low: {sys.get_int_max_str_digits()}"
|
||||
|
||||
|
||||
@lru_cache(maxsize=1000) # massive speedup for the hot path of channel_db.load_data()
|
||||
def validate_features(features: int) -> LnFeatures:
|
||||
"""Raises IncompatibleOrInsaneFeatures if
|
||||
- a mandatory feature is listed that we don't recognize, or
|
||||
|
||||
Reference in New Issue
Block a user