davide/palladum-lightning
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity
Files
5e2f6c50286b3dfb83f181732bced27ed8c84882
palladum-lightning/common
History
Rusty Russell 5e2f6c5028 gossmap: don't stop reading if we hit a redundant channel_announce. 
While this shouldn't happen, it does (pending other fixes), and we stop reading the
gossip store until next time.  The result is partial gossip, demonstrated beautifully
by NicolasDorier's report:

```
lightning_gossipd: gossmap: redundant channel_announce for 864063x1306x1, offsets 1272259 and 1784859!"
```

Gossipd stalld there and don't make more progress.  So gossipd itself
doesn't see the entire gossip_store.

Then things get really batshit:

```
2025-02-04T05:53:28.582Z DEBUG   gossipd: Store compact time: 1429910 msec
```

This took 1429 seconds to process.  Why?

Because it hasn't been processing the gossip store fully, gossipd kept adding "new" records to the end:

```
2025-02-04T05:53:28.583Z DEBUG   gossipd: gossip_store: Read 62716143/1739952/5158256/0 cannounce/cupdate/nannounce/delete from store in 31634458462 bytes, now 31634458440 bytes (populated=true)
```

It has 31GB of gossip in there!  No wonder it took so long...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/8035
Changelog-Fixed: gossipd: corruption in the gossip_store could cause ever-longer startup times and no gossip updates.
2025-02-11 15:11:47 -06:00
..
test
gossmap: add (and use) logging hook.
2025-02-11 15:11:47 -06:00
addr.c
splice: Add plugin for magic “splice all” command
2024-11-12 06:42:52 +10:30
addr.h
splice: Add plugin for magic “splice all” command
2024-11-12 06:42:52 +10:30
amount.c
add ratio ceil and floor operators on amount_msat
2024-11-21 16:17:52 +10:30
amount.h
add ratio ceil and floor operators on amount_msat
2024-11-21 16:17:52 +10:30
autodata.c
…
autodata.h
…
base32.c
…
base32.h
…
base64.c
…
base64.h
…
bech32_util.c
common: Fix off-by-one in from_bech32_charset
2024-05-18 10:02:43 +02:00
bech32_util.h
…
bech32.c
common: rename bech32_encode parameter
2023-05-27 15:06:31 +09:30
bech32.h
common: rename bech32_encode parameter
2023-05-27 15:06:31 +09:30
bigsize.c
…
bigsize.h
…
billboard.c
…
billboard.h
…
bip32.c
…
bip32.h
…
blindedpath.c
BOLTs: more catchup ("BOLT 4: More clarifying changes.")
2024-10-16 07:14:32 +10:30
blindedpath.h
BOLTs: update which renames blinding terminology.
2024-10-16 07:14:32 +10:30
blindedpay.c
BOLTs: update which renames blinding terminology.
2024-10-16 07:14:32 +10:30
blindedpay.h
common: update to latest route-blinding spec.
2023-01-30 13:24:29 +10:30
blinding.c
BOLTs: update which renames blinding terminology.
2024-10-16 07:14:32 +10:30
blinding.h
BOLTs: update which renames blinding terminology.
2024-10-16 07:14:32 +10:30
blockheight_states.c
common: remove type_to_string files altogther.
2024-03-20 13:51:48 +10:30
blockheight_states.h
global: expose all fmt_X functions for direct use, make uniform.
2024-03-20 13:51:48 +10:30
bolt11_json.c
bitcoin/short_channel_id: pass by copy everywhere.
2024-03-20 13:51:48 +10:30
bolt11_json.h
…
bolt11.c
bitcoin/short_channel_id: pass by copy everywhere.
2024-03-20 13:51:48 +10:30
bolt11.h
common/bolt11: remove #if DEVELOPER.
2023-09-21 20:08:24 +09:30
bolt12_id.c
common/bolt12_id: generate alias tweak.
2024-08-01 12:31:48 +09:30
bolt12_id.h
common/bolt12_id: generate alias tweak.
2024-08-01 12:31:48 +09:30
bolt12_merkle.c
lightningd: use tweak on node_id of different key, for createinvoicerequest.
2024-08-01 12:31:48 +09:30
bolt12_merkle.h
lightningd: use tweak on node_id of different key, for createinvoicerequest.
2024-08-01 12:31:48 +09:30
bolt12.c
decode: don't fail to decode just because a bolt12 invoice has expired.
2024-11-30 13:17:55 +01:00
bolt12.h
decode: don't fail to decode just because a bolt12 invoice has expired.
2024-11-30 13:17:55 +01:00
channel_config.c
…
channel_config.h
BOLTs: catch up with minor clarifications.
2024-05-09 16:14:23 -05:00
channel_id.c
common/utils: macros to help get copy/compare across different types right.
2024-07-23 09:54:47 +09:30
channel_id.h
splice: Add plugin for magic “splice all” command
2024-11-12 06:42:52 +10:30
channel_type.c
Doc: update to BOLT which assumes option_var_onion_optin.
2024-06-19 15:54:24 +09:30
channel_type.h
channeld: BOLT update, no longer allow creation of old (experimental-only!) non-zero-fee anchor channels.
2024-06-19 15:54:24 +09:30
close_tx.c
script: consistently take the script length in identification functions
2024-03-18 12:24:49 +10:30
close_tx.h
…
closing_fee.h
…
codex32.c
codex32: use "cl" instead of "ms" as our HRP.
2023-08-02 16:12:32 +09:30
codex32.h
codex32: use "cl" instead of "ms" as our HRP.
2023-08-02 16:12:32 +09:30
coin_mvt.c
coin-moves: when a splice confirms, send a channel_closed event
2024-08-08 12:30:53 -07:00
coin_mvt.h
coin-moves: when a splice confirms, send a channel_closed event
2024-08-08 12:30:53 -07:00
configdir.c
common: Chainparams for testnet4 support
2025-01-13 15:45:19 -08:00
configdir.h
lightningd: add --developer runtime option.
2023-09-21 20:08:24 +09:30
configvar.c
lightningd: configvar style fixes
2024-08-10 15:20:59 +09:30
configvar.h
config: add the ability for plugins to specify that config values should be concealed.
2024-11-13 08:49:55 +10:30
crypto_state.h
…
cryptomsg.c
doc: update to BOLT 7f53a3e46ed59f19463447d5fa63b3e484c708a5 "BOLT 8: make it clear that there are *two* chaining keys."
2023-09-20 13:56:46 +09:30
cryptomsg.h
common: make hdr parameter const
2023-06-23 13:12:42 +09:30
daemon_conn.c
…
daemon_conn.h
…
daemon.c
common: don't crash on send_backtrace() if bt_print is NULL.
2025-01-28 10:53:22 +10:30
daemon.h
pyln-testing: use DEBUG_LIGHTNINGD to start gdb on lightningd during test.
2024-03-20 11:18:36 +10:30
decode_array.c
gossipd: drop zlib support.
2024-07-10 12:21:19 +09:30
decode_array.h
…
deprecation.c
Makefile: add CLN_NEXT_VERSION, functions encoding deprecation schedule.
2024-01-26 10:30:22 +10:30
deprecation.h
Makefile: add CLN_NEXT_VERSION, functions encoding deprecation schedule.
2024-01-26 10:30:22 +10:30
derive_basepoints.c
common/utils: macros to help get copy/compare across different types right.
2024-07-23 09:54:47 +09:30
derive_basepoints.h
common: remove #if DEVELOPER.
2023-09-21 20:08:24 +09:30
descriptor_checksum.c
…
descriptor_checksum.h
…
dev_disconnect.c
dev_disconnect: add '<' to close after an incoming msg.
2025-01-27 11:07:04 +10:30
dev_disconnect.h
dev_disconnect: add '<' to close after an incoming msg.
2025-01-27 11:07:04 +10:30
dijkstra.c
common, pay: actually test Dijkstra and route finding.
2024-03-07 14:09:14 +01:00
dijkstra.h
common: fix dijkstra scoring.
2024-03-07 14:09:14 +01:00
ecdh_hsmd.c
…
ecdh_hsmd.h
…
ecdh.h
…
errcode.h
…
features.c
Gossip: BOLT catch, remove initial_routing_sync.
2024-06-19 15:54:24 +09:30
features.h
BOLT catchup: quiescence is included.
2024-10-16 07:14:32 +10:30
fee_states.c
channeld: Adjust the feerate security margin profile
2024-05-13 14:06:45 -05:00
fee_states.h
channeld: Adjust the feerate security margin profile
2024-05-13 14:06:45 -05:00
fp16.c
…
fp16.h
…
gossip_constants.h
pytest: fix flake in test_gossip_pruning
2024-11-19 17:51:18 +10:30
gossip_store.c
common/gossip_store: move subdaemon-only routines to connectd.
2023-01-30 15:15:41 -06:00
gossip_store.h
common: add gossmap_chan_is_dying() helper to check flags.
2024-02-12 11:43:33 +01:00
gossmap.c
gossmap: don't stop reading if we hit a redundant channel_announce.
2025-02-11 15:11:47 -06:00
gossmap.h
gossmap: add (and use) logging hook.
2025-02-11 15:11:47 -06:00
gossmods_listpeerchannels.c
gossmap: keep capacity for locally-generated channels as well.
2024-10-04 11:27:53 +09:30
gossmods_listpeerchannels.h
common/gossmods_listpeerchannels: include channel capacity in callback.
2024-10-04 11:27:53 +09:30
hash_u5.c
…
hash_u5.h
…
hmac.c
common/utils: macros to help get copy/compare across different types right.
2024-07-23 09:54:47 +09:30
hmac.h
…
hsm_capable.c
channeld: add hsm_capabilities and add hsm_is_capable to common
2024-01-29 12:02:37 +10:30
hsm_capable.h
channeld: add hsm_capabilities and add hsm_is_capable to common
2024-01-29 12:02:37 +10:30
hsm_encryption.c
common/hsm_encryption: use const char * for errors.
2024-11-13 08:49:55 +10:30
hsm_encryption.h
common/hsm_encryption: use const char * for errors.
2024-11-13 08:49:55 +10:30
hsm_version.h
hsmd: create an hsm variant to sign a bolt12 invoice using a tweak on our key.
2024-08-01 12:31:48 +09:30
htlc_state.c
…
htlc_state.h
…
htlc_trim.c
global: thread zero fee option everywhere.
2023-06-29 11:28:47 -04:00
htlc_trim.h
global: thread zero fee option everywhere.
2023-06-29 11:28:47 -04:00
htlc_tx.c
global: thread zero fee option everywhere.
2023-06-29 11:28:47 -04:00
htlc_tx.h
Doc: update to BOLT which assumes option_var_onion_optin.
2024-06-19 15:54:24 +09:30
htlc_wire.c
global: rename blinding to path_key everywhere.
2024-10-16 07:14:32 +10:30
htlc_wire.h
global: rename blinding to path_key everywhere.
2024-10-16 07:14:32 +10:30
htlc.h
…
initial_channel.c
channeld: BOLT update, no longer allow creation of old (experimental-only!) non-zero-fee anchor channels.
2024-06-19 15:54:24 +09:30
initial_channel.h
global: expose all fmt_X functions for direct use, make uniform.
2024-03-20 13:51:48 +10:30
initial_commit_tx.c
common: remove type_to_string files altogther.
2024-03-20 13:51:48 +10:30
initial_commit_tx.h
global: thread zero fee option everywhere.
2023-06-29 11:28:47 -04:00
interactivetx.c
Splice: Stricter interop errors
2025-02-05 12:50:24 -06:00
interactivetx.h
interactivetx: Add support for shared prevtx
2024-11-21 14:15:36 +10:30
iso4217.c
…
iso4217.h
bolt12: update comments to match latest spec.
2022-11-09 15:08:03 +01:00
json_blinded_path.c
BOLT update: catch up ("BOLT 4: rename onionmsg_hop to blinded_path_hop")
2024-10-16 07:14:32 +10:30
json_blinded_path.h
common: move json_to_blinded_path into its own file.
2024-05-12 19:11:43 -05:00
json_channel_type.c
common: move json_add_channel_type from lightningd/
2024-01-29 13:40:34 +10:30
json_channel_type.h
common: move json_add_channel_type from lightningd/
2024-01-29 13:40:34 +10:30
json_command.c
lightningd: check command should return as much detail as possible.
2024-06-25 08:47:50 +09:30
json_command.h
common: move now-enlarged command_fail_badparam into its own source file.
2024-06-25 08:47:50 +09:30
json_filter.c
common/json_filter: routine to turn "filter" JSON into a filter.
2022-11-09 20:25:58 +10:30
json_filter.h
common/json_filter: routine to turn "filter" JSON into a filter.
2022-11-09 20:25:58 +10:30
json_param.c
common: Chainparams for testnet4 support
2025-01-13 15:45:19 -08:00
json_param.h
splice: Add plugin for magic “splice all” command
2024-11-12 06:42:52 +10:30
json_parse_simple.c
signed types: add handlers for signed types
2023-07-13 19:34:47 +09:30
json_parse_simple.h
signed types: add handlers for signed types
2023-07-13 19:34:47 +09:30
json_parse.c
common: round out the short_channel_id_dir JSON routines.
2024-10-04 11:27:53 +09:30
json_parse.h
common: round out the short_channel_id_dir JSON routines.
2024-10-04 11:27:53 +09:30
json_stream.c
route: Change the type of the funding capacity to amount_sat
2024-10-07 14:05:47 +02:00
json_stream.h
route: Change the type of the funding capacity to amount_sat
2024-10-07 14:05:47 +02:00
jsonrpc_errors.h
splice: Clean up error messages for RPC
2024-11-21 14:15:36 +10:30
key_derive.c
common: No longer support new channels without option_static_remotekey.
2024-06-19 15:54:24 +09:30
key_derive.h
lightningd: move bip32_pubkey here from common/, add hsm check.
2023-03-22 13:50:32 +10:30
keyset.c
common: No longer support new channels without option_static_remotekey.
2024-06-19 15:54:24 +09:30
keyset.h
…
lease_rates.c
…
lease_rates.h
…
Makefile
splice: Script for complex splices
2024-11-12 06:42:52 +10:30
memleak.c
common/memleak: show tal_steal operations on memleak.
2023-10-03 10:05:55 +02:00
memleak.h
common/memleak: implement callback arg for dump_memleak.
2023-10-03 10:05:55 +02:00
msg_queue.c
connectd: increase queue length to 250,000.
2024-11-01 16:54:49 +10:30
msg_queue.h
…
node_id.c
common: remove type_to_string files altogther.
2024-03-20 13:51:48 +10:30
node_id.h
global: expose all fmt_X functions for direct use, make uniform.
2024-03-20 13:51:48 +10:30
onion_decode.c
common: provide readable explanation when onion payload is invalid.
2024-11-20 12:29:27 +01:00
onion_decode.h
common: provide readable explanation when onion payload is invalid.
2024-11-20 12:29:27 +01:00
onion_encode.c
BOLTs: update which renames blinding terminology.
2024-10-16 07:14:32 +10:30
onion_encode.h
global: rename blinding to path_key everywhere.
2024-10-16 07:14:32 +10:30
onion_message_parse.c
global: rename blinding to path_key everywhere.
2024-10-16 07:14:32 +10:30
onion_message_parse.h
global: rename blinding to path_key everywhere.
2024-10-16 07:14:32 +10:30
onion_message.c
BOLT update: catch up ("BOLT 4: rename onionmsg_hop to blinded_path_hop")
2024-10-16 07:14:32 +10:30
onion_message.h
global: rename blinding to path_key everywhere.
2024-10-16 07:14:32 +10:30
onionreply.c
…
onionreply.h
…
overflows.h
…
peer_billboard.c
…
peer_billboard.h
…
peer_failed.c
common: add peer_failed_warn_nodisconnect routine for non-disconnecting warnings
2023-10-23 15:48:50 +10:30
peer_failed.h
common: add peer_failed_warn_nodisconnect routine for non-disconnecting warnings
2023-10-23 15:48:50 +10:30
peer_io.c
…
peer_io.h
…
peer_status_wire.csv
common: add peer_failed_warn_nodisconnect routine for non-disconnecting warnings
2023-10-23 15:48:50 +10:30
penalty_base.c
…
penalty_base.h
…
per_peer_state.c
…
per_peer_state.h
…
permute_tx.c
Update libwally to 0.8.8, support PSBTv2
2023-03-23 16:10:55 +10:30
permute_tx.h
…
ping.c
common: remove #if DEVELOPER.
2023-09-21 20:08:24 +09:30
ping.h
…
plugin.c
core: notify plugins when a log line is emitted.
2024-05-17 13:03:12 -05:00
plugin.h
core: notify plugins when a log line is emitted.
2024-05-17 13:03:12 -05:00
psbt_internal.c
global: remove tags from BOLT quotes now dual-funding is in master
2024-05-09 16:14:23 -05:00
psbt_internal.h
channeld: Code to implement splicing
2023-07-31 21:00:22 +09:30
psbt_keypath.c
Support p2tr deposit addresses
2023-07-11 11:41:15 +09:30
psbt_keypath.h
Support p2tr deposit addresses
2023-07-11 11:41:15 +09:30
psbt_open.c
psbt: Add ability to set channel_ids in psbt globals
2024-11-12 06:42:52 +10:30
psbt_open.h
psbt: Add ability to set channel_ids in psbt globals
2024-11-12 06:42:52 +10:30
pseudorand.c
common: add tal_arr_randomize() to shuffle an array.
2023-08-12 09:24:06 +09:30
pseudorand.h
common: add tal_arr_randomize() to shuffle an array.
2023-08-12 09:24:06 +09:30
random_select.c
…
random_select.h
…
read_peer_msg.c
common: move is_peer_error/is_peer_warning from read_peer_msg.c to wire_error.c
2023-10-23 15:48:50 +10:30
read_peer_msg.h
common: move is_peer_error/is_peer_warning from read_peer_msg.c to wire_error.c
2023-10-23 15:48:50 +10:30
route.c
pay: Switch to msat for total_capacity
2024-10-07 14:05:47 +02:00
route.h
pay: Switch to msat for total_capacity
2024-10-07 14:05:47 +02:00
scb_wire.csv
staticbackup: don't use wireaddr_internal.
2023-06-01 09:28:39 +09:30
sciddir_or_pubkey.c
common: add sciddir_or_pubkey type.
2024-05-12 19:11:43 -05:00
sciddir_or_pubkey.h
common: add sciddir_or_pubkey type.
2024-05-12 19:11:43 -05:00
setup.c
wally: rename functions that clash with wally v1.0.0
2024-01-16 15:47:03 +01:00
setup.h
…
shutdown_scriptpubkey.c
script: consistently take the script length in identification functions
2024-03-18 12:24:49 +10:30
shutdown_scriptpubkey.h
lightningd: only allow closing to native segwit
2022-12-13 08:28:12 +10:30
sphinx.c
common: sphinx_path_new to take explicit len.
2024-11-18 11:03:26 +10:30
sphinx.h
common: sphinx_path_new to take explicit len.
2024-11-18 11:03:26 +10:30
splice_script.c
splice: alpine build warning causing failure
2024-12-16 09:36:17 +10:30
splice_script.h
splice: Script for complex splices
2024-11-12 06:42:52 +10:30
status_levels.c
logging: add TRACE between DEBUG and IO
2024-05-08 21:05:49 -05:00
status_levels.h
logging: add TRACE between DEBUG and IO
2024-05-08 21:05:49 -05:00
status_wire.c
…
status_wire.csv
…
status_wire.h
…
status.c
common: don't send trace messages by default, don't ratelimit at all.
2024-12-16 09:48:51 +10:30
status.h
common: don't send trace messages by default, don't ratelimit at all.
2024-12-16 09:48:51 +10:30
subdaemon.c
common: don't send trace messages by default, don't ratelimit at all.
2024-12-16 09:48:51 +10:30
subdaemon.h
common/daemon.c: remove #ifdef DEVELOPER in favor of runtime flag.
2023-09-21 20:08:24 +09:30
timeout.c
…
timeout.h
…
trace.c
common: Make trace debugging output configurate at compile time
2024-11-24 10:24:31 +10:30
trace.h
common: Make trace debugging output configurate at compile time
2024-11-24 10:24:31 +10:30
tx_roles.c
tx_roles: allow to be serialized btw processes
2023-02-07 21:03:36 -06:00
tx_roles.h
tx_roles: allow to be serialized btw processes
2023-02-07 21:03:36 -06:00
utils.c
common: add tal_arr_eq helper.
2024-02-16 15:02:38 +01:00
utils.h
common/utils: macros to help get copy/compare across different types right.
2024-07-23 09:54:47 +09:30
utxo.c
global: thread zero fee option everywhere.
2023-06-29 11:28:47 -04:00
utxo.h
global: thread zero fee option everywhere.
2023-06-29 11:28:47 -04:00
version.c
lightningd: use OPT_EXITS for options which exit.
2023-06-03 10:50:29 +09:30
version.h
lightningd: use OPT_EXITS for options which exit.
2023-06-03 10:50:29 +09:30
wallet.c
…
wallet.h
wallet: remove unused TX_ANNOTATION type in transaction_annotations table.
2023-01-30 15:15:41 -06:00
wire_error.c
channeld: Fix tx_abort encoding
2024-11-21 14:15:36 +10:30
wire_error.h
channeld: Fix tx_abort encoding
2024-11-21 14:15:36 +10:30
wireaddr.c
common: routine to make wireaddr_internal from wireaddr.
2024-11-25 15:39:13 +10:30
wireaddr.h
common: routine to make wireaddr_internal from wireaddr.
2024-11-25 15:39:13 +10:30