lightningd: handle large numbers of command outputs gracefully.

Profiling shows us spending all our time in tal_arr_remove when dealing
with a giant number of output streams.  This applies both for RPC output
and plugin output.

Use linked list instead.

tests/test_coinmoves.py::test_generate_coinmoves (2,000,000, sqlite3):
	Time (from start to end of l2 node):	239 seconds **WAS 518**
	Worst latency:				56.9 seconds **WAS 353**

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This commit is contained in:
Rusty Russell
2025-11-20 12:07:12 +10:30
parent 6006467824
commit 8707b7312a
4 changed files with 27 additions and 37 deletions

View File

@@ -9,6 +9,7 @@
# include <external/jsmn/jsmn.h>
#include <bitcoin/short_channel_id.h>
#include <ccan/list/list.h>
#include <ccan/time/time.h>
#include <common/amount.h>
#include <common/jsonrpc_errors.h>
@@ -36,6 +37,8 @@ struct wireaddr;
struct wireaddr_internal;
struct json_stream {
struct list_node list;
struct json_out *jout;
/* Who is writing to this buffer now; NULL if nobody is. */