We always start a transaction before processing, but there are cases where
we don't need to. Switch to doing it on-demand.
This doesn't make a big difference for sqlite3, but it can for Postgres because
of the latency: 12% or so. Every bit helps!
30 runs, min-max(mean+/-stddev):
Postgres before: 8.842773-9.769030(9.19531+/-0.21)
Postgres after: 8.007967-8.321856(8.14172+/-0.066)
sqlite3 before: 7.486042-8.371831(8.15544+/-0.19)
sqlite3 after: 7.973411-8.576135(8.3025+/-0.12)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Basically, `devtools/reduce-includes.sh */*.c`.
Build time from make clean (RUST=0) (includes building external libs):
Before:
real 0m38.944000-40.416000(40.1131+/-0.4)s
user 3m6.790000-17.159000(15.0571+/-2.8)s
sys 0m35.304000-37.336000(36.8942+/-0.57)s
After:
real 0m37.872000-39.974000(39.5466+/-0.59)s
user 3m1.211000-14.968000(12.4556+/-3.9)s
sys 0m35.008000-36.830000(36.4143+/-0.5)s
Build time after touch config.vars (RUST=0):
Before:
real 0m19.831000-21.862000(21.5528+/-0.58)s
user 2m15.361000-30.731000(28.4798+/-4.4)s
sys 0m21.056000-22.339000(22.0346+/-0.35)s
After:
real 0m18.384000-21.307000(20.8605+/-0.92)s
user 2m5.585000-26.843000(23.6017+/-6.7)s
sys 0m19.650000-22.003000(21.4943+/-0.69)s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And gracefully fail for this case.
There's no such thing for Postgres, but that's because dbs need to be
set up by the admin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The character classification functions in <ctype.h> are designed to
classify characters returned by <stdio.h> getchar() and friends, which
return characters as signed integers in the range 0 to 255 or EOF. The
behavior of the ctype functions is undefined if they are passed a value
outside of that range, which may happen if they are passed a char-typed
value and the system's char type is signed.
<ccan/str/str.h> defines some inline utility functions that perform the
necessary cast to coerce a char-typed argument into the allowed value
range. Call these wrappers instead of the bare ctype functions when
classifying char-typed characters.
Changelog-None
Now you can grep for 'sqlite3 version' and see where we would like
to update.
Debian 11 (Bullseye) and Ubuntu 20.04 (Focal) ship with SQLite 3.31.1.
RHEL 9 ships with 3.34.1. Fedora 38+ uses SQLite 3.40+.
Unfortunately, RHEL8 ships with 3.26.0, and is still on maintenance Support
(security fixes, no new features): runs until May 31, 2029.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids the mess where we override db_fatal for teqsts, and keeps it
generic.
Also allows us to get rid of one #if DEVELOPER, and an ugly global for
bookkeeper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Because it used internal routines, it didn't pass operations through the
db hook! So make it use the generic routines, with the twist that they
are not translated.
And when we use this in a migration hook, we're actually in a
transaction.
This, in turn, introduces an issue: we need to be outside a transaction
to "PRAGMA foreign_keys = OFF", but completing the transaction when
there is a db hook actually enters the io loop, freeing the tmpctx!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
