From noreply at github.com Mon Jul 13 20:15:58 2026 From: noreply at github.com (Kevin Bowling) Date: Mon, 13 Jul 2026 13:15:58 -0700 Subject: [InterNetNews/inn] e8bb46: history: add hissqlite, a SQLite-based method Message-ID: Branch: refs/heads/main Home: https://github.com/InterNetNews/inn Commit: e8bb468e513727225d76c813ed2b04c1c83df470 https://github.com/InterNetNews/inn/commit/e8bb468e513727225d76c813ed2b04c1c83df470 Author: Kevin Bowling Date: 2026-07-13 (Mon, 13 Jul 2026) Changed paths: M .gitignore M MANIFEST M doc/pod/Makefile M doc/pod/expire.pod A doc/pod/hissqlite-convert.pod A doc/pod/hissqlite-util.pod A doc/pod/hissqlite.pod M doc/pod/history.pod M doc/pod/inn.conf.pod M doc/pod/makehistory.pod M expire/expire.c M history/Makefile A history/hissqlite/hismethod.config A history/hissqlite/hismethod.mk A history/hissqlite/hissqlite-convert.c A history/hissqlite/hissqlite-init.sql A history/hissqlite/hissqlite-main.sql A history/hissqlite/hissqlite-private.h A history/hissqlite/hissqlite-read.sql A history/hissqlite/hissqlite-util.in A history/hissqlite/hissqlite.c A history/hissqlite/hissqlite.h M include/inn/history.h M include/inn/innconf.h A include/inn/sqlite-helper.h M lib/Makefile M lib/innconf.c A lib/sqlite-helper-gen.in A lib/sqlite-helper.c M storage/Makefile M storage/ovsqlite/ovmethod.config M storage/ovsqlite/ovmethod.mk M storage/ovsqlite/ovsqlite.c R storage/ovsqlite/sqlite-helper-gen.in R storage/ovsqlite/sqlite-helper.c R storage/ovsqlite/sqlite-helper.h M support/mkmanifest M tests/Makefile M tests/TESTS A tests/lib/hissqlite-convert-t.c A tests/lib/hissqlite-t.c Log Message: ----------- history: add hissqlite, a SQLite-based method Add hissqlite, a SQLite history method selectable instead of hisv6 (hismethod = hissqlite) when INN is built with SQLite. It keeps the whole history in a single transactional database and, unlike ovsqlite, needs no server daemon: innd is the primary writer and updates the database directly, while nnrpd readers and the offline tools open the WAL database read-only, and expiration runs in place rather than rebuilding and swapping files. - The backend uses a single WITHOUT ROWID table clustered on the 16-byte MD5 of the Message-ID (a lookup is one clustered-leaf access); a row is a real article, a remembered Message-ID, or absent. Writes autocommit under WAL with synchronous=NORMAL, giving dbz's durability contract (recent writes may be lost on power loss, never corrupted, peers resend) without batching, which would only starve the second writer. The WAL is checkpointed in the background and truncated on a clean shutdown. - Expiration is two-horizon and in place (real articles past retention become remembered, remembered entries past /remember/ are deleted), streamed in hash-keyset pages. A new HISCTLG_INPLACEEXPIRE capability lets expire(8) reopen the backend read/write with no ICCpause; -d/-f (side-file rebuild) and a bare -x are refused, with -t as the in-place dry run; hisv6's path is unchanged. - The SQL prepared-statement codegen (sqlite-helper) moves from storage/ovsqlite to lib/ so the ovsqlite and hissqlite backends share one copy; libinn now carries the libsqlite3 dependency (LTVERSION bump needed when it is time to release). - Page size, writer and per-nnrpd reader cache sizes, and mmap size are tunable through inn.conf; mmap defaults off, as in ovsqlite. - hissqlite-convert migrates an existing history by walking the source backend and bulk-loading by hash, faithfully preserving the remembered entries and timestamps a from-spool makehistory rebuild cannot. Comes with the hissqlite-util inspection tool, manual pages, and two runtime tests wired into tests/TESTS: a full-vtable backend test and an end-to-end hissqlite-convert test (seed hisv6 -> convert -> verify tokens, timestamps and remembered entries survive). see #349 To unsubscribe from these emails, change your notification settings at https://github.com/InterNetNews/inn/settings/notifications From noreply at github.com Mon Jul 13 20:16:10 2026 From: noreply at github.com (Kevin Bowling) Date: Mon, 13 Jul 2026 13:16:10 -0700 Subject: [InterNetNews/inn] 96a604: tests: history-bench, a hisv6/hissqlite benchmark Message-ID: Branch: refs/heads/main Home: https://github.com/InterNetNews/inn Commit: 96a604974f36178c77d52f5e6dbcd451e147090e https://github.com/InterNetNews/inn/commit/96a604974f36178c77d52f5e6dbcd451e147090e Author: Kevin Bowling Date: 2026-07-13 (Mon, 13 Jul 2026) Changed paths: M .gitignore M MANIFEST M support/mkmanifest M tests/Makefile A tests/lib/history-bench.c Log Message: ----------- tests: history-bench, a hisv6/hissqlite benchmark A standalone benchmark (deliberately not part of the TAP suite) for comparing the hisv6 and hissqlite history backends at scale. It opens each backend exactly as innd does including the in-core index hint and reads tuning from inn.conf, so results reflect production settings; with no inn.conf it falls back to each backend's built-in defaults, which match the inn.conf defaults, so the comparison stays fair either way. Per method it times create+write (one autocommit per article, like innd's steady state), sequential check, full walk (the expireover bloom-rebuild and migration scan), random lookup, and missing lookup, reports the on-disk footprint, and validates correctness. It fails if any inserted entry is missing or any absent entry is found. Lookups use a fixed-seed PRNG reset per phase so every method replays the identical order. Output is both a human readable table and a CSV row. Defaults to 100M entries (sized for large local testing); build with "make -C tests benchmarks" and run tests/lib/history-bench (see -h). see #349 To unsubscribe from these emails, change your notification settings at https://github.com/InterNetNews/inn/settings/notifications From noreply at github.com Mon Jul 13 20:16:24 2026 From: noreply at github.com (=?UTF-8?B?SnVsaWVuIMOJTElF?=) Date: Mon, 13 Jul 2026 13:16:24 -0700 Subject: [InterNetNews/inn] 4e009c: Actually install new hissqlite manual pages Message-ID: Branch: refs/heads/main Home: https://github.com/InterNetNews/inn Commit: 4e009ca190df92998c5c8560644a501be1dfdb7e https://github.com/InterNetNews/inn/commit/4e009ca190df92998c5c8560644a501be1dfdb7e Author: Julien ?LIE Date: 2026-07-13 (Mon, 13 Jul 2026) Changed paths: M doc/man/Makefile Log Message: ----------- Actually install new hissqlite manual pages see #349 To unsubscribe from these emails, change your notification settings at https://github.com/InterNetNews/inn/settings/notifications From noreply at github.com Mon Jul 13 20:16:37 2026 From: noreply at github.com (=?UTF-8?B?SnVsaWVuIMOJTElF?=) Date: Mon, 13 Jul 2026 13:16:37 -0700 Subject: [InterNetNews/inn] 8f6840: Build new history-bench program in CI Message-ID: Branch: refs/heads/main Home: https://github.com/InterNetNews/inn Commit: 8f684087cee4294157539595dc27c49187e4703f https://github.com/InterNetNews/inn/commit/8f684087cee4294157539595dc27c49187e4703f Author: Julien ?LIE Date: 2026-07-13 (Mon, 13 Jul 2026) Changed paths: M ci/test M support/mksnapshot Log Message: ----------- Build new history-bench program in CI see #349 To unsubscribe from these emails, change your notification settings at https://github.com/InterNetNews/inn/settings/notifications From noreply at github.com Mon Jul 13 20:16:50 2026 From: noreply at github.com (=?UTF-8?B?SnVsaWVuIMOJTElF?=) Date: Mon, 13 Jul 2026 13:16:50 -0700 Subject: [InterNetNews/inn] b93ed4: Fix the changelog of INN 2.7.4 Message-ID: Branch: refs/heads/main Home: https://github.com/InterNetNews/inn Commit: b93ed4acf31f62fe876ab6a6309bbf94dc0a9da9 https://github.com/InterNetNews/inn/commit/b93ed4acf31f62fe876ab6a6309bbf94dc0a9da9 Author: Julien ?LIE Date: 2026-07-13 (Mon, 13 Jul 2026) Changed paths: M doc/pod/hacking.pod M doc/pod/news.pod Log Message: ----------- Fix the changelog of INN 2.7.4 There was no SONAME bump for libinn. Mention an extra check in HACKING. To unsubscribe from these emails, change your notification settings at https://github.com/InterNetNews/inn/settings/notifications From noreply at github.com Mon Jul 13 20:17:03 2026 From: noreply at github.com (=?UTF-8?B?SnVsaWVuIMOJTElF?=) Date: Mon, 13 Jul 2026 13:17:03 -0700 Subject: [InterNetNews/inn] a04ea9: hissqlite-convert: Improve logs and documentation Message-ID: Branch: refs/heads/main Home: https://github.com/InterNetNews/inn Commit: a04ea95b747c7cf6e263a7f32cfc2ef8db1f5c90 https://github.com/InterNetNews/inn/commit/a04ea95b747c7cf6e263a7f32cfc2ef8db1f5c90 Author: Julien ?LIE Date: 2026-07-13 (Mon, 13 Jul 2026) Changed paths: M doc/pod/hissqlite-convert.pod M history/hissqlite/hissqlite-convert.c Log Message: ----------- hissqlite-convert: Improve logs and documentation see #349 To unsubscribe from these emails, change your notification settings at https://github.com/InterNetNews/inn/settings/notifications