[InterNetNews/inn] e8bb46: history: add hissqlite, a SQLite-based method

[Kevin Bowling]

  Branch: refs/heads/main
  Home:   https://github.com/InterNetNews/inn
  Commit: e8bb468e513727225d76c813ed2b04c1c83df470
      https://github.com/InterNetNews/inn/commit/e8bb468e513727225d76c813ed2b04c1c83df470
  Author: Kevin Bowling <kevin.bowling at kev009.com>
  Date:   2026-07-13 (Mon, 13 Jul 2026)

  Changed paths:
    M .gitignore
    M MANIFEST
    M doc/pod/Makefile
    M doc/pod/expire.pod
    A doc/pod/hissqlite-convert.pod
    A doc/pod/hissqlite-util.pod
    A doc/pod/hissqlite.pod
    M doc/pod/history.pod
    M doc/pod/inn.conf.pod
    M doc/pod/makehistory.pod
    M expire/expire.c
    M history/Makefile
    A history/hissqlite/hismethod.config
    A history/hissqlite/hismethod.mk
    A history/hissqlite/hissqlite-convert.c
    A history/hissqlite/hissqlite-init.sql
    A history/hissqlite/hissqlite-main.sql
    A history/hissqlite/hissqlite-private.h
    A history/hissqlite/hissqlite-read.sql
    A history/hissqlite/hissqlite-util.in
    A history/hissqlite/hissqlite.c
    A history/hissqlite/hissqlite.h
    M include/inn/history.h
    M include/inn/innconf.h
    A include/inn/sqlite-helper.h
    M lib/Makefile
    M lib/innconf.c
    A lib/sqlite-helper-gen.in
    A lib/sqlite-helper.c
    M storage/Makefile
    M storage/ovsqlite/ovmethod.config
    M storage/ovsqlite/ovmethod.mk
    M storage/ovsqlite/ovsqlite.c
    R storage/ovsqlite/sqlite-helper-gen.in
    R storage/ovsqlite/sqlite-helper.c
    R storage/ovsqlite/sqlite-helper.h
    M support/mkmanifest
    M tests/Makefile
    M tests/TESTS
    A tests/lib/hissqlite-convert-t.c
    A tests/lib/hissqlite-t.c

  Log Message:
  -----------
  history: add hissqlite, a SQLite-based method

Add hissqlite, a SQLite history method selectable instead of hisv6
(hismethod = hissqlite) when INN is built with SQLite.  It keeps the
whole history in a single transactional database and, unlike ovsqlite,
needs no server daemon: innd is the primary writer and updates the
database directly, while nnrpd readers and the offline tools open
the WAL database read-only, and expiration runs in place rather than
rebuilding and swapping files.

- The backend uses a single WITHOUT ROWID table clustered on the
  16-byte MD5 of the Message-ID (a lookup is one clustered-leaf
  access); a row is a real article, a remembered Message-ID, or absent.
  Writes autocommit under WAL with synchronous=NORMAL, giving dbz's
  durability contract (recent writes may be lost on power loss, never
  corrupted, peers resend) without batching, which would only starve
  the second writer.  The WAL is checkpointed in the background and
  truncated on a clean shutdown.

- Expiration is two-horizon and in place (real articles past retention
  become remembered, remembered entries past /remember/ are deleted),
  streamed in hash-keyset pages.  A new HISCTLG_INPLACEEXPIRE
  capability lets expire(8) reopen the backend read/write with no
  ICCpause; -d/-f (side-file rebuild) and a bare -x are refused, with
  -t as the in-place dry run; hisv6's path is unchanged.

- The SQL prepared-statement codegen (sqlite-helper) moves from
  storage/ovsqlite to lib/ so the ovsqlite and hissqlite backends share
  one copy; libinn now carries the libsqlite3 dependency (LTVERSION
  bump needed when it is time to release).

- Page size, writer and per-nnrpd reader cache sizes, and mmap size are
  tunable through inn.conf; mmap defaults off, as in ovsqlite.

- hissqlite-convert migrates an existing history by walking the
  source backend and bulk-loading by hash, faithfully preserving the
  remembered entries and timestamps a from-spool makehistory rebuild
  cannot.

Comes with the hissqlite-util inspection tool, manual pages, and two
runtime tests wired into tests/TESTS: a full-vtable backend test and
an end-to-end hissqlite-convert test (seed hisv6 -> convert -> verify
tokens, timestamps and remembered entries survive).

see #349



To unsubscribe from these emails, change your notification settings at https://github.com/InterNetNews/inn/settings/notifications