[InterNetNews/inn] c40ea6: ovsqlite: add mmapsize configuration

Tue Jun 2 21:04:12 UTC 2026

  Branch: refs/heads/2.7
  Home:   https://github.com/InterNetNews/inn
  Commit: c40ea666213ba0667e471528c796f114a66c74f9
      https://github.com/InterNetNews/inn/commit/c40ea666213ba0667e471528c796f114a66c74f9
  Author: Kevin Bowling <kevin.bowling at kev009.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/ovsqlite.pod
    M samples/ovsqlite.conf
    M storage/ovsqlite/ovsqlite-server.c

  Log Message:
  -----------
  ovsqlite: add mmapsize configuration

close #337

  Commit: f65f46e34e0b63fa2e97468217dab332f2f34e65
      https://github.com/InterNetNews/inn/commit/f65f46e34e0b63fa2e97468217dab332f2f34e65
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/news.pod

  Log Message:
  -----------
  Add a NEWS entry for the previous commit

see #337

  Commit: 7aeb051f179e740c48aea2e74221b429bbca35be
      https://github.com/InterNetNews/inn/commit/7aeb051f179e740c48aea2e74221b429bbca35be
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M configure.ac

  Log Message:
  -----------
  Mention libcanlock in configure output for FreeBSD

Kevin Bowling recently created this package for FreeBSD, so let's
reference it!

  Commit: 903ccb37943a86837e62d854b995a57a0b619c31
      https://github.com/InterNetNews/inn/commit/903ccb37943a86837e62d854b995a57a0b619c31
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M tests/util/inndf.t

  Log Message:
  -----------
  Add DragonFly as a BSD OS in inndf test suite

  Commit: 6c2ae4b4cb31b3b32ac5741fe5c6efc3cf5cd46b
      https://github.com/InterNetNews/inn/commit/6c2ae4b4cb31b3b32ac5741fe5c6efc3cf5cd46b
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/inn.conf.pod
    M doc/pod/news.pod
    M lib/messageid.c
    M nnrpd/post.c
    M tests/lib/messageid-t.c

  Log Message:
  -----------
  Allow Message-IDs without any "@" with laxmid

In his attempt to inject old articles from BNews, Jesse Rehmer found it
useful to accept such articles in the laxmid syntax check.

  Commit: 3811762b83f478559a4b195793856136274b3dd8
      https://github.com/InterNetNews/inn/commit/3811762b83f478559a4b195793856136274b3dd8
  Author: Kevin Bowling <kevin.bowling at kev009.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/ovsqlite.pod
    M samples/ovsqlite.conf
    M storage/ovsqlite/ovsqlite-server.c
    M storage/ovsqlite/sql-main.sql

  Log Message:
  -----------
  ovsqlite: Add WAL mode configuration

close #336
see #338

  Commit: 875bb58c58866d6321dac9eee6d7f55a1d80e2a0
      https://github.com/InterNetNews/inn/commit/875bb58c58866d6321dac9eee6d7f55a1d80e2a0
  Author: Kevin Bowling <kevin.bowling at kev009.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M .gitignore
    M MANIFEST
    M Makefile.global.in
    M doc/pod/ovsqlite.pod
    M samples/ovsqlite.conf
    M storage/Makefile
    M storage/ovsqlite/ovmethod.config
    M storage/ovsqlite/ovmethod.mk
    M storage/ovsqlite/ovsqlite-server.c
    M storage/ovsqlite/ovsqlite.c
    A storage/ovsqlite/sql-read.sql
    M support/mkmanifest
    M tests/Makefile
    M tests/TESTS
    A tests/overview/ovsqlite-integ.t
    A tests/overview/ovsqlite-read-t.c
    A tests/overview/ovsqlite-t.c
    A tests/overview/ovsqlite-write-t.c

  Log Message:
  -----------
  ovsqlite: Add direct reader mode for nnrpd with WAL

When WAL mode is enabled, nnrpd processes now open the overview database
directly with SQLITE_OPEN_READONLY instead of routing reads through
ovsqlite-server.  This eliminates the IPC round-trip and the
single-threaded server bottleneck for read operations, allowing read
performance to scale with the number of concurrent readers.

Direct reader mode activates automatically when walmode is true in
ovsqlite.conf AND the database is actually in WAL mode (verified via
pragma journal_mode).  Without WAL, behavior is unchanged -- readers
connect to ovsqlite-server as before to avoid EXCLUSIVE lock contention.

Key changes:
- ovsqlite.c: direct_open() opens the database read-only with schema
  version validation, journal mode verification, decompression support,
  and goto-based cleanup.  Read functions (groupstats, getartinfo,
  search) branch on direct_reader flag.  Write operations return errors
  in direct reader mode.  Falls back to server path gracefully.
- ovsqlite-server.c: WAL checkpoint at shutdown uses PASSIVE first
  (non-blocking, safe with readers connected) then TRUNCATE only if no
  readers hold pins.  Prevents server shutdown from hanging.
- ovsqlite.c: Fix pre-existing buffer leak in ovsqlite_close():
  request and response buffers from server_connect() were never freed.
- sql-read.sql: read-only prepared statements with busy_timeout=10000
  and query_only=1.
- New readercachesize config parameter (default 8 MB per reader).

Includes unit tests (WAL gating, write rejection) and integration tests
that start a real ovsqlite-server, write compressed data through it,
kill the server, and verify the direct reader can read it back correctly
including a server restart cycle.

see #338

  Commit: aefdb3d888a6120eeb064d2ad03127e7dca707fa
      https://github.com/InterNetNews/inn/commit/aefdb3d888a6120eeb064d2ad03127e7dca707fa
  Author: Kevin Bowling <kevin.bowling at kev009.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M storage/ovsqlite/ovsqlite.c
    M tests/overview/ovsqlite-t.c

  Log Message:
  -----------
  call sqlite3_close_v2 when sqlite3_open_v2 fails

see #338

  Commit: 0eadd8d77dfa35df3a426d594c2f0acef6b3a31f
      https://github.com/InterNetNews/inn/commit/0eadd8d77dfa35df3a426d594c2f0acef6b3a31f
  Author: Kevin Bowling <kevin.bowling at kev009.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/ovsqlite.pod
    M samples/ovsqlite.conf
    M storage/ovsqlite/ovsqlite-server.c
    M storage/ovsqlite/sql-main.sql

  Log Message:
  -----------
  ovsqlite: Add runtime WAL checkpoint to prevent unbounded WAL growth

When WAL mode is enabled with direct readers, the WAL file can grow
without bound if checkpointing never occurs between commits.  Add a
configurable walcheckpointthreshold (default 1000 pages) that triggers
a TRUNCATE checkpoint during the server's idle period.

The checkpoint temporarily lowers the busy_timeout to 10 seconds
(from ~31 years) so the server waits for readers to finish their
current queries but doesn't stall indefinitely.  Readers use
autocommit with sub-millisecond snapshots, so the checkpoint
typically completes in milliseconds.

Also fix notice() messages being lost after daemon() by registering
message_handlers_notice, remove the unused checkpoint_wal prepared
statement from sql-main.sql, and update close_db() comment.

see #338

  Commit: 87c5029b47c83f24b2f1cb94756a7cd9bed317ec
      https://github.com/InterNetNews/inn/commit/87c5029b47c83f24b2f1cb94756a7cd9bed317ec
  Author: Kevin Bowling <kevin.bowling at kev009.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M storage/ovsqlite/ovsqlite-server.c
    M storage/ovsqlite/sql-main.sql

  Log Message:
  -----------
  ovsqlite: Set journal mode from open_db() based on configuration

Move pragma journal_mode out of the sql-main.sql preamble and into
open_db() where it is set conditionally: WAL when walmode is true,
PERSIST otherwise.  Previously the preamble unconditionally set
PERSIST mode before open_db() re-set WAL, causing a pointless
WAL->PERSIST->WAL transition on every startup (or silently failing
if direct readers held the WAL open).

see #338

  Commit: f70d3ff8e49718a103384b3eaa147f10d11e553b
      https://github.com/InterNetNews/inn/commit/f70d3ff8e49718a103384b3eaa147f10d11e553b
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M Makefile
    M contrib/makestorconf.in
    M frontends/cnfsheadconf.in
    M frontends/scanspool.in
    M scripts/inncheck.in

  Log Message:
  -----------
  Add new --warn-label-types perltidy code check

Report labels not written uppercase, unused, or referenced but missing.

Fix the occurrences of these nits, add the author of makestorconf, and
reformat inncheck with the latest upstream perltidy version.

  Commit: 56634ed7beae2c5b55f72ca056cb975ff2da5db1
      https://github.com/InterNetNews/inn/commit/56634ed7beae2c5b55f72ca056cb975ff2da5db1
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M storage/ovsqlite/ovsqlite.c

  Log Message:
  -----------
  ovsqlite: Fix unused variable without zlib support

see #338

  Commit: 79137c49ac40de0389603c878cd6c74388f372e1
      https://github.com/InterNetNews/inn/commit/79137c49ac40de0389603c878cd6c74388f372e1
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/ovsqlite.pod

  Log Message:
  -----------
  Homogenize POD formatting for default values

  Commit: 713c073a87b8421d0faaa0ac6bd8dc4e2d93aa29
      https://github.com/InterNetNews/inn/commit/713c073a87b8421d0faaa0ac6bd8dc4e2d93aa29
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M scripts/innreport_inn.pm

  Log Message:
  -----------
  innreport: Recognize new notice logs for WAL mode

see #338

  Commit: e41c58386368acac5f45295d998beed4862b78cd
      https://github.com/InterNetNews/inn/commit/e41c58386368acac5f45295d998beed4862b78cd
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/news.pod
    M storage/ovsqlite/ovsqlite.c
    M tests/overview/ovsqlite-integ.t
    M tests/overview/ovsqlite-read-t.c
    M tests/overview/ovsqlite-t.c
    M tests/overview/ovsqlite-write-t.c

  Log Message:
  -----------
  Add NEWS entry for additions to ovsqlite.conf

Also reference the author of the new contributed files.

see #338

  Commit: 8721849ed2de3e1810d2f963b0dae1a33a53234c
      https://github.com/InterNetNews/inn/commit/8721849ed2de3e1810d2f963b0dae1a33a53234c
  Author: Bo Lindbergh <blgl at stacken.kth.se>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M storage/ovsqlite/ovsqlite-server.c

  Log Message:
  -----------
  ovsqlite: Improved error messages from sqlite3_open

  Commit: ac97051a306b77f38489ce187ac240dd01f8ac39
      https://github.com/InterNetNews/inn/commit/ac97051a306b77f38489ce187ac240dd01f8ac39
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M scripts/innreport_inn.pm

  Log Message:
  -----------
  innreport: Do not count ctlinnd pause twice

Also remove old flush_file log.

  Commit: 4f9a3c62a78a413ae8b0b35eb280432b04f68ca8
      https://github.com/InterNetNews/inn/commit/4f9a3c62a78a413ae8b0b35eb280432b04f68ca8
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/ovsqlite.pod
    M samples/ovsqlite.conf
    M storage/ovsqlite/ovsqlite.c

  Log Message:
  -----------
  Decrease the default value of readercachesize

The documentation recommends a value smaller than cachesize.

see #338

  Commit: 5abd17395d08b8c60ae60e23bc4785050b9fb234
      https://github.com/InterNetNews/inn/commit/5abd17395d08b8c60ae60e23bc4785050b9fb234
  Author: Bo Lindbergh <blgl at stacken.kth.se>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M storage/ovsqlite/ovsqlite-server.c

  Log Message:
  -----------
  ovsqlite: Ensure the base is in read/write mode

Paranoid verification of successful read/write database open.

  Commit: 9a8cddb6ec74ab153eda988a03a2ea02a193e361
      https://github.com/InterNetNews/inn/commit/9a8cddb6ec74ab153eda988a03a2ea02a193e361
  Author: Kevin Bowling <kevin.bowling at kev009.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M .gitignore
    M MANIFEST
    M doc/pod/expireover.pod
    M doc/pod/inn.conf.pod
    M doc/pod/libinnhist.pod
    M expire/expireover.c
    M history/his.c
    M history/hisinterface.h
    M history/hisv6/hisv6-private.h
    M history/hisv6/hisv6.c
    M history/hisv6/hisv6.h
    A include/inn/bloom.h
    M include/inn/history.h
    M include/inn/innconf.h
    M include/inn/ov.h
    M lib/Makefile
    A lib/bloom.c
    M lib/innconf.c
    M samples/inn.conf.in
    M storage/expire.c
    M storage/ov.c
    M storage/ovinterface.h
    M support/mkmanifest
    M tests/Makefile
    M tests/TESTS
    A tests/lib/bloom-hiswalk-t.c
    A tests/lib/bloom-t.c

  Log Message:
  -----------
  expireover: Bloom filter for fast history checks

Add a bloom filter for fast history existence checks.

expireover checks every article in the overview database against the
history file to detect orphaned entries.  This requires a per-article
HISlookup, which does random pread() calls into the DBZ index and
history text file.  On large spools (1B+ articles), this takes months.

Add a bloom filter that is built from a single sequential HISwalk of
the history file at startup.  The bloom filter acts as a positive-only
cache in OVhisthasmsgid: bloom hits skip the slow HISlookup, bloom
misses fall through to HISlookup for correctness.  False positives
are benign (an orphaned overview entry survives one extra cycle).

The bloom filter is controlled by the new inn.conf parameter
expirebloomfp, which specifies the false positive rate as a reciprocal
(default 10000 = 0.01%).  Setting it to 0 disables the bloom filter.
Memory usage is approximately 20 bits per article (48 MB for 20M
articles, 2.4 GB for 1B articles).

Changes:
- Add lib/bloom.c and include/inn/bloom.h (bloom filter implementation
  using enhanced double hashing, Kirsch & Mitzenmacher 2006)
- Extend HISwalk callback signature to include the message-ID HASH
  (HISwalk has had zero callers since it was added in 2001)
- Set hisv6_walk ignore=true so corrupt lines don't abort the walk
- Add OVTOKENCACHE to OVctl for passing the bloom filter to
  OVhisthasmsgid
- Add expirebloomfp to innconf
- Add unit tests (lib/bloom-t.c) and integration tests
  (lib/bloom-hiswalk-t.c)

close #339

  Commit: f2ab3bf8da6e8faf2f1fdbf9781bff91181ffbdc
      https://github.com/InterNetNews/inn/commit/f2ab3bf8da6e8faf2f1fdbf9781bff91181ffbdc
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M backends/Makefile
    M control/Makefile
    M expire/Makefile
    M history/Makefile
    M innd/Makefile
    M lib/Makefile
    M nnrpd/Makefile
    M storage/Makefile

  Log Message:
  -----------
  Update make dependencies

see #339

  Commit: 0aa4d16abac91a0b4be1a0711692dfb9a1f25ebd
      https://github.com/InterNetNews/inn/commit/0aa4d16abac91a0b4be1a0711692dfb9a1f25ebd
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M history/his.c
    M history/hisinterface.h
    M history/hisv6/hisv6-private.h
    M history/hisv6/hisv6.h
    M include/inn/bloom.h
    M include/inn/history.h
    M include/inn/innconf.h
    M include/inn/ov.h
    M lib/bloom.c
    M lib/innconf.c
    M storage/expire.c
    M storage/ov.c
    M storage/ovinterface.h
    M support/mkmanifest
    M tests/lib/bloom-hiswalk-t.c
    M tests/lib/bloom-t.c

  Log Message:
  -----------
  Update authors and copyright years

see #339

  Commit: c03b7157aa54e0a411609c0e44615f3b479e7253
      https://github.com/InterNetNews/inn/commit/c03b7157aa54e0a411609c0e44615f3b479e7253
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/news.pod

  Log Message:
  -----------
  Add a NEWS entry for the new Bloom filter

see #339

  Commit: 642da9e2cbeb9d93dd51b53669ffe5283d7257e8
      https://github.com/InterNetNews/inn/commit/642da9e2cbeb9d93dd51b53669ffe5283d7257e8
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M MANIFEST
    M doc/pod/expireover.pod
    M doc/pod/inn.conf.pod
    M expire/expireover.c
    M history/hisv6/hisv6.c
    M include/inn/bloom.h
    M lib/bloom.c
    M storage/expire.c
    M tests/lib/bloom-hiswalk-t.c

  Log Message:
  -----------
  Uppercase Bloom

see #339

  Commit: 3cc2a204d89447cb2956867ab1e77bc38195680d
      https://github.com/InterNetNews/inn/commit/3cc2a204d89447cb2956867ab1e77bc38195680d
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M lib/snprintf.c

  Log Message:
  -----------
  snprintf: Silence stringop-overflow GCC warning

Not worth the effort fixing.  This function replacement will probably be
removed in a future release.

see #159

  Commit: 819313718d7ef9edca595fbba879fe0e4824ad94
      https://github.com/InterNetNews/inn/commit/819313718d7ef9edca595fbba879fe0e4824ad94
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M configure.ac

  Log Message:
  -----------
  Silence 3 warnings which are not actual problems

class-varargs, c++-keyword and implicit-void-ptr-cast

  Commit: c1d342532d512aaa989e877a222aa43c273badc0
      https://github.com/InterNetNews/inn/commit/c1d342532d512aaa989e877a222aa43c273badc0
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M history/hisv6/hisv6.c

  Log Message:
  -----------
  Silence overflow warning (mips)

The code works fine but emits a warning on mips because dev_t and
st_dev do not have the same size.

Silence the warning, and update the comment to mention what would be
the proper fix (an Autoconf check).

  Commit: 6d238c1387646a194876ef3091e68f28d32df218
      https://github.com/InterNetNews/inn/commit/6d238c1387646a194876ef3091e68f28d32df218
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M scripts/innreport_inn.pm

  Log Message:
  -----------
  innreport: Recognize ovsqlite-server WAL notices

These notices are attributed to ovsqlite-server, not nnrpd.

see #338

  Commit: a670115141cfc4b876597c280517f25958399e82
      https://github.com/InterNetNews/inn/commit/a670115141cfc4b876597c280517f25958399e82
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/install.pod
    M doc/pod/news.pod
    M m4/blocklist.m4
    M nnrpd/nnrpd.c
    M nnrpd/nnrpd.h
    M nnrpd/perm.c

  Log Message:
  -----------
  blocklist.m4: Also recognize old blacklist name

It will help the transition introduced by the renaming.

see #335

  Commit: 86d56da65231009f5e827739d1a0669bef91e145
      https://github.com/InterNetNews/inn/commit/86d56da65231009f5e827739d1a0669bef91e145
  Author: Kevin Bowling <kevin.bowling at kev009.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M MANIFEST
    M doc/pod/expire.pod
    M doc/pod/expireover.pod
    M doc/pod/inn.conf.pod
    M doc/pod/sm.pod
    M expire/expire.c
    M expire/expireover.c
    M expire/expirerm.in
    M expire/makehistory.c
    M frontends/sm.c
    M include/inn/innconf.h
    M include/inn/ov.h
    M include/inn/storage.h
    A include/inn/tombstone.h
    M innd/art.c
    M lib/innconf.c
    M samples/inn.conf.in
    M storage/Makefile
    M storage/expire.c
    M storage/interface.c
    M storage/ovinterface.h
    A storage/tombstone.c
    M support/mkmanifest
    M tests/Makefile
    M tests/TESTS
    A tests/expire/tombstone-e2e.t
    A tests/expire/tombstone-hisexpire-t.c
    A tests/expire/tombstone-t.c
    A tests/storage/cancel-tombstone-t.c

  Log Message:
  -----------
  expire: skip per-article SMretrieve (cancel log)

Adds an opt-in (expiretombstone in inn.conf, default false)
side-channel that records every cancellation alongside the existing
storage and overview deletions, so a subsequent expire run can drop
matching history entries without doing per-article
SMretrieve(RETR_STAT) syscalls.  At billion-article scale on
tradspool/timehash this turns a billion access(2) calls into a few
thousand hash lookups.

Two log files cooperate:

  ${pathdb}/expireover.tombstone -- written by expireover after each
  successful inline SMcancel; in delayrm mode written up front and
  promoted by expirerm after fastrm succeeds.  Atomic .NEW -> final
  rename under an exclusive non-blocking fcntl POSIX lock.  A
  crashed-or-failed previous run's leftover .NEW is verified
  per-token via SMretrieve and merged into the next run, so partial
  rmfile failures cannot orphan articles on disk.

  ${pathdb}/cancels.tombstone -- appended by innd's ARTcancel and by
  sm -r through a new public SMcanceltombstone() helper.  Shared
  fcntl POSIX lock for appenders, exclusive for expire's snapshot
  rename to .processing; one retry on EAGAIN closes the appender
  vs consumer race window.  POSIX O_APPEND atomicity for sub-
  PIPE_BUF writes keeps single-line tokens from interleaving.

expire reads both files into one hashset, treats every article in
either log as already gone, and falls back to per-article SMretrieve
only for storage methods where SMprobe(SELFEXPIRE) is true (CNFS
wrap-around).  An empty-but-present tombstone correctly says "no
cancels this cycle" and skips SMretrieve for everything; truly absent
files fall back to the slow path.

The format is line-per-token; readers ignore blank and #-prefixed
lines, and expireover writes "# inn-tombstone v1" as a forward-
compatibility header.  Disk and memory footprint scale linearly:
~38 bytes per entry on disk, ~50 bytes in expire's hash table.

Tests cover the public library (tombstone_hash_create, _read,
_rename_for_processing, _present), SMcanceltombstone branches
including expiretombstone=false bypass and unwritable-pathdb
non-crash, two-file merge with dedup, comment-line skip,
HISexpire integration with a callback that mirrors EXPdoline's
full decision tree (kept / tombstoned / selfexpire-gone /
selfexpire-alive), and an end-to-end shell test that exercises
sm + cancels.tombstone + expiretombstone toggling.

Default is false; sites should opt in after validating the
option's behavior against their workload.

see #340

  Commit: fd5fae2736807847e21df19477888af98c5e8d3b
      https://github.com/InterNetNews/inn/commit/fd5fae2736807847e21df19477888af98c5e8d3b
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M Makefile
    M scripts/inncheck.in

  Log Message:
  -----------
  inncheck: Disambiguate the names of the labels

Spotten by the new reuse of label names warning (perltidy).

  Commit: 8913f11acdadcab635b49f820e5d33c7da1b0538
      https://github.com/InterNetNews/inn/commit/8913f11acdadcab635b49f820e5d33c7da1b0538
  Author: Kevin Bowling <kevin.bowling at kev009.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/inn.conf.pod
    M nnrpd/article.c

  Log Message:
  -----------
  nnrpd: fast-path nnrpdcheckart via cancels.tombstone

When both innconf->expiretombstone and PERMaccessconf->nnrpdcheckart
are true, ARTinstorebytoken now consults ${pathdb}/cancels.tombstone
before calling SMretrieve(RETR_STAT).  A token recorded in the log is
reported gone without a syscall; a token absent from the log is
trusted to still exist on storage methods that do not self-expire,
again without a syscall.  Self-expiring backends (CNFS) still go
through SMretrieve because cyclic-buffer wrap-around bypasses the
tombstone.

The same trade-off as the expire-side fast path applies: the
tombstone catches in-band cancellations (innd ARTcancel, sm -r) but
not out-of-band events (manual filesystem deletes, corruption).
Admins running nnrpdcheckart=true to catch the brief overview-vs-
storage race during innd cancel processing, or to catch sm -r
artifacts before the next expireover cleans overview, still get those
paths.  Loss is bounded to events the tombstone subsystem cannot
see, which is consistent with the contract documented for
expiretombstone.

Loading is lazy: the hashset is built on first call into
ARTinstorebytoken and the file is stat()ed on every subsequent call
to detect change, with reload only when mtime or size changes.
Statting a fixed path stays in the dentry cache and is trivially
cheap compared to the per-article SMretrieve syscalls the fast path
elides, so cancellations recorded by other processes become visible
to long-lived connections immediately.  Freshness is keyed on both
mtime and size: mtime alone has 1-second granularity, so a rename-
and-recreate within the same second can land a fresh inode with
the cached mtime; size is monotonic-append between rotations and
catches every modification.  Partial reads (mid-file ferror) leave
the cached freshness key untouched so the next call retries.

The SMprobe(SELFEXPIRE) probe is checked before the tombstone
refresh so pure-CNFS sites pay only that probe (a static per-method
attribute, no I/O) and skip both the hash lookup and the cache load
entirely.

Memory cost is dominated by the hashset (~50 bytes per cancel;
typical sites have hundreds of entries).  When the file is missing
entirely, expiretombstone is off, or nnrpdcheckart is off, the
original SMretrieve path is unchanged.

inn.conf.pod's nnrpdcheckart entry now describes the fast-path
gating, the mtime/size-driven reload model, the manual-rm caveat
on non-self-expiring backends, and the OVERartcheck counter
becoming an undercount when the fast path is active.

see #340

  Commit: 68524273479c8b300dbfa0209b6fb9f501c8c123
      https://github.com/InterNetNews/inn/commit/68524273479c8b300dbfa0209b6fb9f501c8c123
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M .gitignore
    M expire/Makefile
    M frontends/Makefile
    M nnrpd/Makefile
    M storage/Makefile

  Log Message:
  -----------
  Update .gitignore and make dependencies

see #340

  Commit: 17fac84c736672361355ca7ce2821b837449e0f6
      https://github.com/InterNetNews/inn/commit/17fac84c736672361355ca7ce2821b837449e0f6
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M frontends/sm.c
    M include/inn/storage.h
    M include/inn/tombstone.h
    M scripts/innreport_inn.pm
    M storage/interface.c
    M storage/tombstone.c
    M tests/expire/tombstone-e2e.t
    M tests/expire/tombstone-hisexpire-t.c
    M tests/expire/tombstone-t.c
    M tests/storage/cancel-tombstone-t.c

  Log Message:
  -----------
  Update authors and copyright years

see #340

  Commit: fbaad3df6950a7db40c60a837ac829c51e04aeb3
      https://github.com/InterNetNews/inn/commit/fbaad3df6950a7db40c60a837ac829c51e04aeb3
  Author: Bo Lindbergh <blgl at stacken.kth.se>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M perl/INN/ovsqlite_client.pm

  Log Message:
  -----------
  Fix the no-unix-sockets path of ovsqlite_client.pm

  Commit: 198013062c0616a3731a4d8b0fe584337f68fd6f
      https://github.com/InterNetNews/inn/commit/198013062c0616a3731a4d8b0fe584337f68fd6f
  Author: Bo Lindbergh <blgl at stacken.kth.se>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M storage/ovsqlite/ovsqlite-server.c
    M storage/ovsqlite/sql-main.sql

  Log Message:
  -----------
  ovsqlite-server: Use sqlite3_randomness

It simplifies things!

  Commit: 5f06fbeddd9ea2b36a784c84eabc83bf4727e9e8
      https://github.com/InterNetNews/inn/commit/5f06fbeddd9ea2b36a784c84eabc83bf4727e9e8
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/news.pod

  Log Message:
  -----------
  Update NEWS with the latest improvements

  Commit: a9e3b4f2f8d6d167cd1b1fefe88f3a5ee0b44b11
      https://github.com/InterNetNews/inn/commit/a9e3b4f2f8d6d167cd1b1fefe88f3a5ee0b44b11
  Author: Kevin Bowling <kevin.bowling at kev009.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M lib/dbz.c

  Log Message:
  -----------
  dbz: Fix NULL innconf deref with mmaped tables

dbz set() dereferenced innconf->nfswriter unconditionally when writing
to an INCORE_MMAP table, but innconf may be NULL (it defaults to NULL
and is only populated by innconf_read()).  This segfaulted any caller
that opens history for writing without loading innconf first.

The crash surfaced under --enable-tagged-hash because hisv6 forces
pag_incore = INCORE_MMAP unconditionally in tagged-hash builds, whereas
non-tagged builds only use INCORE_MMAP when the HIS_MMAP flag is set.
The lib/bloom-hiswalk and expire/tombstone-hisexpire tests, which open
history for writing without setting up innconf,  segfaulted only in
tagged-hash builds.

Guard the dereference with a NULL check, matching the idiom already used
in other libinn/storage code.  nfswriter defaults to false, so a NULL
innconf correctly skips the NFS-writer msync optimization.

close #339
close #341

  Commit: 5494cc8d85b56cc3e9ee6d1beda6f77f96557825
      https://github.com/InterNetNews/inn/commit/5494cc8d85b56cc3e9ee6d1beda6f77f96557825
  Author: Kevin Bowling <kevin.bowling at kev009.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/ovsqlite.pod
    M samples/ovsqlite.conf
    M storage/ovsqlite/ovsqlite.c

  Log Message:
  -----------
  ovsqlite: honor mmapsize in the direct reader path

The mmapsize parameter was only applied by ovsqlite-server.  When WAL
mode is enabled, nnrpd readers open the database directly via
direct_open(), bypassing the server, so they never set pragma mmap_size.

Read mmapsize in direct_open() and apply it to the read connection,
mirroring the server.  Besides the read-performance benefit, memory-
mapping lets multiple nnrpd readers share the same physical pages through
the OS page cache.  Document the caveat that this helps little on ZFS or
other filesystems without a unified buffer cache, where the mapped pages
would be cached twice.

close #342

  Commit: c894ca85d107ace9b99553f39f5c2375f44aeced
      https://github.com/InterNetNews/inn/commit/c894ca85d107ace9b99553f39f5c2375f44aeced
  Author: Christian Clauss <cclauss at me.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M .github/workflows/build.yaml

  Log Message:
  -----------
  Update GitHub Actions checkout from v4 to v 6

https://github.com/actions/checkout/releases is currently v6.0.2

Why: Stop building on EOL versions of Node.js.

Fixes: Annotation on https://github.com/InterNetNews/inn/actions/runs/26530786871/job/78146587295

> Node.js 20 actions are deprecated. The following actions
are running on Node.js 20 and may not work as expected:
actions/checkout at v4. Actions will be forced to run with Node.js 24
by default starting June 2nd, 2026. Node.js 20 will be removed from
the runner on September 16th, 2026. Please check if updated versions
of these actions are available that support Node.js 24. To opt into
Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true
environment variable on the runner or in your workflow file. Once
Node.js 24 becomes the default, you can temporarily opt out by
setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For
more information, see:
https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners

close #343

  Commit: d4f5045bf595ede2610bb56a0710a5ee8eb474ab
      https://github.com/InterNetNews/inn/commit/d4f5045bf595ede2610bb56a0710a5ee8eb474ab
  Author: Julien ÉLIE <Julien-Elie at users.noreply.github.com>
  Date:   2026-06-02 (Tue, 02 Jun 2026)

  Changed paths:
    M doc/pod/expire.pod
    M doc/pod/expireover.pod

  Log Message:
  -----------
  Mention groupbaseexpiry along with expiretombstone

see #340

Compare: https://github.com/InterNetNews/inn/compare/fce56b09800f...d4f5045bf595

To unsubscribe from these emails, change your notification settings at https://github.com/InterNetNews/inn/settings/notifications