Offloading readers in INN + CNFS

[Dan Merillat]

I've had a co-worker propose a solution to making multiple newservers.

Obviously, duplicating 10s of gigs of storage for each reader box is
infeasable, so he proposed the following solution:

send the overview information to the reader boxes, where they maintain
it.  They then call the storage box to grab specific article-IDs for
the client.

This would work, except the client will be getting quite a few
"article not found" errors, since it's going to be difficult to
determine when the article is overwritten on the storage server.

So, the question is, how difficult would it be to determine
which article is being overwritten in CNFS when a new article is
stored?  if this could be done, a few nice things could happen:

A) for standard INN, overviews could be marked as "dead" as soon as
it happens.

B) for offloaded readers, there would be no need for complex
syncronization of history-like files, as they maintain an up-to-date
concept of what exists and what does not.

Obviously, this would be optional since the overhead on a transit
server would be rather large.   But, for a CNFS only box, it may be
desirable.   Perhaps checking at the 512 byte boundries for something
like \0[msgid-hash]\0, (which should never occur in a valid message)
and then looking up that token for delete processing.

If this sounds possible I'll look into it, since a solution like this
would make for a very scalable INN deployment.

--Dan