tinyfeed miniature transit server

[Christoph Biedl]

Russ Allbery wrote...

> I wrote this a while back for a personal project, and have just finished
> updating it to build against the current INN sources.  I find this pretty
> useful, but it's sort of strange.  Do people think that this is useful
> enough to include in INN and install along with everything else?

Well, I did something similar some time ago and it's still running. My
main intention was a delaying feed i.e. store articles in a local spool
and transmit them as soon as they are ten minutes old. Saves a lot of
traffic (yes I do know Christian Mock's delayer but again a full INN
installation would have been overkill for that box).

>     When tinyfeed starts, it will scan the contents of *spool*, looking for
>     any file names that are integer numbers.  It takes the largest number
>     found as a file name of any file in that directory, increments it by
>     one, and uses that as the first file name for a newly received article. 
>     Each additional article received will be stored in a file named with the
>     next highest number.  (Unpredictable things will happen if the article
>     name overflows an unsigned long.)

Using a number will cause trouble if there's one than one instance of
tinyfeed running on the same spool; or do you take precautings against a
race, say, locking the file?

Furthermore, if "unsigned long" is 32 bit and I transmit a !binary full
feed via tinyproxy with some 400k articles every day, I will run into an
overflow in a quite forseeable time.

May I therefore suggest to use the md5 hash of the Message-ID as file
name? This will also allow to reply with a "Duplicate" if the file is
already present; or even "Try again later" under certain circumstances
(file present but locked/size 0/not ending with "\n.\n").

>     If *processor* is given, it should specify the path to a program.  That
>     program is started when tinyfeed starts, and its current working
>     directory will be *spool*.  For each article received by tinyfeed, a
>     single line will be sent to standard input of *processor*.  That line
>     will consist of the file name of the received article (relative to
>     *spool*), a single space, and the message ID of the received article. 
>     Note that the message ID will be taken from the argument to the IHAVE
>     command and may not match the Message-ID header in the article.  When
>     tinyfeed shuts down, standard input to *processor* will be closed.

I understand that this might lead to files remaining unprocessed after a
crash?  Perhaps tinyfeed should check for files present in the spool at
startup and re-offer them to the processor.

> EXAMPLE
>     Suppose that you want to archive news articles on a particular host
>     (like the FTP server for a newsgroup archive) where you don't want the
>     overhead of running a full-blown news server.  Write a program that
>     reads one line at a time from standard input and treats everything
>     before the first space as the filename of a news article to archive. 
>     Each time the program reads a line, it should archive that file and then
>     delete it, and it should exit when it gets end of file on standard
>     input.

Hm, that's quite close to archive which will also create an according
directory structure - but on the same host only and without a "you've
got new articles".

	Christoph