innduct - a replacement for innfeed/nntpsend, 0.1 alpha

Ian Jackson ijackson at chiark.greenend.org.uk
Wed Aug 28 23:22:16 UTC 2013


Julien ÉLIE writes ("Re: innduct - a replacement for innfeed/nntpsend, 0.1 alpha"):
> First of all, my apologies for not having answered sooner.  I am aware 
> your original message dates back to three years ago!  Better answering 
> now than never, though.

That's approaching some of my my own worst email reply lag.  :-).

> > I've been running nntpsend for quite a while.  I wanted to run
> > realtime feeds but I have a problem with innfeed from a reliability
> > point of view[1].
> >
> > In particular, the program feed protocol spoken between innd and
> > innfeed is lossy: if innfeed dies unexpectedly, articles which innd
> > has written to the pipe to innfeed will be skipped.  innd has no way
> > of telling which articles those are, no useful records, and no code to
> > retry.  I can see no sensible way to solve this problem without using
> > a different approach to getting articles from innd.
> 
> Couldn't you run innfeed in funnel-file mode instead of channel mode? 
> This way, innd would continually write to a file that would continually 
> be processed by innfeed.

In principle, yes, but:

> > I looked at editing innfeed to make it tail the feed file with
> > inotify/kqueue but innfeed is an very large program for what it does -
> > nearly 25kloc.
> 
> Maybe the funnel-file mode I am speaking about is what you are hinting 
> at here.  Instead of editing innfeed to tail the funnel-file on its own, 
> a separate script doing that could be periodically run by crond.

I'm not sure I follow.  In order to have both reliability and
timeliness, the lines need to be read out of the funnel file (or, as
innduct has it, the feed file) immediately via inotify, and then
whatever is transmitting the articles needs to use some appropriate
protocol to record where it got up to.  innfeed doesn't do this.

> > Since this is rather an alpha release, I'm not providing a tarball or
> > (heaven forbid) binary packages or anything.  But you can get the
> > source with:
> >    git clone git://git.chiark.greenend.org.uk/~ian/innduct.git
> 
> Would you mind if we mention innduct in:
> * the "Related Packages" section of README
>      http://www.eyrie.org/~eagle/software/inn/docs/readme.html

Sure.

> * the innfeed man page (maybe in the "BUGS" section, as a workaround to 
> the lossy behaviour you mention, or in a new "ALTERNATIVE" section)
>      http://www.eyrie.org/~eagle/software/inn/docs/innfeed.html

Of course.

> Other INN-related pages you see on which it would be worth mentioning 
> innduct? (maybe nntpsend and innxmit)

Yes.

> Is the URL
>      http://www.chiark.greenend.org.uk/ucgi/~ian/git/innduct.git
> OK for you to use, or do you prefer another one?  (tarball, HTML page 
> describing the project or HTML version of the man page...)

It's not the best page, is it.  But the alternative is probably to
wait for me to set up some kind of automatic gitweb manpage formatter
or something.  So please use that URL.

> Thanks again for your work, and sorry for only getting back to you now...

Thank you!

I'm expecting to have to do some work on innduct fairly soon because
I'll be upgrading to Debian wheezy which has a new version of INN.  I
want, obviously, to be able to rebuild my innduct...

Ian.


More information about the inn-workers mailing list