FAQ Questions?

Christoph Biedl cbiedl at gmx.de
Tue Apr 27 12:54:29 UTC 2004

divya at avnika.com wrote...

> Here are some questions that I have that should probably be in the FAQ.
> (1) I would like to know if there is a way to do something more granular:
>  How do I feed all articles in a given hierarchy from one server to
>  another? For example, if I wanted to just have all the articles in
>  the newsgroups under comp.lang.* fed from one server to another, what
>  is the best way to do so?
> This is refinement of the question http://www.eyrie.org/~eagle/faqs/inn.html#S6.4,

Based on the answers of:
- Are the articles in tradspool and/or do you have tradindexed overview?
- Do you want to feed a single group or complete subhierarchies?
- Will you mind the articles will not be transmitted in chronological order?

You can easily extract the Message-IDs from articles in tradspool or from
tradindexed overview.  After that, grephistory -s will give you the tokens
to be fed to innxmit or similar.  However this will break the
chronological order if you wish to feed more than one group.

Therefore I've set up a special feed entry
| # keep a list of where the articles went to
| newsgroups\
|         :*\
|         :Tf,WtmNb:
that will generate lines like
| 1083016801 <c6k0ot$tuk$1 at newsg3.svr.pol.co.uk> alt.comp.freeware 982
A daily cronjob moves spool/outgoing/newsgroups to
/some/where/else/newsgroups.`date +%d` and does of course a
"ctlinnd flush newsgroups".  This allows me to keep track of the last
month's articles, which groups they've been posted to, when they arrived
etc.  In case I want to re-feed a subhierarchy, some awk/grep will find
the according articles.

> (2) How do I change article storage without breaking article numbering
>  or having to re-initialize the server?

I'd shutdown the server for a day during week-end, feed the articles to
a file, trim the history and re-inject the file.  But that's probably
"re-initialize".  I vaguely remember less drastic ways to do that.

>  I have grown to dislike the storage of articles in tradspool formats.
>  This is because we would rather not expire intranet/private/local
>  groups or have an insanely long article retention policy (like 4 years).
>  Even for intranet groups, this means 40-100K articles. In tradspool,
>  this means 40-100K little files in a single directory which I suspect
>  is not conducive to performance.

Depends on the underlying file system.  Some have a directory indexing
which should keep access times low: reiserfs, ext3 (newer versions/special
option?) or xfs (not sure). "ls -l" is still a bad idea in such a
directory though.

> (4) How do I mark a newsgroup as read only?

ctlinnd changegroup group.name n
(or even 'x')

Or is the group in a controlled hierarchy like comp.*?  (Hm, appearently
not.)  In that case, I'd just ignore the docheckgroup's output of
"non-standard group".  Just curious: How would you determine when it's
time for the group to be finally removed?

>  I didn't understand the readers.conf documentation well enough here.

Doing this via readers.conf sounds like a lot of pain.  Correct me if
I'm wrong.

> (5) Is there any way to disallow non-plain-text articles?
>  Although I don't agree with it totally, many users bitterly complain
>  about users who use Outlook or Netscape/Mozilla news readers often post
>  in HTML format. I agree that is bad netiquette, but don't care much about
>  implementing filters to block that traffic. However, should the neec arise
>  to do so, what approach would be the best?

See perl filters.  Usually cleanfeed does this job quite well.

> (6) Backups of CNFS Buffers?
>  Is there a way to temporary throttle the news server, make a backup 
>  copy of the CNFS buffers and then unthrottle the news servers? This 
>  would allow me to take a snapshot of the news server for backup 
>  purposes.

I'm not sure wheter 'ctlinnd throttle' is save enough for that operation.
Complete shutdown will surely do :)
Anyway, I consider backing up a cnfs buffer rather silly, since:

>  What other data do I need to back up so that I can restore the system
>  to this last known good state at a later time?

You'll have to take care of overview, active and history, too (and
probably some other file I forgot).  INN is not very suitable for a backup
while it's running.  And copying around gigs of buffers and other files,
main parts of them actually not changed, takes a huge amount of storage,
I/O and CPU.

Therefore I've decided a different approach: Every incoming article is
written to a batch file which get rotated on a regular base.  Recovery is
done using rnews.  
Advantage: Relativly small backup, daily incoming, 300-500 MB, can be
Drawback: Long downtime during recovery since INN will have to rebuild
overview and history from the articles.  But having a backup is the best
protection against a crash, isn't it?

> (7) Are CNFS buffers portable? Can I take CNFS buffer from my Linux
>  system and SCP it over to a Solaris SPARC system and expect them to be
>  usable? WHat are the limitations?

No idea.  By the way, if I'll ever find the time to contribute to INN,
programs like fsck.cnfs and dumpfs.cnfs are the things I'd like to write
for quite some time.


More information about the inn-workers mailing list