Overview dump and restore tool

Russ Allbery rra at stanford.edu
Tue Jul 5 01:05:28 UTC 2005


Heath Kehoe <hakehoe at avalon.net> writes:

> I have written a dump and restore tool for overview, which can be used
> to convert an overview database from one storage method to another, or
> to back up an overview database.

> The dump tool acts as a client, doing OVsearch on every group in the
> active file and printing to stdout a portable, text format dump of all
> overview records.  The restore tool reads that dump and uses OVadd to
> place the records into a new overview database.

> Both tools provide command line arguments that allow you to override key
> settings in inn.conf, such as the ovmethod and pathoverview settings, so
> you can do a command line like this:

>    ovdump -m tradindexed -p /old/ov | ovrestore -m ovdb -p /new/ov

> The manual at http://www.avalon.net/~hakehoe/inn/ovdumprestore.html has 
> more usage examples.

> The tools should work with INN 2.4 or newer.
> Download here: http://www.avalon.net/~hakehoe/inn/ovdumprestore.tar.gz

I've now had a chance to look at this in more depth.

For incorporation into the CURRENT branch, it would be best if the tools
were converted to the new overview API.  One of the nice side effects of
this is that the Xref modifications aren't necessary, since with the new
API one can tell the overview database exactly which group and article
number to store the article into, but it does mean that the dump format
needs to be modified to include the newsgroup and article number.

I'd like to standardize the overview dump format between this and overchan
so that ovrestore could be merged with overchan, with the additional
options that ovrestore takes added to overchan.  This means that overchan
needs to learn how to read a dump format that includes the newsgroup name
and article number into which to store the article, rather than requiring
an Xref header in the overview data.

My proposal would be to extend the format so that:

    <newsgroup>:<number>,<newsgroup>:<number> <token> <arr> <exp> <data>

is also valid as a dump format, easily distinguishable from the current
format by looking at the first field and seeing if it starts with @ like
all tokens do.  overchan could then be taught to read either format, and
eventually INN could write the new format, which would mean that Xref
could become optional in overview again if we wanted.  (We've had a few
requests for that from people with specialized needs who put Newsgroups in
overview already, don't care about crosspost-chasing, and care about the
space.)

This would also require modifying ovdump to write the above format and use
a single space rather than a newline as the field delimiter.

Once we complete the conversion to the new overview API, the expires time
will also be returned by the API.

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>

    Please send questions to the list rather than mailing me directly.
     <http://www.eyrie.org/~eagle/faqs/questions.html> explains why.


More information about the inn-workers mailing list