tradspool convert to CNFS

[Richard Todd]

In message <sg3doq9w9s.wl at opaopa.org>, Kenichi Okada writes:
>
>I give up to use the tradspool, so I want to convert it to CNFS.
>What is the best way to do it.

As Katsuhiro Kondou pointed out in his email, there's no tool as shipped in
innd to do the migration of articles.  In principle it wouldn't be hard to 
write a program to read in a list of articles and for each one, read that 
article and write it out anew to the storage manager in whatever place
the current storage.conf tells it to file that article.  In fact, I've got
such a program I wrote a few months back when I needed such a thing.  There
are some major caveats, though:

1) This program is only lightly tested.  I've never tried migrating an entire
news spool, just limited parts of one. 
2) You'll need (temporarily) double the usual amount of space for the articles,
since you'll need to have CNFS cycbuffs large enough to hold all your articles
in place before you start the migration.
3) This program doesn't even try to update history/overview data for the
articles as they move.  You'll want to run makehistory afterwards. 
4) The tool assumes all articles to be migrated already are in some one of
the areas accessible thru the storage manager.  If you're migrating from 
a pre-2.3 traditional spool setup, you'd first need to set up 2.3 with
a "tradspool" entry in storage.conf and do 'makehistory' so the storage
manager can find tokens for all your articles.

Anyway, here's the program, I call it refile.c.  You'd cause it to refile
articles by first editing storage.conf to include both your new CNFS
spool and the tradspool entry, but store everything in the CNFS (i.e. have
an empty Newsgroups: directive in the tradspool section of storage.conf)
Then you'd  do something like 
	awk -F'	' 'NF==3 {print $3}' <history | refile
to do a refile operation on all the spool. (note the thing that looks like a 
space in the first argument to awk is actually a tab!)

And as I said, this hasn't been tested on a full spool migration and may
not work there.  But if it does work for you, we'd all like to know.

/*
** Refile articles into the storage manager under the current storage.conf
** rules, deleting articles from their old place in the spool.
** Written 10-09-99 by rmtodd at servalan.servalan.com
*/

/* include foo needed by libinn/storage manager */
#include <stdio.h>
#include <errno.h>
#include <sys/types.h>
#include "configdata.h"
#include "clibrary.h"
#include "libinn.h"
#include "macros.h"
#include "paths.h"
#include "qio.h"
#include <syslog.h>  

char *ME;

void
ProcessLine(char *line)
{
    char *dateptr;
    char *tokenptr;
    char *dp1;
    char *dp2;
    int len,i;
    ARTHANDLE *art;
    ARTHANDLE newart;
    char *path;
    char *pathseg;
    char *p1, *p2;
    TOKEN token, newtoken;
    char *arttmp;
    time_t arrived;

    tokenptr = line;
    
    /* zap newline at end of tokenptr, if present. */
    len = strlen(tokenptr);
    if (tokenptr[len-1] == '\n') {
	tokenptr[len-1] = '\0';
    }

    token = TextToToken(tokenptr);
    if ((art = SMretrieve(token, RETR_ALL)) == NULL) return;

    len = art->len;
    arrived = art->arrived;
    arttmp = NEW(char, len);
    memcpy(arttmp, art->data, len);
    SMfreearticle(art);
    if (!SMcancel(token)) {
	fprintf(stderr, "%s: cant cancel %s:%s\n", ME, tokenptr, SMerrorstr);
	return;
    }

    newart.data = arttmp;
    newart.len = len;
    newart.arrived = (time_t) 0; /* set current time */
    newart.token = (TOKEN *)NULL;

    newtoken = SMstore(newart);
    if (newtoken.type == TOKEN_EMPTY) {
	fprintf(stderr, "%s: cant store article:%s\n", ME, SMerrorstr);
	return;
    }
    DISPOSE(arttmp);
    printf("refiled %s ",TokenToText(token));
    printf("to %s\n", TokenToText(newtoken));
    return;
}

void usage(void) {
    fprintf(stderr, "Usage: %s [-d directory] [-h hopcount] [-t timedelta]\n");
    exit(1);
}

main(int argc, char **argv)
{
    int		c;
    int 	one = 1;
    char	buff[SMBUF];

    openlog(argv[0], L_OPENLOG_FLAGS | LOG_PID, LOG_INN_PROG);

    if (ReadInnConf() < 0) { exit(1); }

    ME = argv[0];

    if (!SMsetup(SM_PREOPEN, &one) || !SMsetup(SM_RDWR, (void *)&one)) {
	fprintf(stderr, "can't init storage manager");
	exit(1);
    }
    if (!SMinit()) {
	fprintf(stderr, "Can't init storage manager: %s", SMerrorstr);
    }
    while (fgets(buff, SMBUF, stdin)) {
	ProcessLine(buff);
    }
    exit(0);
}