Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Hi persons,

background: I'm working for a small ISP in the Netherlands and have been ru=
nning
inn for ages.

a few months ago we reinstalled the newsserver on new hardware and at the
same time decided to go for CNFS buffers for a few fastflowing hierchies.
No problem there.

Now it seemed my history file went up to 2GB. Ouch! Problem. After a lot
of analysing and trying to find out what the problem was (I have moderate
experiation times for most group, like 7 days. It should never account for
21*10e6 (miljoen?) records!)

Okay, suddenly i notices that two of my cnfs buffers where hardly getting
any data in. (BIGARTICLES and ARTICLES). They never cycled in 155 days
and to top it off, every article was mentioned in the history file.
For those 155 days :( Wow. that went fast.

Than, working from the assumption that the history file is only used
to know what articles to expire and to remember if some articles has
been seen in the last /remember/ days, i concluded that all lines
referencing CNFS old that /remember/ days are superflocious and
removable.

I created a small commandline utility which purges the history file of
old cnfs based entries (old as beeing user defined). However I could
see the need that expire did such a thing itself. And maybe not only
for CNFS but all self-expiring storagemethods.
You can run it be either throttling innd (or just stopping it),=20
running the tool and afterwards makedbz. Than unthrottle innd.

    Met vriendelijke groet,
        Pauline Middelink
--=20
GPG Key fingerprint =3D 2D5B 87A7 DDA6 0378 5DEA  BD3B 9A50 B416 E2D0 C3C2
For more details look at my website http://www.polyware.nl/~middelink


-- Attached file included as plaintext by Ecartis --

#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>
#include <fcntl.h>
#include <time.h>

#define STORAGE_CNFS	"03"

int main(int argc, char * const argv[])
{
	FILE *fp;
	char	line[256];
	char	hash[128];
	char	date[128];
	time_t	adate;
	char	token[128];
	int	days = 7;
	time_t	remember;

	while (1) {
		int ch = getopt(argc,argv,"r:");
		if (ch == -1)
			break;

		switch (ch) {
		 case -1: break;
		 case 'r':
				days = atoi(optarg);
				break;
		}
	}

	if (optind+1 != argc) {
		fprintf(stderr, "Usage: %s [-r daystoremember ] <historyfile>\n",argv[0]);
		exit(1);
	}

	time(&remember);
	remember -= days*86400L;

	fp = fopen(argv[optind],"r");
	if (fp==NULL) {
		perror(argv[optind]);
		return 1;
	}
	while (fgets(line,sizeof(line)-1,fp)) {
		char *p;

		if (sscanf(line,"%s\t%s\t@%s\n",hash,date,token) != 3) {
			fputs(line,stdout);
			continue;
		}
		adate = -1;
		p = strchr(date,'~');
		if (p) {
			*p = '\0';
			adate = atol(date);
			*p = '~';
		}

		// determine the storage method
		// if the storage method is CNFS and its older than
		// my remember period, lets forget about it
		if (memcmp(token,STORAGE_CNFS,2)==0 && adate<remember)
			continue;

		// we got a left-over line
		fputs(line,stdout);
	}
	fclose(fp);
	return 0;
}



-- Binary/unsupported file stripped by Ecartis --
-- Type: application/x-pkcs7-signature