Slow Expire: Over 24 hours to finish

[James F. Hranicky]

Expiring articles on my new news server is taking a long time
(24 hours):

   Expire messages:
   expireover start Tue May  1 02:02:06 EDT 2001: \
     ( -z/usr/local/var/news/expire.rm -Z/usr/local/var/news/expire.lowmark)
       Article lines processed  3013082
       Articles dropped             690
       Overview index dropped       924
   expireover end Tue May  1 15:09:12 EDT 2001
   lowmarkrenumber begin Tue May  1 15:09:12 EDT 2001: \
     (/usr/local/var/news/expire.lowmark)
   lowmarkrenumber end Tue May  1 15:09:13 EDT 2001
   	expirerm start Tue May  1 15:09:13 EDT 2001
   	expirerm end Tue May  1 15:09:21 EDT 2001
   expire begin Tue May  1 15:09:51 EDT 2001: (-v1)
       Article lines processed  7106072
       Articles retained        6658748
       Entries expired           447324
       Old entries dropped       446523
       Old entries retained     2131061
   expire end Wed May  2 02:24:56 EDT 2001
   	all done Wed May  2 02:24:56 EDT 2001

History sizes:

   </usr/local/libdata/news/db> # du -sk
   1552    active
   1552    active.old
   15      active.times
   774528  history
   1       history.dir
   86232   history.hash
   114968  history.index
   184     newsgroups

Overview size:

   </usr/local/libdata/news/overview> # du -sk
   6647913 .

innd size:
    PID USERNAME     PRI NICE  SIZE    RES STATE    TIME   WCPU    CPU COMMAND
    258 news           2   0   136M 55008K select 262:45  2.98%  2.98% innd

Here's the config:

	FreeBSD 4.3-STABLE
	650 MHz PIII
	Intel GX440 motherboard
	256MB RAM
	Adaptec 39160
	8 external 18G LVD (80MBs) drives
	  - spool 	: 6 drives RAID 0 (104G available)
	  - overview	: 1 drive 	  (17.3G available)
	  - hist	: 1 drive 	  (17.3G available)

	Spool: 
	  - alt.binaries	: 5G cycbuff
	  - alt			: 20G cycbuff
	  - control.cancel	: 1G cycbuff
	  - *			: tradspool

	overview	: ovdb
	overchan	: yes 
	  - overview!:*,!control.cancel:Tc,WnteO:/usr/local/bin/overchan

Here the compilation config:

	o ./configure --prefix=/usr/local --with-control-dir=/usr/local/bin 
	  --with-db-dir=/usr/local/libdata/news/db --with-etc-dir=/usr/local/etc/news 
	  --with-filter-dir=/usr/local/libexec/news --with-lib-dir=/usr/local/lib 
	  --with-log-dir=/usr/local/var/news --with-run-dir=/usr/local/var/news/run 
	  --with-spool-dir=/usr/local/spool/news --with-tmp-path=/usr/local/var/news/tmp 
	  --with-sendmail=/usr/sbin/sendmail --enable-libtool --with-openssl 
	  --with-berkeleydb=/usr/local --with-perl

	o OpenSSL 0.9.6
	o Berkeley DB 3.2.9
	o perl 5.6.0
	

Anyone know why expireover and expire are taking so long? I know
tradspool isn't terribly efficient, but a find on the spool takes
just under 3 minutes.

I even had to rebuild the ovdb due to a hung server that had to 
be powercycled, and that took under 6 hours.

Do I simply need to add more memory?

Thanks,

----------------------------------------------------------------------
| Jim Hranicky, Senior SysAdmin                   UF/CISE Department |
| E314D CSE Building                            Phone (352) 392-1499 |
| jfh at cise.ufl.edu                      http://www.cise.ufl.edu/~jfh |
----------------------------------------------------------------------
        - If I can't share your intellectual property, - 
        -  why can you share my personal information ? -
                      - Vote for Privacy -