slow expire/expireover, 12/31/99 snapshot

[Paul Theodoropoulos]

solaris7 64bit kernel, 384M, U5 server. inn snapshot of 12/31/99, built 
with largefiles support. rawdisk cnfs spool with small additional 
tradspool. rawdisk buffindexed overview.

i am going into swap-hell while expire runs, and expireover is taking about 
ten times longer to run than with the old snapshot (before the group-based 
expiry mods to buffindexed) - yesterday's run took 5.5 hours. history and 
overview were fully rebuilt when i went to the 12/31 snapshot, adn the 
server runs essentially fine, otherwise.

a pmap of the expire process shows, among other things,

root-nnrp /usr/local/news/db% /usr/proc/bin/pmap 13627
13627:  expire -v1
00010000    144K read/exec         /usr/local/news/bin/expire
00042000     16K read/write/exec   /usr/local/news/bin/expire
00046000 318616K read/write/exec     [ heap ]

which pretty much explains why i'm going into massive swapping.

my history file is 1396589553 bytes, with article retention (outside of 
binaries) running between 34 and 53 days. There are very few readers at 
this point, rarely more than two concurrent. Going by the formula in the 
INSTALL, with a tablesize of 23017915, i should have enough ram to not have 
to use tagged hash - yet it looks like i may need to.

am i reading the formula wrong, or is something perhaps fubar with my config?

(addendum, i just ran a brief truss on expireover, and noticed that it's 
spewing a lot of the following, wondering if this is a problem or not:

pwrite64(49, " 1 0 0 8 9\t R e :   * G".., 427, 0x3974DE35) = 427
pwrite64(52, "\0\0 ' i\001CBA0\0010302".., 56, 0x397A0B00) = 56
pwrite64(52, "\0\0\0\0FFFF\0\0\0\0 ' 7".., 16, 0x397A0000) = 16
fcntl(51, F_SETLKW64, 0xFFBECDA0)               Err#11 EAGAIN
time()                                          = 947103909
fcntl(51, F_SETLKW64, 0xFFBECDA0)               Err#11 EAGAIN
fcntl(50, F_SETLKW64, 0xFFBECDA0)               Err#11 EAGAIN
time()                                          = 947103909
fcntl(50, F_SETLKW64, 0xFFBECDA0)               Err#11 EAGAIN
fcntl(49, F_SETLKW64, 0xFFBECDA0)               Err#11 EAGAIN
time()                                          = 947103909
fcntl(49, F_SETLKW64, 0xFFBECDA0)               Err#11 EAGAIN
fcntl(49, F_SETLKW64, 0xFFBECDA0)               Err#11 EAGAIN
time()                                          = 947103909
fcntl(49, F_SETLKW64, 0xFFBECDA0)               Err#11 EAGAIN
fcntl(53, F_SETLKW64, 0xFFBECDA0)               Err#11 EAGAIN
time()                                          = 947103909
fcntl(53, F_SETLKW64, 0xFFBECDA0)               Err#11 EAGAIN
fcntl(52, F_SETLKW64, 0xFFBECDA0)               Err#11 EAGAIN
time()                                          = 947103909
fcntl(52, F_SETLKW64, 0xFFBECDA0)               Err#11 EAGAIN
fcntl(51, F_SETLKW64, 0xFFBECDA0)               Err#11 EAGAIN
time()                                          = 947103909

sorry for the length of this message...
-----------------------------------------------------------------
Paul Theodoropoulos                    Advanced TelCom Group Inc.
Senior Unix Systems Administrator      Internet Services Division
paul at atgi.net                          Santa Rosa, California, US
Work: http://www.atgi.net         Play: http://www.anastrophe.com