CNFS buffers wrap much too soon

[Miquel van Smoorenburg]

I have INN-CVS of 31 januari running on one of our reader servers.
It was compiled under linux 2.4 with glibc 2.2 and LFS support.

The binary spool is spread over 4 disks, each with a 140 GB
partition for CNFS which together form one big metacycbuff.

However those buffers unexpectedly rollover at 1-5 GB instead of
at 140 GB. I've logged the output of cnfsstat every 5 minutes
so that you can see what's happening (the line "for groups matching"
doesn't make much sense, it's probably just taking the first line
of a matching storage.conf entry while there are multiple):

For example, hdc3 rolls over at 1.89 GB:

== time now: Wed Feb 12 02:35:01 CET 2003

Class bin  for groups matching "alt.fan.*,*binaries*", article size min/max: 0/50000
 Buffer hda3, size:   142 GBytes, position:  5.73 GBytes  38.04 cycles
  Newest: 2003-02-12  2:35:00,    0 days,  0:00:01 ago
 Buffer hdc3, size:   142 GBytes, position:  1.89 GBytes  40.01 cycles
  Newest: 2003-02-12  2:35:00,    0 days,  0:00:01 ago
 Buffer hde3, size:   142 GBytes, position:  66.4 MBytes  41.00 cycles
  Newest: 2003-02-12  2:35:00,    0 days,  0:00:01 ago
 Buffer hdg3, size:   142 GBytes, position:  96.0 MBytes  41.00 cycles
  Newest: 2003-02-12  2:35:00,    0 days,  0:00:01 ago

== time now: Wed Feb 12 02:40:01 CET 2003

Class bin  for groups matching "alt.fan.*,*binaries*", article size min/max: 0/50000
 Buffer hda3, size:   142 GBytes, position:  5.76 GBytes  38.04 cycles
  Newest: 2003-02-12  2:39:48,    0 days,  0:00:14 ago
 Buffer hdc3, size:   142 GBytes, position:  41.6 MBytes  41.00 cycles
  Newest: 2003-02-12  2:39:48,    0 days,  0:00:14 ago
 Buffer hde3, size:   142 GBytes, position:  92.8 MBytes  41.00 cycles
  Newest: 2003-02-12  2:39:48,    0 days,  0:00:14 ago
 Buffer hdg3, size:   142 GBytes, position:   127 MBytes  41.00 cycles
  Newest: 2003-02-12  2:39:48,    0 days,  0:00:14 ago

.. and hda3 rolls over at 5.9 GB:

== time now: Wed Feb 12 03:15:01 CET 2003

Class bin  for groups matching "alt.fan.*,*binaries*", article size min/max: 0/50000
 Buffer hda3, size:   142 GBytes, position:  5.91 GBytes  38.04 cycles
  Newest: 2003-02-12  3:14:57,    0 days,  0:00:04 ago
 Buffer hdc3, size:   142 GBytes, position:   192 MBytes  41.00 cycles
  Newest: 2003-02-12  3:14:57,    0 days,  0:00:04 ago
 Buffer hde3, size:   142 GBytes, position:   256 MBytes  41.00 cycles
  Newest: 2003-02-12  3:14:57,    0 days,  0:00:04 ago
 Buffer hdg3, size:   142 GBytes, position:   285 MBytes  41.00 cycles
  Newest: 2003-02-12  3:14:57,    0 days,  0:00:04 ago

== time now: Wed Feb 12 03:20:01 CET 2003

Class bin  for groups matching "alt.fan.*,*binaries*", article size min/max: 0/50000
 Buffer hda3, size:   142 GBytes, position:  51.3 MBytes  39.00 cycles
  Newest: 2003-02-12  3:19:28,    0 days,  0:00:33 ago
 Buffer hdc3, size:   142 GBytes, position:   211 MBytes  41.00 cycles
  Newest: 2003-02-12  3:19:28,    0 days,  0:00:33 ago
 Buffer hde3, size:   142 GBytes, position:   277 MBytes  41.00 cycles
  Newest: 2003-02-12  3:19:28,    0 days,  0:00:33 ago
 Buffer hdg3, size:   142 GBytes, position:   303 MBytes  41.00 cycles
  Newest: 2003-02-12  3:19:28,    0 days,  0:00:33 ago

Weird, right? Wrapping at 2 GB or 4 GB I could see, but this looks
pretty random to me.

I have this running on other servers, but I cheated and compiled
only cnfs.c in 64 bit mode and the rest of the server in 32 bit
mode, and that does actually work. I never tried a full LFS
compile before with large CNFS buffers, so I can't tell if this
has been working correctly before.

I'm going to put some debug messages into cnfs.c probably around
the place where it says "CNFS-sm: cycbuff hda3 rollover to cycle .."
but if anyone has seen this before I'd like to know.

This is cycbuff.conf:

cycbuff:hda1:/dev/hda1:10240000
cycbuff:hda2:/dev/hda2:1027000
cycbuff:hda3:/dev/hda3:148809000
cycbuff:hdc1:/dev/hdc1:10240000
cycbuff:hdc2:/dev/hdc2:1027000
cycbuff:hdc3:/dev/hdc3:148809000
cycbuff:hde1:/dev/hde1:10240000
cycbuff:hde2:/dev/hde2:1027000
cycbuff:hde3:/dev/hde3:148809000
cycbuff:hdg1:/dev/hdg1:10240000
cycbuff:hdg2:/dev/hdg2:1027000
cycbuff:hdg3:/dev/hdg3:148809000
metacycbuff:text:hda1,hdc1,hde1,hdg1:INTERLEAVE
metacycbuff:control:hda2,hdc2,hde2,hdg2:INTERLEAVE
metacycbuff:bin:hda3,hdc3,hde3,hdg3:INTERLEAVE

and this is storage.conf:

# Small articles from the bin hierarchy
# must be put next to the binaries themselves.
method cnfs {
        newsgroups: alt.fan.*,*binaries*
        class: 1
        size: 0,50000
        options: bin
}

# Put control articles in the control spool
method cnfs {
        newsgroups: control.*
        class: 2
        size: 0,50000
        options: control
}

# Put small articles in the text spool
method cnfs {
        newsgroups: *
        class: 3
        size: 0,50000
        options: text
}

# Now the defaults: store as binary.
method cnfs {
        newsgroups: *
        class: 4
        size: 0,0
        options: bin
}

-- 
Anyone who is capable of getting themselves made President should
on no account be allowed to do the job -- Douglas Adams.

-- 
The From: and Reply-To: addresses are internal news2mail gateway addresses.
Reply to the list or to miquels at cistron-office.nl (Miquel van Smoorenburg)