Overview changes

[Alex Kiernan]

Russ Allbery <rra at stanford.edu> writes:

> Alex Kiernan <alexk at demon.net> writes:
> > Russ Allbery <rra at stanford.edu> writes:
> 
> >> This by itself doesn't make a lot of sense to me in that I can't see
> >> how the overview interface could support it in a way that would be any
> >> faster than just calling OVopensearch for the full range and then
> >> discarding anything below the lower arrival time bound.  But maybe it's
> >> intended for use in combination with the second one?
> 
> > For ov3 (which is all I've implemented it for) it relies on the fact
> > that articles get numbered in order of arrival (thinks... have I
> > introduced a dependency between innd and the overview manager?).
> 
> Note that this doesn't necessary hold in the presence of Xref slaving,
> although it should be okay in other circumstances, and even mostly for
> Xref slaving.
> 

OK, I'll work something in for that when I manage to get time to merge
the code.

> > So once the .IDX mapped you can do a binary search through it to locate
> > a starting point, before starting the linear search. Like I said I
> > hacked it in, so currently the code I'm running does:
> 
> >   OVopensearch()
> >   ov3binsearch()
> >   OVsearch()
> 
> > Where ov3binsearch then advances search->cur based on the binary
> > required arrival time.
> 
> > For other overview methods based on databases clearly you could add
> > another index, for buffindexed, I don't know.
> 
> Ah, okay.  Out of curiosity, do you have any benchmarking on how much time
> this saves for heavy NEWNEWS usage?
> 

Nothing formal, but we brought down our load average at peak times
from ~200 to ~40 - though more than half of our users use NEWNEWS.

> The basic idea seems reasonable to me, although there is a possibility
> that it won't return the same results as the current method given that
> there isn't a strict correlation between arrival time and article numbers.
> 

I actually cheat and always pick an arrival time which is slightly too
early then weed those out using the existing check during the scan.

-- 
Alex Kiernan, Principal Engineer, Development, THUS plc