XOVER ranges

Jeffrey M. Vinocur jeff at litech.org
Fri Apr 4 12:25:55 UTC 2008


On Thu, 3 Apr 2008, Russ Allbery wrote:

> "Jeffrey M. Vinocur" <jeff at litech.org> writes:
> 
> >   (2) Change CMDgetrange() to not adjust the watermarks
> 
> I think 2 is clearly the right solution if all of the overview backends
> can handle ranges outside the current watermarks.  However, I haven't
> looked at any of the code for long enough that I don't remember how it
> works.

Oh dear, did I just volunteer myself to look at the other overview 
methods?

I guess there's also:

   (4) Incorporate the refreshing of high watermarks and limiting of 
       actual range into the calling code, so that the OVsearch code
       doesn't have to deal with adjusting the watermarks (or 
       remapping the index files) at all.  I don't know how many other
       places rely on OVsearch's ability to do the remapping though.


> > P.S.  Russ, any thoughts about my original supposition?  
> 
> I'm not sure that I saw your original suggestion, unless it was in a
> message that I replied to.  I thought I'd replied to all your mail.

I meant this bit below (in contrast to the bit about XOVER ranges):

 
> >       I'm still worried there might be a bug lurking in the
> >       unmap_file/map_file that can take place in the middle of the search,
> >       since SendIOv may still have references to data in the old location.
> 
> Oh, yes.  Given that tradindexed advertises OVSTATICSEARCH false, it
> shouldn't be doing that.  Good catch.  It either needs to advertise
> OVSTATICSEARCH true to force nnrpd to copy the data or it needs to never
> remap in the middle of a search.  I think the latter is more correct.

Yay, I found a bug.

I just wish it was actually the cause of the strange behavior I'm seeing!

 
> That may require other changes to not remap the index file in the middle
> of a search either.  We really should only remap the files at the time of
> opensearch.

I was going to go ahead and do this, but I had trouble understanding why 
you thought we might ever need to remap in the middle of a search.

Right now you check for a stale high watermark when the search is opened
and remap the index file, and then during the search you check for data 
file offsets that are past the end and remap the data file there.

I thought about just updating the open code to remap both if it found a
stale watermark, but I suppose that potentially we could get in trouble
there if there's a write to the data file that hasn't flushed yet?  We
could try checking during open to make sure the data file entry
corresponding to the high watermark is within the mapped region, but I 
don't know what we would do if it isn't.


-- 
Jeffrey M. Vinocur
jeff at litech.org


More information about the inn-workers mailing list