INN not keeping up with incoming news

[Alex Kiernan]

Fabien Tassin <fta at sofaraway.org> writes:

> According to Alex Kiernan:
> > 
> > We've just taken to locking the history.{hash,index} in core; its
> > worked wonders
> 
> without touching to the INN code ?
> 

Without touching the INN code.

The reason it works (and this is Solaris specific, but may apply to
every OS with a page cache, depending on exactly how they deal with
mmap and IO):

The files are MAP_SHARED, so shareable amongst all processes. When the
hash file is used in inn (mmaped in dbz), you just need to take minor
page faults in the caller to link them into the memory map. When you
use the index file (not mmaped in dbz), you enter the kernel and
because all IO happens via mmap you take minor page faults to link
them into the kernel memory map, whereon we copy to user
land. Modifying INN to mmap the index file so you avoid this copy
looks like an optimisation.

> > - we've gone from spending hours per day in HIS* to under 10
> > minutes (although I've no doubt that some of the time is hidden
> > elsewhere). Thinking about how you do page cleaning in an OS, I
> > can imagine that the INN history hash is a hard problem to solve
> > when its mmap()ed. There's an interesting comment from Matt Dillon
> > I turned up:
> > http://groups.google.com/groups?start=10&hl=en&lr=&safe=off&th=d0e8de5661dacd45&rnum=15&seld=955735264&ic=1
> 
> the whole thread is *very* interesting.
> 

It was, wasn't it.

> > If anyone's interested, here's the code we're using (it doubtless has
> > some Solaris assumptions, I haven't tried building it anywhere else):
> 
> I haven't tried your program yet but I wonder if this job should not be
> done in dbz.c automatically.

The problem is you need to be root to use mlock() - I played with the
idea of helper processes which INN communicates with (so I could avoid
the polling), but in the end keeping it simple was what I went with.

> I also wonder about its behaviour during and after
> expire/shutdown/crashes/sync etc. If the files are never committed on disks,
> the risk is very high to corrupt or lose everything.
> 

The crash risk is partially already there - on exactly what basis does
each platform flush modified mmaped pages, how each platform then
manages syncs from pages which are mlocked is another part of the
problem (AFAICT Solaris manages both problems in the same way, the
mlocked pages are just ignored by the page scanner when stealing
pages).

I guess we could reduce the risk by msyncing the pages every time the
code wakes up.

Also your text history file is complete, I'm not playing games with
that one, so you can always rebuild.

I'm not sure I understand what risk you see on expire/shutdown/sync?

-- 
Alex Kiernan, Principal Engineer, Development, Thus PLC