[bind10-dev] Hotspot cache sharing, was re: Multicore Auth

[Shane Kerr]

Jinmei, 

On Thu, 2011-07-14 at 18:15 -0700, JINMEI Tatuya / 神明達哉 wrote:
> - Another thing to be considered is how/whether to share hot spot
>   cache between instances (threads/processes) running on different
>   cores.  I've not fully thought about that, but I suspect we'd end up
>   each instance having separate cache.  That would lead to increase
>   total memory footprint, which may or may not be a big issue.  If
>   it's substantial, we need to consider a nice way to deal with that.

It seems to me the hotspot cache sharing is not *that* different from
cache sharing in general. We discussed this long, long ago and IIRC came
up with the basic two approaches:

     1. Separate caches
     2. Shared cache

(To avoid killing my fingers, every time you read 'cache' below, think
'hotspot cache'.) 

The disadvantages of the separate cache are not only the memory
duplication as you describe, but also lower cache hit rates. That is, if
task A gets a query and loads it into its cache, then task B has to
perform that work again to load it into its separate cache.

The disadvantage of shared cache is that we need to synchronize reads &
writes. This does not seem too different from the problems we have been
talking about with accessing the same in-memory data source. However the
searching is probably simpler for a cache, although there is the whole
expiration issue to worry about.

I was actually thinking of a model that combines separate and shared
caches. The idea is that there would be:

      * A single cache, read-only to query-handling processes. This
        could be managed in a similar style to Stephen's 2-copy solution
        for in-memory database.
      * Smaller separate caches, local to each query-handling process.

Periodically the query-handling processes would send the contents of
their caches to a central process, which would collect them and update
the main, shared cache.

This model is more complicated, but it would give us most of the benfits
of both a central and a separate cache. In addition to complexity, there
is a certain amount of 'sloppiness' that would mean that some queries
would be executed multiple times before the central cache is updated.

--
Shane