[bind10-dev] recursor cache requirements - input required

[Michal 'vorner' Vaner]

Hello

On Wed, Dec 08, 2010 at 11:04:12PM +0000, Stephen Morris wrote:
> On 7 Dec 2010, at 10:32, Michal 'vorner' Vaner wrote:
> > How about an interface where we would put a dns message into the cache, it would
> > split it and store only the RR sets and we could ask for individual RR sets
> > then?
> 
> It may not be as easy as that.  If the data in the DNS message were compressed, it would be likely that some decompression would need to occur first.  The compression pointers might point to parts of the message that are not stored.

I somehow expected here that the cache is inside the program and does not care
about the network format ‒ that it gets something like isc::dns::MessagePtr.
Then it could use the fact that it already contains created RRsets and it is
linked there by shared pointers, so just link it into itself ‒ it wouldn't
create anything, just prevent them from being destroyed.

The message for user would be rendered again at the very end, after creation,
multiple queries to the cache, etc.

> > Actually, it might be possible to act in two ways: allow add/remove
> > operations for when dumping or loading during runtime, and disallow for
> > faster, lock-free operation when starting up or shutting down.
> 
> For dumping, could we not fork the process and have the child write the copy of the cache at our leisure?  That would work on Unix-based systems at least.
> 
> As to loading, it depends how long loading will typically take.  If a short time, we can construct the cache before we begin operations; if a long time, a separate component that works through the dump in the background and does an "add if not here and if the stored data has not yet expired" should work.

This one seems like a nice, elegant solution :-).

> > We may also want a way to completely empty the cache. If nothing else
> > this can be useful in debugging. :)
> 
> If the cache is a single object, how about deleting it and creating a new instance?

And then copy the data from the old one to the new one?

> The compression technique involve pointers to absolute offsets in the message.  If an RRset is found in more than one message, (potentially) we will have to have one compressed version of the RRset for each message it is in.  It may well be simpler to cache the wire format of each message as-is and create associated index structures mapping it to the constituent RRsets (and vice-versa).  Then if any of the RRsets is updated (other than a TTL update that we can explicitly write back into the message), drop the message and re-query.

Again, I think that storing wire format here, deep inside the application, does
not sound right to me. The cache will interact with other parts inside and the
other parts expect the data to be parsed. I doubt we ever want to send the same
message as we received, so even when sending we need a parsed versions of RRsets
and build a new message of it.

> > Why I'd like to search the large one first is, the cache might have never
> > (better/authoritative) data than the small one. I want to use the better data
> > from large cache.
> 
> I think the idea is that the sets of data in each cache are disjoint.  The L1 cache contains local zones (and authoritative data).  L2 contains everything else.

Well, the L1 and L2 cache are really different features then (and we are mixing
auth into it right now and remember, we might not be able to load the whole
authoritative zone we have into some cache, it might be huge, generated
on-demand, etc).

With the cookies, the cookies would be small and short-lived, but really not
disjoint from the main cache. I believe, in most cases, the cookie would not be
used.

> > I do not think you should keep any message in the cache. I have been involved in more than one recursive server and have never seen the response message from the upstream cache fill or the answer created for the client kept. You always have to regenerate the answer from the validated data in the cache to prevent cache poisoning attacks, so keeping the upstream message is a bad idea. You also have situations where the answer from before is no longer correct. (a classic example of this is that the first answer used the glue NS rrset for the authority data and that rrset was later replaced by the authoritative zone NS rrset or a new query within the zone included a new NS rrset in its authority section.) You need to build everything you need from the current cache information.
> 
> I appreciate that, yet I would think that in some situations - queries for www.google.com for example - a pre-formatted reply that can be sent immediately could be a big performance improvement.

But I think this is a different kind of cache than this and would be on a
different level/different place in the big idea of recursor. It would be some
kind of hot cache or something.

Have a nice day

-- 
When a fly lands on the ceiling, does it do a half roll or a half loop?

Michal 'vorner' Vaner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <https://lists.isc.org/pipermail/bind10-dev/attachments/20101209/f91dc214/attachment.bin>