any requests

[Vernon Schryver]

> From: Dave Warren <davew at hireahit.com>

> >> I thought Google Public DNS re-fetched RRsets as they were expiring in
> >> >order to keep the cache populated, which would explain what you see,

> > I don't understand how they could pre-fetch the gazillions of RRsets
> > that are rarely requested.

> ...
> I'm not convinced they really bother with any of that though, I wonder 
> if they don't just have giant shared caches on powerful, well connected 
> boxes.

I don't know about "shared," but there's no doubt about the rest of that.


> Either way, when you're playing with a single test domain, 
> experimentally, they'll absolutely expire just the way anybody else does.

If I were doing what I might understand them to be doing, and if I
were obsessed with initial page loading speed, then I might play an
easy game with TTLs and extra recursing.
When the remaining TTL of an RRset is positive but small,
answer the DNS request from the cache as usual but also start fetching.
When the remaining TTL is large or <=0, do the usual things.
That would keep popular RRsets in the cache without having needing
to decide explicitly which domains are popular today or for a
particular instance of the recursive server.
If "positive but small" were a small multiple of the RTT to the authority
(including the authority's queue and processing time), such as a constant
0.75 or 1.0 seconds for all RRsets everywhere,
then its ratio to the original TTL would prevent more than 1% on
average of the pre-emptive fetches from being wasted.

It sounds hard to see whether they are playing that sort of game from
outside.  On the other hand, I think too many of the records in their
responses to my ANY requests for my test domain have TTLs of 0.

I think it would not be too hard to hack that early recursion into
BIND, and so perhaps other DNS implementations.  Making an ANY
psuedo-RRset sounds messy, because of maintaining its TTL as the minimum
of the TTLs of the RRsets at the node even as non-ANY requests refresh
individividual RRsets.

The point of all of this that is not mere gossip is:
  - Don't just assume that ANY requests are useless.
  - Don't just assume that no applications have used, use them now, or 
     will use them in the future.
  - Don't just assume all DNS servers treat ANY requests the same.
  - Don't just assume that no significant DNS servers respond to them in
     any way that is not explicitly outlawed by a standards track RFC.
     And you might need to deal with some outlawed responses.

ANY requests are like the SMTP <> sender, missued and abused by attackers
and filtered by operators based on dubious assumptions.
Filtering ANY is not as bad is blocking all ICMP or blocking TCP/53,
but it comes from the same school of security "expertise."


Vernon Schryver    vjs at rhyolite.com