[bind10-dev] comments on the statistics design

[Naoki Kambe]

Hello,

From: JINMEI Tatuya / 神明達哉 <jinmei at isc.org>
Date: Sat, 28 Jul 2012 21:20:49 -0700
> At Fri, 27 Jul 2012 14:06:09 +0900 (JST),
> Naoki Kambe <kambe at jprs.co.jp> wrote:
> 
> > > - is each module expected to reset their statistics to 0 (if that's
> > >   resettable) every time it responds to a request from the stats?  I
> > >   guess so because otherwise the accumulated data at stats won't make
> > >   sense, but it's not clear from the document.  What if it's not
> > >   "resettable" (such as number of RRs of some zone/cache, etc,
> > >   assuming we consider it "statistics" and want to maintain it)?
> > 
> > To be honest, I think the stats module just keeps a copy of statistics
> > data in each module. If the target module say "the new value is 100",
> > the stats replaces the old value with the new value.
> 
> But how does (e.g.) b10-auth maintain per-zone statistics on millions
> of zones (and some/many of which may not be in-memory)?  And how can
> it reasonably send the statistics data upon request?
> 
> My understanding of our discussion at the f2f meeting is that b10-auth
> will keep a limited-size buffered statistics in-memory (but the limit
> won't normally be reached unless it has a huge number of zones) and
> flushes the in-memory statistics every time it responds to a request
> from stats or it spontaneously sends it to stats due to a buffer-full
> condition.

Regarding statistics data, it doesn't have to support millions of
zones by the end of Sept., I think. However, for preventing lots of
memory usage from Auth, we can introduce a zone-size limitation. But
it can be handled on another ticket related to Auth.

> 
> > > - I guess we need to revisit the representation of statistic
> > >   (counters), especially in terms of the spec.   Things like per
> > >   RR-type counter aren't easily represented in this style.
> > 
> > I think JSON format isn't basically friendly to represent varying
> > statistics data. :(
> 
> I'm afraid not either, and so I suspect we need to redesign it.

We three will attempt to define items like per RR-type counter as well
as other already implemented items in the spec. Can we handle this
representation issue after this Oct.?

> > > - Now that we switch to the "request model", I think it also makes
> > >   sense to consider more synchronized update upon user request (via
> > >   bindctl or http).
> > 
> > Do you mean a new command for the stats module to collect statistics
> > data is needed? e.g a "collect-immediate" command. That makes sense
> > for me if so. But if a administrator sets 'poll-interval' to 1,
> > statistics data would be refreshed after a second. But I didn't make
> > such a command because of preventing a high load in system.
> 
> What I envisioned is a revision of the "show" command; the new version
> basically immediately invokes the "send statistics" request from stats
> to the module(s).  We probably want to rate-limit this, e.g., at most
> 1 request per second though.  The stats daemon wait for the
> response(s), and sends the result back to cmdctl.

Stats doesn't request other module for one second or less since the
last request. It shows only the result of the last request of one
second or less past.  Otherwise, it requests and then it shows the
latest result of that. In that case, it takes longer time to
return. That makes sense for me.

Naoki Kambe