[bind10-dev] Comment to Statistics Ideas

[fujiwara at wide.ad.jp]

I wrote my idea of statistics daemon in this mail and comment to
"Statistics Ideas" at http://bind10.isc.org/wiki/StatisticsIdeas.

Please comment.

My idea of statistics daeon was:

* Assumption
 - Each AuthSrv has the same dataset in one BIND 10 system.
 - Each AuthSrv has each statistics counters
 - Stats daemon collects statistics data (counter set) from all AuthSrvs
 - Stats daemon publishes statistics
 - Querylog is managed by another system

* My idea was
 - Stats daemon sends a collecting command to each AuthSrv via the CC channel
    periodically
 - Stats daemon totals each counter collected from AuthSrvs
 - Stats daemon publishes one counter set
 - Stats daeomn holds short time data only
 - SNMP Stats server respond the recent couter set

What we need to define and fix:

 - Stats couter list
   - static or dynamic?
   - if dynamic, need to consider counter managing command.
 - Statistics collecting command/protocol in CC channel
   - data format
 - Collecting frequency
 - How to publish
   - SNMP
   - XML ?
   - command?
 - How to implement internal database.

After here, comment to StatisticsIdeas in-line.
-------------------------------------------------------------------------------
| = Statistics Ideas =
| 
| This wikipage is for brainstorming ideas for the statistics interface.
| 
| == Existing BIND 9 features ==
| 
|   * have around 130 counters, such as IPv6 requests received, Requests with TSIG received, Zone transfer requests rejected, etc.
|   * almost all statistics counters supported in BIND 8

Need to define which counters each AuthSrv has.

|   * XML-based statistics interface

Is the interface outputs totals of all AuthSrvs' counters?

|   * statistics counters about internal status (sockets/tasks/memory usage)

These counters cannot be totaled/merged.

|   * file based statistics (but not needed if going to just XML)
|   * per zone statistics

number of zones may be large number.
need to define how to add/remove zones to collect zone's statistics.

| == Ideas ==
| 
|   * per remote host statistics

need to consider how to collect.

# I use querylog to evaluate per remote host statistics.

|   * RESTful interface to get to specific statistics in XML (not everything at once)

need to define output interface

|   * all BIND 10 components may have statistics counters (even if not DNS related)

need to define.

|   * Separate daemon to handle HTTP requests and/or generate XML reports; BIND 10 will have separate processes running so having a central location to submit counter information might be useful.

Is it statistics daemon?

|   * will the daemon store details on all statistics?

Internal database ? external database ?

|   * use CC / msgq for subscribing to statistics and to send or receive stats data

Yes, I assumed.

|   * How much will the stats daemon hold?

Short time?

|   * Will the stats daemon periodically write to disk to store its collected data?

Yes. Stats daemon periodically write to on-memory or on-disk database.

|   * "statistics daemon" (ala syslogger) that is generic to listen for stats messages, then keeps counters or other growing stats data, and then can export them in some format(s) which can be used for reports and graphs. (Evi Nemeth shared the initial idea for this stats daemon at June 2009 face-to-face meeting.)

True, I think.

|   * It will also listen for queries so can provide the individual stats counters (or data) in near real-time.

I don't think so. It's AuthSrv's job.

|   * A daemon because many different programs may be sending stats data to it simultaneously; for example, a HTTPD webserver could submit various counters to it and a DNS server could submit counters to it also.

need to define counter list.

|   * Does it need to be a daemon? Can't it be a tool that's invoked by things that works on files?

If all components run on the same machine,
shared memory is the best solution, I think.

But for example, query counter should be added up each counter from each AuthSrv.

|   * What's wrong with many different programs invoking an utility to alter an on-disk database?
|   * On-disk may be too slow? So this would only write to disk periodically (even if every second).

The stat daemon may store data to disk database, I think.

|   * Then again, the initial program submitting the stats would have its own counters and doesn't need to submit them in real time. So maybe speed doesn't matter.

If there are multiple AuthSrvs which have same data, Stats need to add
up each counters and it is preferable to collect information from each
AuthSrv on the same time.

|   * What about top-ten lists, such as top ten records getting queries? If the server is hosting 1 million zones, would the stats daemon need to keep track of all of these? Or should the auth server component itself keep track?

need to define what AuthSrv collects and collecting protocol.

|   * Why not just query the individual components for their details and just have non-daemon tool do these queries and generate custom reports? (So no stats daemon needed or always running.)

Stats program may start by SNMP queries, or periodically, or always running.

|   * What about just having the zone databases also have data fields for different per-record and per-zone counters?

I don't know. need to define each AuthSrv has same data or not.

|   * Would it be useful to keep track of record creation/birth time, record change/modification time? record last access/read time? (per record or per RRset?)

I think it may be another function.
DNS Update may be logged by logging daemon or AuthSrv.

---------------------------- 

Regards,

--
Kazunori Fujiwara, JPRS