[bind10-dev] Statistics gathering and publishing

[Jeremy C. Reed]

In June, Evi suggested some type of "stats daemon" similar to syslogd for 
listening to and handling statistics.

It would be nice to have sometype of generic handler for this, maybe could 
be reused by other projects.

Gathering ideas:

- Received over a UNIX domain datagram log socket (like /dev/log)

- received over TCP / maybe using XMPP?

- received over UDP / maybe using SNMP?

Message data tagged with:

- sender: program, hostname

- identifiers: the type of data (more specific that syslog facility). This 
  clearly describes the data so it can be stored and reused correctly.

Store ideas:

The "stats" daemon could be configured to:

- dump to flat file

- forward to another stats daemon server

- reformat and log via syslog (for low quantities)

- pipe into a storage tool (like rrdtool)

- create RRD (rrdtool compatible Round Robin Database files)

- store in memory, configured for a certain limits and stop when full or 
  cycle

- maybe configuration for this could also have threshold checks and 
  possible notifications -- i.e. simple real-time monitoring and 
  reporting.

Retrieval:

- if configured to store in memory, then need a way to ask for it.

- be able to request / filter type of data

Publishing:

- separate tool to use previously stored data or real-time retrieved data 
  from stats daemon and generate BIND 9 style XML

- use rrdtool (for time-based stats) to generate reports

- use outside tools to generate graphs.

-=-=-=-=-=-=

I found a tool but haven't tried it yet:

collectd
http://collectd.org/

- It gathers statistics.

- Can listen on Unix domain socket, unicast, IPv6, multicast.

- It can write RRD files.

- Data is collected and sent (pushed) to a multicast group or server.

- It doesn't generate graphs. Other tools for that.

- Over 70 plugins. For example for BIND 9 XML and libpcap DNS traffic:

	http://collectd.org/wiki/index.php/Plugin:BIND

	http://collectd.org/wiki/index.php/Plugin:DNS

- Can encrypt traffic.

- Can use signed or encrypted traffic.

- SNMP support

- Can do threshold checking for notifications (as a monitoring type 
system).

- supposedly lightweight and efficient where one daemon can handle 
hundreds of clients

- I don't see any plugin for collectd to listen on or send data via XMPP.

- collectd is licensed with GPLv2

Maybe collectd could work for BIND 10 stats collection.

Or maybe it should be looked at for ideas.

  Jeremy C. Reed
  ISC