[bind10-dev] comment to StatsModule 2010/6/1

Naoki Kambe kambe at jprs.co.jp
Fri Jun 4 11:10:27 UTC 2010


Fujiwara,

Thanks for your comments. I replied to your comments in trac ticket
#170. See http://bind10.isc.org/ticket/170#comment:6
This discussion may move to trac ticket later.

Naoki Kambe

From: Kazunori Fujiwara <fujiwara at wide.ad.jp>
Subject: [bind10-dev] comment to StatsModule 2010/6/1
Date: Tue, 01 Jun 2010 23:49:51 +0900 (JST)

> I will comment "StatsModule" idea.
> 
> http://bind10.isc.org/wiki/StatsModule
> 
> |+------ Sending modules -----+
> || +------+ +------+ +------+ |
> || | Boss | | Auth | | etc. | | <- *1
> || +------+ +------+ +------+ |
> |+-----^--------^--------^----+
> |      |        |        |
> |      +--[CC protocol]--+
> |               |               <- *2
> |               v
> |        +--------------+  
> |        |    Stats     |       <- *3
> |        +--------------+ 
> |               |               <- *4
> |               v 
> |      +-----------------+
> |      |    Cmd-Ctrld    |      <- *5
> |      +-----------------+
> |
> |*1 Modules except Boss and Auth, which send stats data to stats module,
> |   is not supported in initial version of stats module
> 
> No, it has nothing to do with the statistics specification.
> 
> |== Procedure of stats module ==
> |=== Basic procedure ===
> | * Initial process:
> |   0. Boss starts stats daemon and other modules.
> | * Main process in loop:
> 
> First, "statistics module" contacts config manager.
> 
> Configuration changes and commands from bindctl are come from config
> manager.
> 
> |   1. Stats starts to subscribe in stats channel.
> |   1. Other modules send stats data to stats module periodically.
> |   1. Stats module collects data and then aggregates it.
> |   1. When print_stats command is invoked via bindctl, stats daemon
> |      reports formatted statistics data via bindctl.
> | * Final process:
> |   X. When Boss is shutting down, stats module and other modules are
> |      killed.
> 
> |== Collecting items ==
> |Stats module collects following items from Boss and Auth.
> | * In general (for both modules)
> |   * version -- A version number of this stats data definition
> 
> version is not necessary in the protocol because the version number
> will be written in *.spec configuration file.
> 
> |   * module -- A module name which sends the  stats data
> |   * process_id -- A process id of the module
> 
> process_id is not used in another part of BIND 10.
> So, local name defined in msgq may be better.
> 
> |   * processes -- A number of processes of the same module, if
> |     multiple processes of the module are running
> 
> then, "process_id" may be a list of processes.
> 
> |   * send_time -- Milli-seconds of current time since epoch time
> |     (1970-01-01T00:00:00Z)
> 
> why milli second?
> I prefer unixtime + microsecond (struct timeval) format.
> 
> What are "T" and "Z" characters?
> Text printable format is hard to parse.
> 
> |   * sequence -- A sequence number which must be unique and consistent
> |     in the sending module
> 
> Is this necessary?
> 
> | * For Boss module
> |   * boot_time -- A date time when BIND 10 starts up, format is
> |     YYYY-MM-DDTHH:MM:SSZ
> | * For Auth module
> |   * queries_in [[BR]]
> |     * tcp -- A number of query counts per a process which Auth servers receives in
> |       TCP since it sends last
> |     * udp -- A number of query counts per a process which Auth servers receives in
> |       UDP since it sends last
> 
> CC module has good counters.
> 
> |== Reporting items ==
> |Stats module reports following items via bindctl.
> 
> This format will be generated by parsing each *.spec file.
> 
> | * Local name -- A localname, which is returned from msgq module in CC protocol
> 
> Local name is assigned for each process.
> 
> | * Boot time  -- A date time when BIND 10 process starts
> | * Reported time -- A date time when stats module reports
> | * Process id    -- Process ids of all related modules
> | * Incoming Queries (TCP) -- A calculated query counts by stats module
> | * Incoming Queries (UDP) -- A calculated query counts by stats module
> |
> |This is an example of output image via bindctl.
> |{{{
> |    ++ BIND 10 Statistics Report ++
> |    Local name: 4bea7903_4 at host
> |    Boot time:  2010-05-13T05:19:43Z
> |    Report time:      2010-05-13T05:44:41Z
> |    Process id(Boss):         777
> |    Process id(Auth):         888
> |    Process id(Stats):         999
> |    Incoming Queries (TCP):     8888
> |    Incoming Queries (UDP):     9999
> |    ++ BIND 10 Statistics Report ++
> |}}}
> 
> The output format requires another knowledge.
> The statistics module only knows input data format.
> Or we must define output data format definition.
> 
> My idea was:
> 
> * BIND 10 Statistics report
>   Report time: ...
> 
>   bind10.LocalName: xxxx at localhost
>   bind10.BootTime: ...
> 
>   auth.LocalName: xxx
>   auth.queries.tcp: ...
>   auth.queries.udp: ...
> 
> |== Available commands in bindctl ==
> |Two commands via bindctl are available in initial version of stats
> |module.
> | * "print_stats" command:
> |    Stats module aggregates current numbers and prints the list of
> |    them by using formatted text.
> 
> print_stats command may have module name argments.
> print_status without arguments show all statistics.
> 
> | * "print_clear" command:
> 
> typo. It is "clear_statistics".
> 
> |    Stats module resets query counts to zero. If this command is
> |    invoked, then at first 'Are you sure?' prompt to confirm it.
> 
> The command may also have module name arguments.
> 
> |== Backend DB for stats module ==
> |'''(TBD)'''
> |A specific DB, like sqlite3 or Berkeley DB, is not used in stats
> |module in initial version. It's assumed that stats module keeps
> |aggregated data in memory.
> 
> It is not defined, I think.
> 
> |== Message format ==
> |'''Message format from Boss module to Stats:'''
> |{{{
> |#!js
> |{
> |  "stats_data":
> |    {
> |      "General":
> |        {
> |          "version": "1.0",
> 
> It is not necessary.
> 
> |          "module": "Auth",
> 
> The parameter may be included in the data itself.
> 
> |          "process_id": 777,
> 
> The localname in the envelope is sufficient.
> 
> |          "send_time": "2010-05-13T05:40:41Z",
> 
> It may be included in the data itself.
> 
> |          "sequence": 2345,
> 
> It is not necessary.
> 
> |        },
> |      "Boss":
> 
> Is it a mistake? Is it "Auth:"?
> But It contains module name.
> Then "General" section is not necessary.
> 
> |'''Message format from Auth module to Stats:'''
> |{{{
> |#!js
> |{
> |  "stats_data":
> |    {
> |      "General":
> |        {
> |          "version": "1.0",
> |          "module": "Auth",
> |          "process_id": 888,
> |          "processes": 2,
> |          "send_time": "2010-05-13T05:40:41Z",
> |          "sequence": 2345,
> |        },
> |      "Auth":
> |        {
> |          "queries_in":
> |            {
> |              "tcp" : 123,
> |              "udp": 4567
> |            }
> |        }
> |    }
> |}
> |}}}
> 
> To simplify the format, I propose a new data format.
> Add a "timestamp" as a unixtime in the outermost.
> >From or Local name is obtained from envelope.
> Module name is "Auth" which exists in the data.
> 
>       {
>        "timestamp": unixtime,
> |      "Auth":
> |        {
> |          "queries_in":
> |            {
> |              "tcp" : 123,
> |              "udp": 4567
> |            }
> |        }
>       }
> 
> |== Data schema ==
> |A schema which defines above massage formats, filename of which is configured in
> |spec file for stats module.[[BR]]
> |'''stats_data_schema.spec:'''
> |{{{
> |#!js
> |{
> |  "stats_data":
> |    {
> |      "description": "A schema for BIND 10 stats data definitions \
> |      		      using JSON schema syntax (http://json-schema.org/)",
> |      "type": "object",
>                ~~~~~~~~
>                  I prefer "dict".
> 
> The "stats_data" schema is defined for each module.
> I prefer it will be written in *.spec file.
> 
> I prefer new spec file format will be:
>   { "module_spec": { ... },
>     "commands": { ... },
>     "stats_spec": { ... }
>   }
> 
> The "stats_spec" format should be compatible with "module_spec" and
> "commands" format.
> 
> -- 
> Kazunori Fujiwara
> _______________________________________________
> bind10-dev mailing list
> bind10-dev at lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind10-dev
> 



More information about the bind10-dev mailing list