BIND 10 #2136: update the stats daemon according to the new statistics model

[BIND 10 Development]

#2136: update the stats daemon according to the new statistics model
-------------------------------------+-------------------------------------
                   Reporter:         |                 Owner:  UnAssigned
  naokikambe                         |                Status:  reviewing
                       Type:         |             Milestone:
  enhancement                        |  Sprint-20120731
                   Priority:         |            Resolution:
  medium                             |             Sensitive:  0
                  Component:         |           Sub-Project:  DNS
  statistics                         |  Estimated Difficulty:  8? (depends
                   Keywords:         |  on #2135)
            Defect Severity:  N/A    |           Total Hours:  0
Feature Depending on Ticket:         |
        Add Hours to Ticket:  0      |
                  Internal?:  0      |
-------------------------------------+-------------------------------------

Comment (by naokikambe):

 Replying to [comment:6 jinmei]:
 > - This sounds awkward to me:
 > {{{
 > +      and so on. <command>b10-stats</command> periodically requests
 statistics
 > +      data to each module and receives. The interval time can be
 configured
 > }}}
 >   Maybe something like "b10-stats periodically requests each module to
 >   return statistic data"?
 >
 That is OK for me. Thank you.

 > On the main changes: from a quick look it makes quite a lot of
 > substantial changes and doesn't seem to be easy to understand the
 > overall idea.  I'm afraid I myself have enough time to complete the
 > review due to my absence next week, so if the actual reviewer is okay
 > with that, that's fine, but if I were a reviewer, I'd like to see some
 > guideline on how to review it.  Ideally, it's better if the entire
 > diff can be staged, so the review can be done step-by-step.

 Thank you for the suggestion and sorry for so huge diffs.
 I categorized these diffs into 6 groups. Most of changes are for
 refactoring or for applying test codes for the new specification. I'd like
 to ask a reviewer to review the changes rather than these changes,
 especially top 3 groups. And then I'd like to ask to review partially test
 codes corresponding to the changes of the main code if it's allowed. This
 can be helpful for the reviewer?

 * Adding a new config 'poll-interval'
 {{{
 a6b0955 [2136] introduced a new configration item 'poll-interval'
 97a53cf [2136] added config setup
 397d4b7 [2136] added update of 'poll-interval'
 }}}

 * Implementing polling of the stats module for collecting statistics data
 {{{
 613c10d [2136] updated stats.py and added a related testcase
 7f94e12 [2136] revised the debug message of request
 80ac056 [2136] implemented polling to each modules for collecting
 statistics data
 }}}

 * Implementing preservation of statistics data from an inavtive instance
 {{{
 89d79f4 [2136] preserve all statistics data including the inactive
 instances' one while stats module is working
 }}}

 * Removing the obsolete command
 {{{
 1c195bb [2136] removed 'set' command of stats module due to its
 unnecessity
 }}}

 * Documentation
 {{{
 e7739c1 [2136] updated bind10-guide.* due to removing set command from
 stats module
 7e6d71c [2136] some small suggested wording fixes to doc.
 }}}

 * Refactoring or applying test codes for new specification
 {{{
 d2d7cbe [2136] removed unnecessary waits
 4b27d27 [2136] added a test dbfile of b10-config for testing stats with
 the Mock boss
 ef4095e [2136] revised test_util.py miscellaneously
 5980e50 [2136] removed the obsoleted command 'sendstats' from the mock
 modules
 4df681f [2136] renamed 'pid' to 'mid' and change the default value -1 to
 None on the second argument of the the 'update_statistics_data' method
 4a50ef0 [2136] added deeply checking statistics values of each of module
 and instance
 e206cb2 [2136] added a new message id for getting invalid statistics data
 from the module
 6d44728 [2136] do group_sendmsg too all modules then do group_recvmsg in
 do_polling for efficiency
 6c0bc0b [2136] kept consistency in unittest due to the previous changes
 4b300f5 [2136] moved creation of a stats.Stats object in each test case
 8f7ae09 [2136] Updated not to calculate next timing based on current
 timestamp in every polling
 2b1cbf9 [2136] Refactoring unittest and update to evaluate the
 next_polltime value as a integer
 be0d496 [2136] Removed 'trees' argument from 'getstats' command
 6d20444 [2136] Fixed wrong checking: The variable key was assumed to be
 constant.
 c46ccb2 [2136] Fixed unfinished MockAuth objects
 }}}

-- 
Ticket URL: <http://bind10.isc.org/ticket/2136#comment:8>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development