BIND 10 trac2225_statistics, updated. cd917b3660e1d944d749fb2fb19ce9dfb622c39c [2225_statistics] reload the counters module before starting tests
BIND 10 source code commits
bind10-changes at lists.isc.org
Tue Jan 8 05:04:08 UTC 2013
The branch, trac2225_statistics has been updated
discards be7153799cfb4d3f75a7140d19f48fc42a777d8b (commit)
discards 46618002ad19df9514ff135ae42cf64caa2b8662 (commit)
discards 4688552db82b8117a0bcf11c40e506784832ccb1 (commit)
discards 2aa30dca729ac289c2a8bc2353e48183567be497 (commit)
discards bfb02252f885e30c5d24968f59f21308add72599 (commit)
discards db1ad53bb81b1f61d52d28758f08f80ba2b84221 (commit)
discards 289118e3413d0790600aedb1de464ca8765c500c (commit)
discards 1f05684cc6681ccfb9197ccdd2ed10fbfc4dc447 (commit)
discards 868558c3f7286500ff7a9d1569101c5b3f5feca7 (commit)
discards 0d0de6a5c721791fa761d21be5385a69d02a792b (commit)
discards 7ec3307e272fd9a4ebf5f3f3123bda2d8c2cc5ff (commit)
discards 4ae14bdf3b1b19c3023d9025a82c75b09e004819 (commit)
discards 301ccaf97379ccf29d02ca4332ba8d456c0a5391 (commit)
discards 2f7080c8e71f58b5c086e0514caecca46f16bc9f (commit)
discards dbf92ead97b1b74ef9ac12c75a91f38c6258605c (commit)
discards 4b368b5e65ad55d8e22236b354253a1ac317ba19 (commit)
discards 6a4b9582cebc33c3ad6e82343e6d2cd9af7b71fd (commit)
discards d4e066a2e7be1e075c41622cb658cbbec968f450 (commit)
via cd917b3660e1d944d749fb2fb19ce9dfb622c39c (commit)
via cc0c7a0a7b9b4abdb94b7c5a2007334b914218f8 (commit)
via e12dee22bbc97e49c7e5641cf50e39f64e5cbed2 (commit)
via 927c2a94869d76fdbb30018861ba235ddf9389f2 (commit)
via 6443e079d1d0dd3b0de766196a2f9b711bd4f4f5 (commit)
via 78f11bb1db706b793f2b45d94da3c613c2f8ca9e (commit)
via 08d9023ba335f084034ec31cf7e304964b267acb (commit)
via dc7e03f9cca07ef79b5741f4353f570f383dbb9b (commit)
via bc768d296fbd5b86b9d100672792605c79c81739 (commit)
via a9f24702174e022332b528b8d39a21c203b67d3e (commit)
via ef7776ea48e1ddbfb3b727836722cbfc8b65f4fa (commit)
via f098bc21fd0c528fdd55bdbd8f99f6fec36c9222 (commit)
via 607d55c17a1d1becedf6e0fb81673627cab763f8 (commit)
via 903087f3c9e778e762483cf11992e4f5ddc9b7d1 (commit)
via 65bbcd0970063c0923f597ee4413ccbcc969ba58 (commit)
via e8ed9fd77960f2e3c4cfc5948d356b47524d441c (commit)
via 31b16f3568b95073fc15b68e435215572f25c5b8 (commit)
via 6fe2c80c65634db3c5b9ef7fab2e2f48a507fc94 (commit)
via 87ff7feca9801dc6c78a5553c090819a66afe918 (commit)
via 48974a8f8351415321dcabf039296e9e7854fbe1 (commit)
via 652859d50201a5fd7d7ec3bc4266ead82fc86c4b (commit)
via aa70659de52c486695a6fc822b673adf51a2abcf (commit)
via c9e1f35632323f09a1e61f96f88df384e337f885 (commit)
via ad322965f290c5d594e084eb57f9bf16eff87717 (commit)
via ad6e16d25942d9bb308fcc1cf4ec839769956981 (commit)
via 2b861742dc6b8461984a1c1c0437b46a313f672b (commit)
This update added new revisions after undoing existing revisions. That is
to say, the old revision is not a strict subset of the new revision. This
situation occurs when you --force push a change and generate a repository
containing something like this:
* -- * -- B -- O -- O -- O (be7153799cfb4d3f75a7140d19f48fc42a777d8b)
\
N -- N -- N (cd917b3660e1d944d749fb2fb19ce9dfb622c39c)
When this happens we assume that you've already had alert emails for all
of the O revisions, and so we here report only the revisions in the N
branch from the common base, B.
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit cd917b3660e1d944d749fb2fb19ce9dfb622c39c
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Tue Jan 8 13:27:14 2013 +0900
[2225_statistics] reload the counters module before starting tests
commit cc0c7a0a7b9b4abdb94b7c5a2007334b914218f8
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Tue Jan 8 13:23:41 2013 +0900
[2225_statistics] add multi-module tests for the scenarios
The following two scenarios are intended here:
- The counters are incremented in the scenario, which notify_out is solely
invoked and default statistics spec is used.
- The counters are incremented in the scenario, which XfroutServer is invoked
and the same spec file is loaded in several times
commit e12dee22bbc97e49c7e5641cf50e39f64e5cbed2
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Mon Jan 7 21:41:34 2013 +0900
[2225_statistics] correct the module name according to renaming to counters.py
commit 927c2a94869d76fdbb30018861ba235ddf9389f2
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Mon Jan 7 20:40:58 2013 +0900
[2225_statistics] simplify too detailed documentation in start_timer() and stop_timer()
commit 6443e079d1d0dd3b0de766196a2f9b711bd4f4f5
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Mon Jan 7 20:19:13 2013 +0900
[2225_statistics] correct documentation of inc() and dec()
commit 78f11bb1db706b793f2b45d94da3c613c2f8ca9e
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Mon Jan 7 20:17:14 2013 +0900
[2225_statistics] add a common helper function for incrementing or decrementing a counter
also inc() and dec() share its implementation.
commit 08d9023ba335f084034ec31cf7e304964b267acb
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Mon Jan 7 20:03:20 2013 +0900
[2225_statistics] add note to _concat()
commit dc7e03f9cca07ef79b5741f4353f570f383dbb9b
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Mon Jan 7 19:52:29 2013 +0900
[2225_statistics] rename counter.py to counters.py and make the related changes for the review comment
commit bc768d296fbd5b86b9d100672792605c79c81739
Author: Mukund Sivaraman <muks at isc.org>
Date: Wed Jan 2 15:58:01 2013 +0530
[2225_statistics] Make various documentation updates
These fix typos, grammar and also rewrite some comments so that they are
more clear to the reader.
commit a9f24702174e022332b528b8d39a21c203b67d3e
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 18:44:08 2012 +0900
[2225_statistics] rename dump_statistics() to get_statistics()
It's due to the review comment.
commit ef7776ea48e1ddbfb3b727836722cbfc8b65f4fa
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 18:40:27 2012 +0900
[2225_statistics] update document as it is
It's due to the review comment.
commit f098bc21fd0c528fdd55bdbd8f99f6fec36c9222
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 18:29:02 2012 +0900
[2225_statistics] add helpful comments to the complicated expression
Also rsplit() is used instead of split() to be simplified.
It's due to the review comment.
commit 607d55c17a1d1becedf6e0fb81673627cab763f8
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 17:54:51 2012 +0900
[2225_statistics] rename to clear_all()
It's due to the review comment.
commit 903087f3c9e778e762483cf11992e4f5ddc9b7d1
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 17:49:00 2012 +0900
[2225_statistics] rename start()/stop() to start_timer()/stop_timer()
Due to the review comment.
commit 65bbcd0970063c0923f597ee4413ccbcc969ba58
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 17:45:47 2012 +0900
[2225_statistics] remove 'else' statement
In stead of being removed, it just returns if DataNotFoundError is raised.
Due to the review comment.
commit e8ed9fd77960f2e3c4cfc5948d356b47524d441c
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 17:42:29 2012 +0900
[2225_statistics] update document of Counters.stop()
explicitly distinguishing duration time from date time.
Due to the review comment.
commit 31b16f3568b95073fc15b68e435215572f25c5b8
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 17:25:40 2012 +0900
[2225_statistics] use timedelta.total_seconds() instead of redundant computation
Due to the review comment.
commit 6fe2c80c65634db3c5b9ef7fab2e2f48a507fc94
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 17:15:40 2012 +0900
[2225_statistics] remove an unnecessary formating
Due to the review comment.
commit 87ff7feca9801dc6c78a5553c090819a66afe918
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 17:12:20 2012 +0900
[2225_statistics] remove an unnecessary try/expect statement
Due to the review comment.
commit 48974a8f8351415321dcabf039296e9e7854fbe1
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 17:31:31 2012 +0900
[2225_statistics] add step as kwarg into inc() and use it in dec() with step=-1
Due to the review comment.
commit 652859d50201a5fd7d7ec3bc4266ead82fc86c4b
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 17:34:04 2012 +0900
[2225_statistics] add a helper method _concat and its test
And replace it with '/'.join(args)
Due to the review comment.
commit aa70659de52c486695a6fc822b673adf51a2abcf
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 16:32:58 2012 +0900
[2225_statistics] synchronize when accessing self._disabled
And timer cannot be updated if self._disabled is true.
Due to the review comment.
commit c9e1f35632323f09a1e61f96f88df384e337f885
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 16:09:56 2012 +0900
[2225_statistics] document about cases when a spec file is specified and omitted
Due to the review comment.
commit ad322965f290c5d594e084eb57f9bf16eff87717
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 16:01:25 2012 +0900
[2225_statistics] correct and add documentation of the Counters class
Due to the review comment.
commit ad6e16d25942d9bb308fcc1cf4ec839769956981
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 15:39:13 2012 +0900
[2225_statistics] correct documentation about an undefined method
Due to the review comment.
commit 2b861742dc6b8461984a1c1c0437b46a313f672b
Author: Naoki Kambe <kambe at jprs.co.jp>
Date: Fri Dec 14 13:41:57 2012 +0900
[2225_statistics] rename the class to the plural form (`Counters`) as it manages multiple counters
Due to the review comment.
-----------------------------------------------------------------------
Summary of changes:
src/lib/python/isc/statistics/__init__.py | 2 +-
.../isc/statistics/{counter.py => counters.py} | 51 ++++---
src/lib/python/isc/statistics/tests/Makefile.am | 2 +-
.../tests/{counter_test.py => counters_test.py} | 156 +++++++++++++++++---
4 files changed, 168 insertions(+), 43 deletions(-)
rename src/lib/python/isc/statistics/{counter.py => counters.py} (91%)
rename src/lib/python/isc/statistics/tests/{counter_test.py => counters_test.py} (69%)
-----------------------------------------------------------------------
diff --git a/src/lib/python/isc/statistics/__init__.py b/src/lib/python/isc/statistics/__init__.py
index 2daa4b7..9e77ed6 100644
--- a/src/lib/python/isc/statistics/__init__.py
+++ b/src/lib/python/isc/statistics/__init__.py
@@ -1 +1 @@
-from isc.statistics.counter import *
+from isc.statistics.counters import *
diff --git a/src/lib/python/isc/statistics/counter.py b/src/lib/python/isc/statistics/counter.py
deleted file mode 100644
index 9d402d5..0000000
--- a/src/lib/python/isc/statistics/counter.py
+++ /dev/null
@@ -1,394 +0,0 @@
-# Copyright (C) 2012 Internet Systems Consortium.
-#
-# Permission to use, copy, modify, and distribute this software for any
-# purpose with or without fee is hereby granted, provided that the above
-# copyright notice and this permission notice appear in all copies.
-#
-# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
-# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
-# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
-# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
-# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
-# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
-# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-
-"""BIND 10 statistics counters module
-
-This module handles the statistics counters for BIND 10 modules. For
-using the module `counter.py`, first a counters object should be created
-in each module (like b10-xfrin or b10-xfrout) after importing this
-module. A spec file can be specified as an argument when creating the
-counters object:
-
- from isc.statistics import Counters
- self.counters = Counters("/path/to/foo.spec")
-
-The first argument of Counters() can be specified, which is the location
-of the specification file (like src/bin/xfrout/xfrout.spec). If Counters
-is constructed this way, statistics counters can be accessed from each
-module. For example, in case that the item `xfrreqdone` is defined in
-statistics_spec in xfrout.spec, the following methods are
-callable. Since these methods require the string of the zone name in the
-first argument, if we have the following code in b10-xfrout:
-
- self.counters.inc('zones', zone_name, 'xfrreqdone')
-
-then the counter for xfrreqdone corresponding to zone_name is
-incremented. For getting the current number of this counter, we can use
-the following code:
-
- number = self.counters.get('zones', zone_name, 'xfrreqdone')
-
-then the current count is obtained and set in the variable
-`number`. Such a getter method would be mainly used for unit-testing.
-As other example, for the item `axfr_running`, the decrementer method is
-also callable. This method is used for decrementing a counter. For the
-item `axfr_running`, an argument like zone name is not required:
-
- self.counters.dec('axfr_running')
-
-These methods are effective in other modules. For example, in case that
-this module `counter.py` is once imported in a main module such as
-b10-xfrout, then for the item `notifyoutv4`, the `inc()` method can be
-invoked in another module such as notify_out.py, which is firstly
-imported in the main module.
-
- self.counters.inc('zones', zone_name, 'notifyoutv4')
-
-In this example this is for incrementing the counter of the item
-`notifyoutv4`. Thus, such statement can be also written in another
-library like isc.notify.notify_out. If this module `counter.py` isn't
-imported in the main module but imported in such a library module as
-isc.notify.notify_out, in this example, empty methods would be invoked,
-which is directly defined in `counter.py`.
-"""
-
-import threading
-import isc.config
-from datetime import datetime
-
-# static internal functions
-def _add_counter(element, spec, identifier):
- """Returns value of the identifier if the identifier is in the
- element. Otherwise, sets a default value from the spec and
- returns it. If the top-level type of the identifier is named_set
- and the second-level type is map, it sets a set of default values
- under the level and then returns the default value of the
- identifier. This method raises DataNotFoundError if the element is
- invalid for spec."""
- try:
- return isc.cc.data.find(element, identifier)
- except isc.cc.data.DataNotFoundError:
- pass
- # check whether spec and identifier are correct
- isc.config.find_spec_part(spec, identifier)
- # examine spec of the top-level item first
- spec_ = isc.config.find_spec_part(spec, identifier.split('/')[0])
- if spec_['item_type'] == 'named_set' and \
- spec_['named_set_item_spec']['item_type'] == 'map':
- map_spec = spec_['named_set_item_spec']['map_item_spec']
- for name in isc.config.spec_name_list(map_spec):
- spec_ = isc.config.find_spec_part(map_spec, name)
- id_str = '%s/%s/%s' % \
- tuple(identifier.split('/')[0:2] + [name])
- isc.cc.data.set(element, id_str, spec_['item_default'])
- else:
- spec_ = isc.config.find_spec_part(spec, identifier)
- isc.cc.data.set(element, identifier, spec_['item_default'])
- return isc.cc.data.find(element, identifier)
-
-def _set_counter(element, spec, identifier, value):
- """Invokes _add_counter() for checking whether the identifier is
- in the element. If not, it creates a new identifier in the element
- and set the default value from the spec. After that, it sets the
- value specified in the arguments."""
- _add_counter(element, spec, identifier)
- isc.cc.data.set(element, identifier, value)
-
-def _get_counter(element, identifier):
- """Returns the value of the identifier in the element"""
- return isc.cc.data.find(element, identifier)
-
-def _inc_counter(element, spec, identifier, step=1):
- """Increments the value of the identifier in the element to the
- step from the current value. If the identifier isn't in the
- element, it creates a new identifier in the element."""
- isc.cc.data.set(element, identifier,
- _add_counter(element, spec, identifier) + step)
-
-def _start_timer():
- """Returns the current datetime as a datetime object."""
- return datetime.now()
-
-def _stop_timer(start_time, element, spec, identifier):
- """Sets duration time in seconds as a value of the identifier in
- the element, which is in seconds between start_time and the
- current time and is float-type."""
- delta = datetime.now() - start_time
- sec = delta.total_seconds()
- _set_counter(element, spec, identifier, sec)
-
-def _concat(*args, sep='/'):
- """Concatenates words in args with a separator('/')
- """
- return sep.join(args)
-
-class _Statistics():
- """Statistics data set"""
- # default statistics data
- _data = {}
- # default statistics spec used in case the specfile is omitted when
- # constructing a Counters() object
- _spec = [
- {
- "item_name": "zones",
- "item_type": "named_set",
- "item_optional": False,
- "item_default": {
- "_SERVER_" : {
- "notifyoutv4" : 0,
- "notifyoutv6" : 0
- }
- },
- "item_title": "Zone names",
- "item_description": "Zone names",
- "named_set_item_spec": {
- "item_name": "zonename",
- "item_type": "map",
- "item_optional": False,
- "item_default": {},
- "item_title": "Zone name",
- "item_description": "Zone name",
- "map_item_spec": [
- {
- "item_name": "notifyoutv4",
- "item_type": "integer",
- "item_optional": False,
- "item_default": 0,
- "item_title": "IPv4 notifies",
- "item_description": "Number of IPv4 notifies per zone name sent out"
- },
- {
- "item_name": "notifyoutv6",
- "item_type": "integer",
- "item_optional": False,
- "item_default": 0,
- "item_title": "IPv6 notifies",
- "item_description": "Number of IPv6 notifies per zone name sent out"
- }
- ]
- }
- }
- ]
-
-class Counters():
- """A class for holding and manipulating all statistics counters
- for a module. A Counters object may be created by specifying a spec
- file of the module in argument. According to statistics
- specification in the spec file, a counter value can be incremented,
- decremented or obtained. Methods such as inc(), dec() and get() are
- useful for this. Counters objects also have timer functionality.
- The timer can be started and stopped, and the duration between
- start and stop can be obtained. Methods such as start_timer(),
- stop_timer() and get() are useful for this. Saved counters can be
- cleared by the method clear_all(). Manipulating counters and
- timers can be temporarily disabled. If disabled, counter values are
- not changed even if methods to update them are invoked. Including
- per-zone counters, a list of counters which can be handled in the
- class are like the following:
-
- zones/example.com./notifyoutv4
- zones/example.com./notifyoutv6
- zones/example.com./xfrrej
- zones/example.com./xfrreqdone
- zones/example.com./soaoutv4
- zones/example.com./soaoutv6
- zones/example.com./axfrreqv4
- zones/example.com./axfrreqv6
- zones/example.com./ixfrreqv4
- zones/example.com./ixfrreqv6
- zones/example.com./xfrsuccess
- zones/example.com./xfrfail
- zones/example.com./time_to_ixfr
- zones/example.com./time_to_axfr
- ixfr_running
- axfr_running
- socket/unixdomain/open
- socket/unixdomain/openfail
- socket/unixdomain/close
- socket/unixdomain/bindfail
- socket/unixdomain/acceptfail
- socket/unixdomain/accept
- socket/unixdomain/senderr
- socket/unixdomain/recverr
- socket/ipv4/tcp/open
- socket/ipv4/tcp/openfail
- socket/ipv4/tcp/close
- socket/ipv4/tcp/connfail
- socket/ipv4/tcp/conn
- socket/ipv4/tcp/senderr
- socket/ipv4/tcp/recverr
- socket/ipv6/tcp/open
- socket/ipv6/tcp/openfail
- socket/ipv6/tcp/close
- socket/ipv6/tcp/connfail
- socket/ipv6/tcp/conn
- socket/ipv6/tcp/senderr
- socket/ipv6/tcp/recverr
- """
-
- # '_SERVER_' is a special zone name representing an entire
- # count. It doesn't mean a specific zone, but it means an
- # entire count in the server.
- _entire_server = '_SERVER_'
- # zone names are contained under this dirname in the spec file.
- _perzone_prefix = 'zones'
- # default statistics data set
- _statistics = _Statistics()
-
- def __init__(self, spec_file_name=None):
- """A constructor for the Counters class. A path to the spec file
- can be specified in spec_file_name. Statistics data based on
- statistics spec can be accumulated if spec_file_name is
- specified. If omitted, a default statistics spec is used. The
- default statistics spec is defined in a hidden class named
- _Statistics().
- """
- self._zones_item_list = []
- self._start_time = {}
- self._disabled = False
- self._rlock = threading.RLock()
- if not spec_file_name: return
- # change the default statistics spec
- self._statistics._spec = \
- isc.config.module_spec_from_file(spec_file_name).\
- get_statistics_spec()
- if self._perzone_prefix in \
- isc.config.spec_name_list(self._statistics._spec):
- self._zones_item_list = isc.config.spec_name_list(
- isc.config.find_spec_part(
- self._statistics._spec, self._perzone_prefix)\
- ['named_set_item_spec']['map_item_spec'])
-
- def clear_all(self):
- """clears all statistics data"""
- with self._rlock:
- self._statistics._data = {}
-
- def disable(self):
- """disables incrementing/decrementing counters"""
- with self._rlock:
- self._disabled = True
-
- def enable(self):
- """enables incrementing/decrementing counters"""
- with self._rlock:
- self._disabled = False
-
- def inc(self, *args, step=1):
- """A incrementer for per-zone counter. Locks the thread
- because it is considered to be invoked by a multi-threading
- caller. isc.cc.data.DataNotFoundError is raised when
- incrementing the counter of the item undefined in the spec
- file. step must not be specified by the caller."""
- identifier = _concat(*args)
- with self._rlock:
- if self._disabled: return
- _inc_counter(self._statistics._data,
- self._statistics._spec,
- identifier, step)
-
- def dec(self, *args, step=-1):
- """A decrementer for axfr or ixfr running. Locks the thread
- because it is considered to be invoked by a multi-threading
- caller. isc.cc.data.DataNotFoundError is raised when
- decrementing the counter of the item undefined in the spec
- file. step must not be specified by the caller."""
- self.inc(*args, step=step)
-
- def get(self, *args):
- """A getter method for counters. It returns the current number
- of the specified counter. isc.cc.data.DataNotFoundError is
- raised when the counter doesn't have a number yet."""
- identifier = _concat(*args)
- return _get_counter(self._statistics._data, identifier)
-
- def start_timer(self, *args):
- """Sets the value returned from _start_timer() as a value of
- the identifier in the self._start_time which is dict-type"""
- identifier = _concat(*args)
- with self._rlock:
- if self._disabled: return
- isc.cc.data.set(self._start_time, identifier, _start_timer())
-
- def stop_timer(self, *args):
- """Sets duration time in seconds between corresponding date
- time in self._start_time and current date time into the value
- of the identifier. It deletes corresponding time in
- self._start_time after setting is successfully done. In case
- of stopping the timer which has never been started, it raises
- and does nothing. But in case of stopping the time which isn't
- defined in the spec file, it raises DataNotFoundError"""
- identifier = _concat(*args)
- with self._rlock:
- if self._disabled: return
- try:
- start_time = isc.cc.data.find(self._start_time,
- identifier)
- except isc.cc.data.DataNotFoundError:
- # do not set the end time if the timer isn't started
- return
- # set the end time
- _stop_timer(
- start_time,
- self._statistics._data,
- self._statistics._spec,
- identifier)
- # A datetime value of once used timer should be deleted
- # for a future use.
- # Here, names of branch and leaf are obtained from a
- # string of identifier. The branch name is equivalent to
- # the position of datetime to be deleted and the leaf name
- # is equivalent to the value of datetime to be deleted.
- (branch, leaf) = identifier.rsplit('/', 1)
- # Then map of branch is obtained from self._start_time by
- # using isc.cc.data.find().
- branch_map = isc.cc.data.find(self._start_time, branch)
- # Finally a value of the leaf name is deleted from the
- # map.
- del branch_map[leaf]
-
- def get_statistics(self):
- """Calculates an entire server's counts, and returns statistics
- data in a format to send out to the stats module, including each
- counter. If nothing is counted yet, then it returns an empty
- dictionary."""
- # entire copy
- statistics_data = self._statistics._data.copy()
- # If there is no 'zones' found in statistics_data,
- # i.e. statistics_data contains no per-zone counter, it just
- # returns statistics_data because calculating total counts
- # across the zone names isn't necessary.
- if self._perzone_prefix not in statistics_data:
- return statistics_data
- zones = statistics_data[self._perzone_prefix]
- # Start calculation for '_SERVER_' counts
- zones_spec = isc.config.find_spec_part(self._statistics._spec,
- self._perzone_prefix)
- zones_attrs = zones_spec['item_default'][self._entire_server]
- zones_data = {}
- for attr in zones_attrs:
- id_str = '%s/%s' % (self._entire_server, attr)
- sum_ = 0
- for name in zones:
- if attr in zones[name]:
- sum_ += zones[name][attr]
- if sum_ > 0:
- _set_counter(zones_data, zones_spec,
- id_str, sum_)
- # insert entire-server counts
- statistics_data[self._perzone_prefix] = dict(
- statistics_data[self._perzone_prefix],
- **zones_data)
- return statistics_data
diff --git a/src/lib/python/isc/statistics/counters.py b/src/lib/python/isc/statistics/counters.py
new file mode 100644
index 0000000..6eb3dc9
--- /dev/null
+++ b/src/lib/python/isc/statistics/counters.py
@@ -0,0 +1,407 @@
+# Copyright (C) 2012 Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+"""BIND 10 statistics counters module
+
+This module handles the statistics counters for BIND 10 modules. For
+using the module `counter.py`, first a counters object should be created
+in each module (like b10-xfrin or b10-xfrout) after importing this
+module. A spec file can be specified as an argument when creating the
+counters object:
+
+ from isc.statistics import Counters
+ self.counters = Counters("/path/to/foo.spec")
+
+The first argument of Counters() can be specified, which is the location
+of the specification file (like src/bin/xfrout/xfrout.spec). If Counters
+is constructed this way, statistics counters can be accessed from each
+module. For example, in case that the item `xfrreqdone` is defined in
+statistics_spec in xfrout.spec, the following methods are
+callable. Since these methods require the string of the zone name in the
+first argument, if we have the following code in b10-xfrout:
+
+ self.counters.inc('zones', zone_name, 'xfrreqdone')
+
+then the counter for xfrreqdone corresponding to zone_name is
+incremented. For getting the current number of this counter, we can use
+the following code:
+
+ number = self.counters.get('zones', zone_name, 'xfrreqdone')
+
+then the current count is obtained and set in the variable
+`number`. Such a getter method would be mainly used for unit-testing.
+As other example, for the item `axfr_running`, the decrementer method is
+also callable. This method is used for decrementing a counter. For the
+item `axfr_running`, an argument like zone name is not required:
+
+ self.counters.dec('axfr_running')
+
+These methods are effective in other modules. For example, in case that
+this module `counter.py` is once imported in a main module such as
+b10-xfrout, then for the item `notifyoutv4`, the `inc()` method can be
+invoked in another module such as notify_out.py, which is firstly
+imported in the main module.
+
+ self.counters.inc('zones', zone_name, 'notifyoutv4')
+
+In this example this is for incrementing the counter of the item
+`notifyoutv4`. Thus, such statement can be also written in another
+library like isc.notify.notify_out. If this module `counter.py` isn't
+imported in the main module but imported in such a library module as
+isc.notify.notify_out, in this example, empty methods would be invoked,
+which is directly defined in `counter.py`.
+"""
+
+import threading
+import isc.config
+from datetime import datetime
+
+# static internal functions
+def _add_counter(element, spec, identifier):
+ """Returns value of the identifier if the identifier is in the
+ element. Otherwise, sets a default value from the spec and
+ returns it. If the top-level type of the identifier is named_set
+ and the second-level type is map, it sets a set of default values
+ under the level and then returns the default value of the
+ identifier. This method raises DataNotFoundError if the element is
+ invalid for spec."""
+ try:
+ return isc.cc.data.find(element, identifier)
+ except isc.cc.data.DataNotFoundError:
+ pass
+ # check whether spec and identifier are correct
+ isc.config.find_spec_part(spec, identifier)
+ # examine spec of the top-level item first
+ spec_ = isc.config.find_spec_part(spec, identifier.split('/')[0])
+ if spec_['item_type'] == 'named_set' and \
+ spec_['named_set_item_spec']['item_type'] == 'map':
+ map_spec = spec_['named_set_item_spec']['map_item_spec']
+ for name in isc.config.spec_name_list(map_spec):
+ spec_ = isc.config.find_spec_part(map_spec, name)
+ id_str = '%s/%s/%s' % \
+ tuple(identifier.split('/')[0:2] + [name])
+ isc.cc.data.set(element, id_str, spec_['item_default'])
+ else:
+ spec_ = isc.config.find_spec_part(spec, identifier)
+ isc.cc.data.set(element, identifier, spec_['item_default'])
+ return isc.cc.data.find(element, identifier)
+
+def _set_counter(element, spec, identifier, value):
+ """Invokes _add_counter() for checking whether the identifier is
+ in the element. If not, it creates a new identifier in the element
+ and set the default value from the spec. After that, it sets the
+ value specified in the arguments."""
+ _add_counter(element, spec, identifier)
+ isc.cc.data.set(element, identifier, value)
+
+def _get_counter(element, identifier):
+ """Returns the value of the identifier in the element"""
+ return isc.cc.data.find(element, identifier)
+
+def _inc_counter(element, spec, identifier, step=1):
+ """Increments the value of the identifier in the element to the
+ step from the current value. If the identifier isn't in the
+ element, it creates a new identifier in the element."""
+ isc.cc.data.set(element, identifier,
+ _add_counter(element, spec, identifier) + step)
+
+def _start_timer():
+ """Returns the current datetime as a datetime object."""
+ return datetime.now()
+
+def _stop_timer(start_time, element, spec, identifier):
+ """Sets duration time in seconds as a value of the identifier in
+ the element, which is in seconds between start_time and the
+ current time and is float-type."""
+ delta = datetime.now() - start_time
+ sec = delta.total_seconds()
+ _set_counter(element, spec, identifier, sec)
+
+def _concat(*args, sep='/'):
+ """A helper function that is used to generate an identifier for
+ statistics item names. It concatenates words in args with a
+ separator('/')
+ """
+ return sep.join(args)
+
+class _Statistics():
+ """Statistics data set"""
+ # default statistics data
+ _data = {}
+ # default statistics spec used in case the specfile is omitted when
+ # constructing a Counters() object
+ _spec = [
+ {
+ "item_name": "zones",
+ "item_type": "named_set",
+ "item_optional": False,
+ "item_default": {
+ "_SERVER_" : {
+ "notifyoutv4" : 0,
+ "notifyoutv6" : 0
+ }
+ },
+ "item_title": "Zone names",
+ "item_description": "Zone names",
+ "named_set_item_spec": {
+ "item_name": "zonename",
+ "item_type": "map",
+ "item_optional": False,
+ "item_default": {},
+ "item_title": "Zone name",
+ "item_description": "Zone name",
+ "map_item_spec": [
+ {
+ "item_name": "notifyoutv4",
+ "item_type": "integer",
+ "item_optional": False,
+ "item_default": 0,
+ "item_title": "IPv4 notifies",
+ "item_description": "Number of IPv4 notifies per zone name sent out"
+ },
+ {
+ "item_name": "notifyoutv6",
+ "item_type": "integer",
+ "item_optional": False,
+ "item_default": 0,
+ "item_title": "IPv6 notifies",
+ "item_description": "Number of IPv6 notifies per zone name sent out"
+ }
+ ]
+ }
+ }
+ ]
+
+class Counters():
+ """A class for holding and manipulating all statistics counters
+ for a module. A Counters object may be created by specifying a spec
+ file of the module in argument. According to statistics
+ specification in the spec file, a counter value can be incremented,
+ decremented or obtained. Methods such as inc(), dec() and get() are
+ useful for this. Counters objects also have timer functionality.
+ The timer can be started and stopped, and the duration between
+ start and stop can be obtained. Methods such as start_timer(),
+ stop_timer() and get() are useful for this. Saved counters can be
+ cleared by the method clear_all(). Manipulating counters and
+ timers can be temporarily disabled. If disabled, counter values are
+ not changed even if methods to update them are invoked. Including
+ per-zone counters, a list of counters which can be handled in the
+ class are like the following:
+
+ zones/example.com./notifyoutv4
+ zones/example.com./notifyoutv6
+ zones/example.com./xfrrej
+ zones/example.com./xfrreqdone
+ zones/example.com./soaoutv4
+ zones/example.com./soaoutv6
+ zones/example.com./axfrreqv4
+ zones/example.com./axfrreqv6
+ zones/example.com./ixfrreqv4
+ zones/example.com./ixfrreqv6
+ zones/example.com./xfrsuccess
+ zones/example.com./xfrfail
+ zones/example.com./time_to_ixfr
+ zones/example.com./time_to_axfr
+ ixfr_running
+ axfr_running
+ socket/unixdomain/open
+ socket/unixdomain/openfail
+ socket/unixdomain/close
+ socket/unixdomain/bindfail
+ socket/unixdomain/acceptfail
+ socket/unixdomain/accept
+ socket/unixdomain/senderr
+ socket/unixdomain/recverr
+ socket/ipv4/tcp/open
+ socket/ipv4/tcp/openfail
+ socket/ipv4/tcp/close
+ socket/ipv4/tcp/connfail
+ socket/ipv4/tcp/conn
+ socket/ipv4/tcp/senderr
+ socket/ipv4/tcp/recverr
+ socket/ipv6/tcp/open
+ socket/ipv6/tcp/openfail
+ socket/ipv6/tcp/close
+ socket/ipv6/tcp/connfail
+ socket/ipv6/tcp/conn
+ socket/ipv6/tcp/senderr
+ socket/ipv6/tcp/recverr
+ """
+
+ # '_SERVER_' is a special zone name representing an entire
+ # count. It doesn't mean a specific zone, but it means an
+ # entire count in the server.
+ _entire_server = '_SERVER_'
+ # zone names are contained under this dirname in the spec file.
+ _perzone_prefix = 'zones'
+ # default statistics data set
+ _statistics = _Statistics()
+
+ def __init__(self, spec_file_name=None):
+ """A constructor for the Counters class. A path to the spec file
+ can be specified in spec_file_name. Statistics data based on
+ statistics spec can be accumulated if spec_file_name is
+ specified. If omitted, a default statistics spec is used. The
+ default statistics spec is defined in a hidden class named
+ _Statistics().
+ """
+ self._zones_item_list = []
+ self._start_time = {}
+ self._disabled = False
+ self._rlock = threading.RLock()
+ if not spec_file_name: return
+ # change the default statistics spec
+ self._statistics._spec = \
+ isc.config.module_spec_from_file(spec_file_name).\
+ get_statistics_spec()
+ if self._perzone_prefix in \
+ isc.config.spec_name_list(self._statistics._spec):
+ self._zones_item_list = isc.config.spec_name_list(
+ isc.config.find_spec_part(
+ self._statistics._spec, self._perzone_prefix)\
+ ['named_set_item_spec']['map_item_spec'])
+
+ def clear_all(self):
+ """clears all statistics data"""
+ with self._rlock:
+ self._statistics._data = {}
+
+ def disable(self):
+ """disables incrementing/decrementing counters"""
+ with self._rlock:
+ self._disabled = True
+
+ def enable(self):
+ """enables incrementing/decrementing counters"""
+ with self._rlock:
+ self._disabled = False
+
+ def _incdec(self, *args, step=1):
+ """A common helper function for incrementing or decrementing a
+ counter. It locks the thread because it is considered to be
+ invoked by a multi-threading
+ caller. isc.cc.data.DataNotFoundError is raised when
+ incrementing the counter of the item undefined in the spec
+ file."""
+ identifier = _concat(*args)
+ with self._rlock:
+ if self._disabled: return
+ _inc_counter(self._statistics._data,
+ self._statistics._spec,
+ identifier, step)
+
+ def inc(self, *args):
+ """A incrementer for a counter. Locks the thread because it is
+ considered to be invoked by a multi-threading
+ caller. isc.cc.data.DataNotFoundError is raised when
+ incrementing the counter of the item undefined in the spec
+ file. It calls internally a common helper function
+ _incdec()"""
+ return self._incdec(*args)
+
+ def dec(self, *args):
+ """A decrementer for a counter. Locks the thread because it is
+ considered to be invoked by a multi-threading
+ caller. isc.cc.data.DataNotFoundError is raised when
+ decrementing the counter of the item undefined in the spec
+ file. It calls internally a common helper function
+ _incdec()"""
+ return self._incdec(*args, step=-1)
+
+ def get(self, *args):
+ """A getter method for counters. It returns the current number
+ of the specified counter. isc.cc.data.DataNotFoundError is
+ raised when the counter doesn't have a number yet."""
+ identifier = _concat(*args)
+ return _get_counter(self._statistics._data, identifier)
+
+ def start_timer(self, *args):
+ """Starts a timer which is identified by args and keeps it
+ running until stop_timer() is called. It's also considered to
+ be called from multi-threads."""
+ identifier = _concat(*args)
+ with self._rlock:
+ if self._disabled: return
+ isc.cc.data.set(self._start_time, identifier, _start_timer())
+
+ def stop_timer(self, *args):
+ """Stops a timer which is identified by args. It's also
+ considered to be called from multi-threads. If the timer
+ isn't started by start_timer() yet, it raises no
+ exception. However if args aren't defined in the spec file, it
+ raises DataNotFoundError.
+ """
+ identifier = _concat(*args)
+ with self._rlock:
+ if self._disabled: return
+ try:
+ start_time = isc.cc.data.find(self._start_time,
+ identifier)
+ except isc.cc.data.DataNotFoundError:
+ # do not set the end time if the timer isn't started
+ return
+ # set the end time
+ _stop_timer(
+ start_time,
+ self._statistics._data,
+ self._statistics._spec,
+ identifier)
+ # A datetime value of once used timer should be deleted
+ # for a future use.
+ # Here, names of branch and leaf are obtained from a
+ # string of identifier. The branch name is equivalent to
+ # the position of datetime to be deleted and the leaf name
+ # is equivalent to the value of datetime to be deleted.
+ (branch, leaf) = identifier.rsplit('/', 1)
+ # Then map of branch is obtained from self._start_time by
+ # using isc.cc.data.find().
+ branch_map = isc.cc.data.find(self._start_time, branch)
+ # Finally a value of the leaf name is deleted from the
+ # map.
+ del branch_map[leaf]
+
+ def get_statistics(self):
+ """Calculates an entire server's counts, and returns statistics
+ data in a format to send out to the stats module, including each
+ counter. If nothing is counted yet, then it returns an empty
+ dictionary."""
+ # entire copy
+ statistics_data = self._statistics._data.copy()
+ # If there is no 'zones' found in statistics_data,
+ # i.e. statistics_data contains no per-zone counter, it just
+ # returns statistics_data because calculating total counts
+ # across the zone names isn't necessary.
+ if self._perzone_prefix not in statistics_data:
+ return statistics_data
+ zones = statistics_data[self._perzone_prefix]
+ # Start calculation for '_SERVER_' counts
+ zones_spec = isc.config.find_spec_part(self._statistics._spec,
+ self._perzone_prefix)
+ zones_attrs = zones_spec['item_default'][self._entire_server]
+ zones_data = {}
+ for attr in zones_attrs:
+ id_str = '%s/%s' % (self._entire_server, attr)
+ sum_ = 0
+ for name in zones:
+ if attr in zones[name]:
+ sum_ += zones[name][attr]
+ if sum_ > 0:
+ _set_counter(zones_data, zones_spec,
+ id_str, sum_)
+ # insert entire-server counts
+ statistics_data[self._perzone_prefix] = dict(
+ statistics_data[self._perzone_prefix],
+ **zones_data)
+ return statistics_data
diff --git a/src/lib/python/isc/statistics/tests/Makefile.am b/src/lib/python/isc/statistics/tests/Makefile.am
index 7e2252b..8dcc296 100644
--- a/src/lib/python/isc/statistics/tests/Makefile.am
+++ b/src/lib/python/isc/statistics/tests/Makefile.am
@@ -1,5 +1,5 @@
PYCOVERAGE_RUN=@PYCOVERAGE_RUN@
-PYTESTS = counter_test.py
+PYTESTS = counters_test.py
EXTRA_DIST = $(PYTESTS)
EXTRA_DIST += testdata/test_spec1.spec
EXTRA_DIST += testdata/test_spec2.spec
diff --git a/src/lib/python/isc/statistics/tests/counter_test.py b/src/lib/python/isc/statistics/tests/counter_test.py
deleted file mode 100644
index 63842e5..0000000
--- a/src/lib/python/isc/statistics/tests/counter_test.py
+++ /dev/null
@@ -1,317 +0,0 @@
-# Copyright (C) 2012 Internet Systems Consortium.
-#
-# Permission to use, copy, modify, and distribute this software for any
-# purpose with or without fee is hereby granted, provided that the above
-# copyright notice and this permission notice appear in all copies.
-#
-# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
-# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
-# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
-# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
-# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
-# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
-# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-
-'''Tests for isc.statistics.counter'''
-
-import unittest
-import threading
-from datetime import timedelta
-import os
-import imp
-import isc.config
-
-TEST_ZONE_NAME_STR = "example.com."
-TESTDATA_SRCDIR = os.getenv("TESTDATASRCDIR")
-
-from isc.statistics import counter
-
-def setup_functor(event, cycle, functor, *args):
- """Waits until the event is started, and then invokes the functor
- by times of the cycle with args."""
- event.wait()
- for i in range(cycle): functor(*args)
-
-def start_functor(concurrency, number, functor, *args):
- """Creates the threads of the number and makes them start. Sets
- the event and waits until these threads are finished."""
- threads = []
- event = threading.Event()
- for i in range(concurrency):
- threads.append(threading.Thread(\
- target=setup_functor, \
- args=(event, number, functor,) + args))
- for th in threads: th.start()
- event.set()
- for th in threads: th.join()
-
-class TestBasicMethods(unittest.TestCase):
- TEST_SPECFILE_LOCATION = TESTDATA_SRCDIR + os.sep + 'test_spec1.spec'
-
- def setUp(self):
- self.counters = counter.Counters(self.TEST_SPECFILE_LOCATION)
-
- def tearDown(self):
- self.counters.clear_all()
-
- def test_clear_counters(self):
- self.assertRaises(isc.cc.data.DataNotFoundError,
- self.counters.get, 'counter')
- self.counters.inc('counter')
- self.assertEqual(self.counters.get('counter'), 1)
- self.counters.clear_all()
- self.assertRaises(isc.cc.data.DataNotFoundError,
- self.counters.get, 'counter')
-
- def test_enablediable(self):
- self.assertFalse(self.counters._disabled)
- self.counters.disable()
- self.assertTrue(self.counters._disabled)
- self.counters.enable()
- self.assertFalse(self.counters._disabled)
-
- def test_add_counter_normal(self):
- element = {'counter' : 1}
- self.assertEqual(\
- counter._add_counter(element, [], 'counter'), 1)
-
- def test_add_counter_wrongspec(self):
- self.assertRaises(isc.cc.data.DataNotFoundError,
- counter._add_counter,
- {}, [], 'counter')
-
- def test_add_counter_empty(self):
- self.assertEqual(\
- counter._add_counter(
- {},
- [ { 'item_name' : 'counter',
- 'item_type' : 'integer',
- 'item_default' : 0 } ],
- 'counter'), 0)
-
- def test_add_counter_empty_namedset(self):
- elem = {}
- spec = [ { 'item_name': 'dirs',
- 'item_type': 'named_set',
- 'named_set_item_spec': {
- 'item_name': 'dir',
- 'item_type': 'map',
- 'map_item_spec': [
- { 'item_name': 'counter1',
- 'item_type': 'integer',
- 'item_default': 0 },
- { 'item_name': 'counter2',
- 'item_type': 'integer',
- 'item_default': 0 } ]}
- }]
- self.assertEqual(\
- counter._add_counter(elem, spec, 'dirs/foo/counter1'), 0)
- self.assertEqual(\
- counter._add_counter(elem, spec, 'dirs/bar/counter2'), 0)
-
- def test_timer(self):
- t1 = counter._start_timer()
- t2 = t1 - timedelta(seconds=1)
- self.assertEqual((t1 - t2).seconds, 1)
- elem = {}
- spec = [{ 'item_name': 'time',
- 'item_type': 'real',
- 'item_default': 0.0 }]
- counter._stop_timer(t2, elem, spec, 'time')
- self.assertGreater(counter._get_counter(elem,'time'), 1)
-
- def test_rasing_incrementers(self):
- """ use Thread"""
- concurrency = 3 # number of the threads
- number = 10000 # number of counting per thread
- counter_name = "counter"
- timer_name = "seconds"
- start_time = counter._start_timer()
- start_functor(concurrency, number, self.counters.inc,
- counter_name)
- counter._stop_timer(start_time,
- self.counters._statistics._data,
- self.counters._statistics._spec,
- timer_name)
- self.assertEqual(
- counter._get_counter(self.counters._statistics._data,
- counter_name),
- concurrency * number)
- self.assertGreater(
- counter._get_counter(self.counters._statistics._data,
- timer_name), 0)
-
- def test_concat(self):
- # only strings
- a = ( 'a','b','c','d' )
- self.assertEqual('a/b/c/d', counter._concat(*a))
- self.assertEqual('aTbTcTd', counter._concat(*a, sep='T'))
- self.assertEqual('a\\b\\c\\d', counter._concat(*a, sep='\\'))
- # mixed with other types
- b = a + (1,)
- self.assertRaises(TypeError, counter._concat, *b)
- b = a + (1.1,)
- self.assertRaises(TypeError, counter._concat, *b)
- b = a + ([],)
- self.assertRaises(TypeError, counter._concat, *b)
- b = a + ({},)
- self.assertRaises(TypeError, counter._concat, *b)
-
-class BaseTestCounters():
-
- def setUp(self):
- self._statistics_data = {}
- self.counters = counter.Counters(self.TEST_SPECFILE_LOCATION)
- self._entire_server = self.counters._entire_server
- self._perzone_prefix = self.counters._perzone_prefix
-
- def tearDown(self):
- self.counters.clear_all()
-
- def check_get_statistics(self):
- """Checks no differences between the value returned from
- get_statistics() and locally collected statistics data. Also
- checks the result isn't changed even after the method is
- invoked twice. Finally checks it is valid for the the
- statistics spec."""
- self.assertEqual(self.counters.get_statistics(),
- self._statistics_data)
- # Idempotency check
- self.assertEqual(self.counters.get_statistics(),
- self._statistics_data)
- if self.TEST_SPECFILE_LOCATION:
- self.assertTrue(isc.config.module_spec_from_file(
- self.TEST_SPECFILE_LOCATION).validate_statistics(
- False, self._statistics_data))
- else:
- self.assertTrue(isc.config.ModuleSpec(
- {'module_name': 'Foo',
- 'statistics': self.counters._statistics._spec}
- ).validate_statistics(
- False, self._statistics_data))
-
- def test_perzone_counters(self):
- # for per-zone counters
- for name in self.counters._zones_item_list:
- args = (self._perzone_prefix, TEST_ZONE_NAME_STR, name)
- if name.find('time_to_') == 0:
- self.counters.start_timer(*args)
- self.counters.stop_timer(*args)
- self.assertGreater(self.counters.get(*args), 0)
- sec = self.counters.get(*args)
- for zone_str in (self._entire_server, TEST_ZONE_NAME_STR):
- isc.cc.data.set(self._statistics_data,
- '%s/%s/%s' % (args[0], zone_str, name), sec)
- # twice exec stopper, then second is not changed
- self.counters.stop_timer(*args)
- self.assertEqual(self.counters.get(*args), sec)
- else:
- self.counters.inc(*args)
- self.assertEqual(self.counters.get(*args), 1)
- # checks disable/enable
- self.counters.disable()
- self.counters.inc(*args)
- self.assertEqual(self.counters.get(*args), 1)
- self.counters.enable()
- self.counters.inc(*args)
- self.assertEqual(self.counters.get(*args), 2)
- for zone_str in (self._entire_server, TEST_ZONE_NAME_STR):
- isc.cc.data.set(self._statistics_data,
- '%s/%s/%s' % (args[0], zone_str, name), 2)
- self.check_get_statistics()
-
- def test_xfrrunning_counters(self):
- # for counters of xfer running
- _suffix = 'xfr_running'
- _xfrrunning_names = \
- isc.config.spec_name_list(self.counters._statistics._spec,
- "", True)
- for name in _xfrrunning_names:
- if name.find(_suffix) != 1: continue
- args = name.split('/')
- self.counters.inc(*args)
- self.assertEqual(self.counters.get(*args), 1)
- self.counters.dec(*args)
- self.assertEqual(self.counters.get(*args), 0)
- # checks disable/enable
- self.counters.disable()
- self.counters.inc(*args)
- self.assertEqual(self.counters.get(*args), 0)
- self.counters.enable()
- self.counters.inc(*args)
- self.assertEqual(self.counters.get(*args), 1)
- self.counters.disable()
- self.counters.dec(*args)
- self.assertEqual(self.counters.get(*args), 1)
- self.counters.enable()
- self.counters.dec(*args)
- self.assertEqual(self.counters.get(*args), 0)
- self._statistics_data[name] = 0
- self.check_get_statistics()
-
- def test_socket_counters(self):
- # for ipsocket/unixsocket counters
- _prefix = 'socket/'
- _socket_names = \
- isc.config.spec_name_list(self.counters._statistics._spec,
- "", True)
- for name in _socket_names:
- if name.find(_prefix) != 0: continue
- args = name.split('/')
- self.counters.inc(*args)
- self.assertEqual(self.counters.get(*args), 1)
- # checks disable/enable
- self.counters.disable()
- self.counters.inc(*args)
- self.assertEqual(self.counters.get(*args), 1)
- self.counters.enable()
- self.counters.inc(*args)
- self.assertEqual(self.counters.get(*args), 2)
- isc.cc.data.set(
- self._statistics_data, '/'.join(args), 2)
- self.check_get_statistics()
-
- def test_undefined_item(self):
- # test DataNotFoundError raising when specifying item defined
- # in the specfile
- self.assertRaises(isc.cc.data.DataNotFoundError,
- self.counters.inc, '__undefined__')
- self.assertRaises(isc.cc.data.DataNotFoundError,
- self.counters.dec, '__undefined__')
- self.counters.start_timer('__undefined__')
- self.assertRaises(isc.cc.data.DataNotFoundError,
- self.counters.stop_timer, '__undefined__')
- self.assertRaises(isc.cc.data.DataNotFoundError,
- self.counters.get, '__undefined__')
-
-class TestCounters0(unittest.TestCase, BaseTestCounters):
- TEST_SPECFILE_LOCATION = None
- def setUp(self):
- BaseTestCounters.setUp(self)
- def tearDown(self):
- BaseTestCounters.tearDown(self)
-
-class TestCounters1(unittest.TestCase, BaseTestCounters):
- TEST_SPECFILE_LOCATION = TESTDATA_SRCDIR + os.sep + 'test_spec1.spec'
- def setUp(self):
- BaseTestCounters.setUp(self)
- def tearDown(self):
- BaseTestCounters.tearDown(self)
-
-class TestCounters2(unittest.TestCase, BaseTestCounters):
- TEST_SPECFILE_LOCATION = TESTDATA_SRCDIR + os.sep + 'test_spec2.spec'
- def setUp(self):
- BaseTestCounters.setUp(self)
- def tearDown(self):
- BaseTestCounters.tearDown(self)
-
-class TestCounters3(unittest.TestCase, BaseTestCounters):
- TEST_SPECFILE_LOCATION = TESTDATA_SRCDIR + os.sep + 'test_spec3.spec'
- def setUp(self):
- BaseTestCounters.setUp(self)
- def tearDown(self):
- BaseTestCounters.tearDown(self)
-
-if __name__== "__main__":
- unittest.main()
diff --git a/src/lib/python/isc/statistics/tests/counters_test.py b/src/lib/python/isc/statistics/tests/counters_test.py
new file mode 100644
index 0000000..4ac3919
--- /dev/null
+++ b/src/lib/python/isc/statistics/tests/counters_test.py
@@ -0,0 +1,429 @@
+# Copyright (C) 2012 Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+'''Tests for isc.statistics.counter'''
+
+import unittest
+import threading
+from datetime import timedelta
+import os
+import imp
+import isc.config
+
+TEST_ZONE_NAME_STR = "example.com."
+TESTDATA_SRCDIR = os.getenv("TESTDATASRCDIR")
+
+from isc.statistics import counters
+
+def setup_functor(event, cycle, functor, *args):
+ """Waits until the event is started, and then invokes the functor
+ by times of the cycle with args."""
+ event.wait()
+ for i in range(cycle): functor(*args)
+
+def start_functor(concurrency, number, functor, *args):
+ """Creates the threads of the number and makes them start. Sets
+ the event and waits until these threads are finished."""
+ threads = []
+ event = threading.Event()
+ for i in range(concurrency):
+ threads.append(threading.Thread(\
+ target=setup_functor, \
+ args=(event, number, functor,) + args))
+ for th in threads: th.start()
+ event.set()
+ for th in threads: th.join()
+
+class TestBasicMethods(unittest.TestCase):
+ TEST_SPECFILE_LOCATION = TESTDATA_SRCDIR + os.sep + 'test_spec1.spec'
+
+ def setUp(self):
+ self.counters = counters.Counters(self.TEST_SPECFILE_LOCATION)
+
+ def tearDown(self):
+ self.counters.clear_all()
+
+ def test_clear_counters(self):
+ self.assertRaises(isc.cc.data.DataNotFoundError,
+ self.counters.get, 'counter')
+ self.counters.inc('counter')
+ self.assertEqual(self.counters.get('counter'), 1)
+ self.counters.clear_all()
+ self.assertRaises(isc.cc.data.DataNotFoundError,
+ self.counters.get, 'counter')
+
+ def test_enablediable(self):
+ self.assertFalse(self.counters._disabled)
+ self.counters.disable()
+ self.assertTrue(self.counters._disabled)
+ self.counters.enable()
+ self.assertFalse(self.counters._disabled)
+
+ def test_add_counter_normal(self):
+ element = {'counter' : 1}
+ self.assertEqual(\
+ counters._add_counter(element, [], 'counter'), 1)
+
+ def test_add_counter_wrongspec(self):
+ self.assertRaises(isc.cc.data.DataNotFoundError,
+ counters._add_counter,
+ {}, [], 'counter')
+
+ def test_add_counter_empty(self):
+ self.assertEqual(\
+ counters._add_counter(
+ {},
+ [ { 'item_name' : 'counter',
+ 'item_type' : 'integer',
+ 'item_default' : 0 } ],
+ 'counter'), 0)
+
+ def test_add_counter_empty_namedset(self):
+ elem = {}
+ spec = [ { 'item_name': 'dirs',
+ 'item_type': 'named_set',
+ 'named_set_item_spec': {
+ 'item_name': 'dir',
+ 'item_type': 'map',
+ 'map_item_spec': [
+ { 'item_name': 'counter1',
+ 'item_type': 'integer',
+ 'item_default': 0 },
+ { 'item_name': 'counter2',
+ 'item_type': 'integer',
+ 'item_default': 0 } ]}
+ }]
+ self.assertEqual(\
+ counters._add_counter(elem, spec, 'dirs/foo/counter1'), 0)
+ self.assertEqual(\
+ counters._add_counter(elem, spec, 'dirs/bar/counter2'), 0)
+
+ def test_timer(self):
+ t1 = counters._start_timer()
+ t2 = t1 - timedelta(seconds=1)
+ self.assertEqual((t1 - t2).seconds, 1)
+ elem = {}
+ spec = [{ 'item_name': 'time',
+ 'item_type': 'real',
+ 'item_default': 0.0 }]
+ counters._stop_timer(t2, elem, spec, 'time')
+ self.assertGreater(counters._get_counter(elem,'time'), 1)
+
+ def test_rasing_incrementers(self):
+ """ use Thread"""
+ concurrency = 3 # number of the threads
+ number = 10000 # number of counting per thread
+ counter_name = "counter"
+ timer_name = "seconds"
+ start_time = counters._start_timer()
+ start_functor(concurrency, number, self.counters.inc,
+ counter_name)
+ counters._stop_timer(start_time,
+ self.counters._statistics._data,
+ self.counters._statistics._spec,
+ timer_name)
+ self.assertEqual(
+ counters._get_counter(self.counters._statistics._data,
+ counter_name),
+ concurrency * number)
+ self.assertGreater(
+ counters._get_counter(self.counters._statistics._data,
+ timer_name), 0)
+
+ def test_concat(self):
+ # only strings
+ a = ( 'a','b','c','d' )
+ self.assertEqual('a/b/c/d', counters._concat(*a))
+ self.assertEqual('aTbTcTd', counters._concat(*a, sep='T'))
+ self.assertEqual('a\\b\\c\\d', counters._concat(*a, sep='\\'))
+ # mixed with other types
+ b = a + (1,)
+ self.assertRaises(TypeError, counters._concat, *b)
+ b = a + (1.1,)
+ self.assertRaises(TypeError, counters._concat, *b)
+ b = a + ([],)
+ self.assertRaises(TypeError, counters._concat, *b)
+ b = a + ({},)
+ self.assertRaises(TypeError, counters._concat, *b)
+
+class BaseTestCounters():
+
+ def setUp(self):
+ self._statistics_data = {}
+ self.counters = counters.Counters(self.TEST_SPECFILE_LOCATION)
+ self._entire_server = self.counters._entire_server
+ self._perzone_prefix = self.counters._perzone_prefix
+
+ def tearDown(self):
+ self.counters.clear_all()
+
+ def check_get_statistics(self):
+ """Checks no differences between the value returned from
+ get_statistics() and locally collected statistics data. Also
+ checks the result isn't changed even after the method is
+ invoked twice. Finally checks it is valid for the the
+ statistics spec."""
+ self.assertEqual(self.counters.get_statistics(),
+ self._statistics_data)
+ # Idempotency check
+ self.assertEqual(self.counters.get_statistics(),
+ self._statistics_data)
+ if self.TEST_SPECFILE_LOCATION:
+ self.assertTrue(isc.config.module_spec_from_file(
+ self.TEST_SPECFILE_LOCATION).validate_statistics(
+ False, self._statistics_data))
+ else:
+ self.assertTrue(isc.config.ModuleSpec(
+ {'module_name': 'Foo',
+ 'statistics': self.counters._statistics._spec}
+ ).validate_statistics(
+ False, self._statistics_data))
+
+ def test_perzone_counters(self):
+ # for per-zone counters
+ for name in self.counters._zones_item_list:
+ args = (self._perzone_prefix, TEST_ZONE_NAME_STR, name)
+ if name.find('time_to_') == 0:
+ self.counters.start_timer(*args)
+ self.counters.stop_timer(*args)
+ self.assertGreater(self.counters.get(*args), 0)
+ sec = self.counters.get(*args)
+ for zone_str in (self._entire_server, TEST_ZONE_NAME_STR):
+ isc.cc.data.set(self._statistics_data,
+ '%s/%s/%s' % (args[0], zone_str, name), sec)
+ # twice exec stopper, then second is not changed
+ self.counters.stop_timer(*args)
+ self.assertEqual(self.counters.get(*args), sec)
+ else:
+ self.counters.inc(*args)
+ self.assertEqual(self.counters.get(*args), 1)
+ # checks disable/enable
+ self.counters.disable()
+ self.counters.inc(*args)
+ self.assertEqual(self.counters.get(*args), 1)
+ self.counters.enable()
+ self.counters.inc(*args)
+ self.assertEqual(self.counters.get(*args), 2)
+ for zone_str in (self._entire_server, TEST_ZONE_NAME_STR):
+ isc.cc.data.set(self._statistics_data,
+ '%s/%s/%s' % (args[0], zone_str, name), 2)
+ self.check_get_statistics()
+
+ def test_xfrrunning_counters(self):
+ # for counters of xfer running
+ _suffix = 'xfr_running'
+ _xfrrunning_names = \
+ isc.config.spec_name_list(self.counters._statistics._spec,
+ "", True)
+ for name in _xfrrunning_names:
+ if name.find(_suffix) != 1: continue
+ args = name.split('/')
+ self.counters.inc(*args)
+ self.assertEqual(self.counters.get(*args), 1)
+ self.counters.dec(*args)
+ self.assertEqual(self.counters.get(*args), 0)
+ # checks disable/enable
+ self.counters.disable()
+ self.counters.inc(*args)
+ self.assertEqual(self.counters.get(*args), 0)
+ self.counters.enable()
+ self.counters.inc(*args)
+ self.assertEqual(self.counters.get(*args), 1)
+ self.counters.disable()
+ self.counters.dec(*args)
+ self.assertEqual(self.counters.get(*args), 1)
+ self.counters.enable()
+ self.counters.dec(*args)
+ self.assertEqual(self.counters.get(*args), 0)
+ self._statistics_data[name] = 0
+ self.check_get_statistics()
+
+ def test_socket_counters(self):
+ # for ipsocket/unixsocket counters
+ _prefix = 'socket/'
+ _socket_names = \
+ isc.config.spec_name_list(self.counters._statistics._spec,
+ "", True)
+ for name in _socket_names:
+ if name.find(_prefix) != 0: continue
+ args = name.split('/')
+ self.counters.inc(*args)
+ self.assertEqual(self.counters.get(*args), 1)
+ # checks disable/enable
+ self.counters.disable()
+ self.counters.inc(*args)
+ self.assertEqual(self.counters.get(*args), 1)
+ self.counters.enable()
+ self.counters.inc(*args)
+ self.assertEqual(self.counters.get(*args), 2)
+ isc.cc.data.set(
+ self._statistics_data, '/'.join(args), 2)
+ self.check_get_statistics()
+
+ def test_undefined_item(self):
+ # test DataNotFoundError raising when specifying item defined
+ # in the specfile
+ self.assertRaises(isc.cc.data.DataNotFoundError,
+ self.counters.inc, '__undefined__')
+ self.assertRaises(isc.cc.data.DataNotFoundError,
+ self.counters.dec, '__undefined__')
+ self.counters.start_timer('__undefined__')
+ self.assertRaises(isc.cc.data.DataNotFoundError,
+ self.counters.stop_timer, '__undefined__')
+ self.assertRaises(isc.cc.data.DataNotFoundError,
+ self.counters.get, '__undefined__')
+
+class TestCounters0(unittest.TestCase, BaseTestCounters):
+ TEST_SPECFILE_LOCATION = None
+ @classmethod
+ def setUpClass(cls):
+ imp.reload(counters)
+ def setUp(self):
+ BaseTestCounters.setUp(self)
+ def tearDown(self):
+ BaseTestCounters.tearDown(self)
+
+class TestCounters1(unittest.TestCase, BaseTestCounters):
+ TEST_SPECFILE_LOCATION = TESTDATA_SRCDIR + os.sep + 'test_spec1.spec'
+ @classmethod
+ def setUpClass(cls):
+ imp.reload(counters)
+ def setUp(self):
+ BaseTestCounters.setUp(self)
+ def tearDown(self):
+ BaseTestCounters.tearDown(self)
+
+class TestCounters2(unittest.TestCase, BaseTestCounters):
+ TEST_SPECFILE_LOCATION = TESTDATA_SRCDIR + os.sep + 'test_spec2.spec'
+ @classmethod
+ def setUpClass(cls):
+ imp.reload(counters)
+ def setUp(self):
+ BaseTestCounters.setUp(self)
+ def tearDown(self):
+ BaseTestCounters.tearDown(self)
+
+class TestCounters3(unittest.TestCase, BaseTestCounters):
+ TEST_SPECFILE_LOCATION = TESTDATA_SRCDIR + os.sep + 'test_spec3.spec'
+ @classmethod
+ def setUpClass(cls):
+ imp.reload(counters)
+ def setUp(self):
+ BaseTestCounters.setUp(self)
+ def tearDown(self):
+ BaseTestCounters.tearDown(self)
+
+class BaseDummyModule():
+ """A base dummy class"""
+ TEST_SPECFILE_LOCATION = TESTDATA_SRCDIR + os.sep + 'test_spec2.spec'
+ def __init__(self):
+ self.counters = counters.Counters(self.TEST_SPECFILE_LOCATION)
+
+ def get_counters(self):
+ return self.counters.get_statistics()
+
+ def clear_counters(self):
+ self.counters.clear_all()
+
+class DummyNotifyOut(BaseDummyModule):
+ """A dummy class equivalent to notify.notify_out.NotifyOut"""
+ def __init__(self):
+ self.counters = counters.Counters()
+
+ def inc_counters(self):
+ """increments counters"""
+ self.counters.inc('zones', TEST_ZONE_NAME_STR, 'notifyoutv4')
+ self.counters.inc('zones', TEST_ZONE_NAME_STR, 'notifyoutv6')
+
+class DummyXfroutSession(BaseDummyModule):
+ """A dummy class equivalent to XfroutSession in b10-xfrout"""
+ def inc_counters(self):
+ """increments counters"""
+ self.counters.inc('zones', TEST_ZONE_NAME_STR, 'xfrreqdone')
+ self.counters.inc('zones', TEST_ZONE_NAME_STR, 'xfrrej')
+ self.counters.inc('axfr_running')
+ self.counters.inc('ixfr_running')
+ self.counters.dec('axfr_running')
+ self.counters.dec('ixfr_running')
+
+class DummyUnixSockServer(BaseDummyModule):
+ """A dummy class equivalent to UnixSockServer in b10-xfrout"""
+ def inc_counters(self):
+ """increments counters"""
+ self.counters.inc('socket', 'unixdomain', 'open')
+ self.counters.inc('socket', 'unixdomain', 'close')
+
+class DummyXfroutServer(BaseDummyModule):
+ """A dummy class equivalent to XfroutServer in b10-xfrout"""
+ def __init__(self):
+ super().__init__()
+ self.xfrout_sess = DummyXfroutSession()
+ self.unix_socket_server = DummyUnixSockServer()
+ self.notifier = DummyNotifyOut()
+
+ def inc_counters(self):
+ self.xfrout_sess.inc_counters()
+ self.unix_socket_server.inc_counters()
+ self.notifier.inc_counters()
+
+class TestDummyNotifyOut(unittest.TestCase):
+ """Tests counters are incremented in which the spec file is not
+ loaded"""
+ @classmethod
+ def setUpClass(cls):
+ imp.reload(counters)
+
+ def setUp(self):
+ self.notifier = DummyNotifyOut()
+ self.notifier.inc_counters()
+
+ def tearDown(self):
+ self.notifier.clear_counters()
+
+ def test_counters(self):
+ self.assertEqual(
+ {'zones': {'_SERVER_': {'notifyoutv4': 1, 'notifyoutv6': 1},
+ TEST_ZONE_NAME_STR: {'notifyoutv4': 1, 'notifyoutv6': 1}}},
+ self.notifier.get_counters())
+
+class TestDummyXfroutServer(unittest.TestCase):
+ """Tests counters are incremented or decremented in which the same
+ spec file is multiply loaded in each child class"""
+ @classmethod
+ def setUpClass(cls):
+ imp.reload(counters)
+
+ def setUp(self):
+ self.xfrout_server = DummyXfroutServer()
+ self.xfrout_server.inc_counters()
+
+ def tearDown(self):
+ self.xfrout_server.clear_counters()
+
+ def test_counters(self):
+ self.assertEqual(
+ {'axfr_running': 0, 'ixfr_running': 0,
+ 'socket': {'unixdomain': {'open': 1, 'close': 1}},
+ 'zones': {'_SERVER_': {'notifyoutv4': 1,
+ 'notifyoutv6': 1,
+ 'xfrrej': 1, 'xfrreqdone': 1},
+ TEST_ZONE_NAME_STR: {'notifyoutv4': 1,
+ 'notifyoutv6': 1,
+ 'xfrrej': 1,
+ 'xfrreqdone': 1}}},
+ self.xfrout_server.get_counters())
+
+if __name__== "__main__":
+ unittest.main()
More information about the bind10-changes
mailing list