BIND 10 trac213, updated. 64864ebe66d735bdca2ebd4050e83de3f1efb0e3 Propagation of config with tests
BIND 10 source code commits
bind10-changes at lists.isc.org
Fri Oct 14 20:07:03 UTC 2011
The branch, trac213 has been updated
via 64864ebe66d735bdca2ebd4050e83de3f1efb0e3 (commit)
via 544eddbd71a9e721b8a7f4df9440712128c6daa0 (commit)
via 54e45bdd0c4cbc6fcf946a3f9a3614f432d895c4 (commit)
via 4548fb5208e174b188374deefac98b1ede014c0e (commit)
via efeedc6fcc731f671173f6d4e8c4e5f1313733c2 (commit)
via 0d15980542bb1a1d4a3e82dfdd98aba236642110 (commit)
via 86a0673fe4b3b855120d70a07e632010cf80af60 (commit)
via 40e47cae104a30b318348eae3bd40b5305cae620 (commit)
from e8e7e9bd02c509454760eb25542e45d8e8374276 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 64864ebe66d735bdca2ebd4050e83de3f1efb0e3
Author: Michal 'vorner' Vaner <michal.vaner at nic.cz>
Date: Fri Oct 14 22:06:43 2011 +0200
Propagation of config with tests
commit 544eddbd71a9e721b8a7f4df9440712128c6daa0
Author: Michal 'vorner' Vaner <michal.vaner at nic.cz>
Date: Fri Oct 14 21:08:15 2011 +0200
More dead test code removed
commit 54e45bdd0c4cbc6fcf946a3f9a3614f432d895c4
Author: Michal 'vorner' Vaner <michal.vaner at nic.cz>
Date: Fri Oct 14 21:05:25 2011 +0200
Purge invalid items in TODO
commit 4548fb5208e174b188374deefac98b1ede014c0e
Author: Michal 'vorner' Vaner <michal.vaner at nic.cz>
Date: Fri Oct 14 20:58:17 2011 +0200
Mess 4
* Drop a whole bunch of tests that check starting and stopping of
components. This is now delegated to the Configurator
* Disable brittle tests, brittle does not work for now. Maybe we don't
need it.
* Cleanup in the boss code, dropping bunch of dead code parts
* Repaired the get_processes method.
commit efeedc6fcc731f671173f6d4e8c4e5f1313733c2
Author: Michal 'vorner' Vaner <michal.vaner at nic.cz>
Date: Fri Oct 14 18:27:29 2011 +0200
Logging in components & configurator
commit 0d15980542bb1a1d4a3e82dfdd98aba236642110
Author: Michal 'vorner' Vaner <michal.vaner at nic.cz>
Date: Fri Oct 14 17:43:58 2011 +0200
Component cleanups
commit 86a0673fe4b3b855120d70a07e632010cf80af60
Author: Michal 'vorner' Vaner <michal.vaner at nic.cz>
Date: Fri Oct 14 16:38:50 2011 +0200
Asi vypÃná
commit 40e47cae104a30b318348eae3bd40b5305cae620
Author: Michal 'vorner' Vaner <michal.vaner at nic.cz>
Date: Fri Oct 14 16:29:19 2011 +0200
Few fixes
-----------------------------------------------------------------------
Summary of changes:
src/bin/bind10/TODO | 6 -
src/bin/bind10/bind10_messages.mes | 53 +++
src/bin/bind10/bind10_src.py.in | 90 +++--
src/bin/bind10/tests/bind10_test.py.in | 501 +++++----------------
src/lib/python/isc/bind10/component.py | 103 ++++-
src/lib/python/isc/bind10/tests/component_test.py | 61 +++-
6 files changed, 349 insertions(+), 465 deletions(-)
-----------------------------------------------------------------------
diff --git a/src/bin/bind10/TODO b/src/bin/bind10/TODO
index eb0abcd..6f50dbd 100644
--- a/src/bin/bind10/TODO
+++ b/src/bin/bind10/TODO
@@ -1,19 +1,13 @@
- Read msgq configuration from configuration manager (Trac #213)
https://bind10.isc.org/ticket/213
- Provide more administrator options:
- - Get process list
- Get information on a process (returns list of times started & stopped,
plus current information such as PID)
- - Add a component (not necessary for parking lot, but...)
- Stop a component
- Force-stop a component
- Mechanism to wait for child to start before continuing
-- Way to ask a child to die politely
-- Start statistics daemon
-- Statistics interaction (?)
- Use .spec file to define comands
- Rename "c-channel" stuff to msgq for clarity
-- Use logger
- Reply to shutdown message?
- Some sort of group creation so termination signals can be sent to
children of children processes (if any)
diff --git a/src/bin/bind10/bind10_messages.mes b/src/bin/bind10/bind10_messages.mes
index 4bac069..a4a62b5 100644
--- a/src/bin/bind10/bind10_messages.mes
+++ b/src/bin/bind10/bind10_messages.mes
@@ -20,6 +20,27 @@ The boss process is starting up and will now check if the message bus
daemon is already running. If so, it will not be able to start, as it
needs a dedicated message bus.
+% BIND10_COMPONENT_RESTART component %1 is about to restart
+The named component failed previously and we will try to restart it to provide
+as flawless service as possible, but it should be investigated what happened,
+as it could happen again.
+
+% BIND10_COMPONENT_START component %1 is starting
+The named component is about to be started by the boss process.
+
+% BIND10_COMPONENT_START_EXCEPTION component %1 failed to start: %2
+An exception (mentioned in the message) happened during the startup of the
+named component. The componet is not considered started and further actions
+will be taken about it.
+
+% BIND10_COMPONENT_STOP component %1 is being stopped
+A component is about to be asked to stop willingly by the boss.
+
+% BIND10_COMPONENT_UNSATISFIED component %1 is required to run and failed
+A component failed for some reason (see previous messages). It is either a core
+component or needed component that was just started. In any case, the system
+can't continue without it and will terminate.
+
% BIND10_CONFIGURATION_START_AUTH start authoritative server: %1
This message shows whether or not the authoritative server should be
started according to the configuration.
@@ -28,6 +49,38 @@ started according to the configuration.
This message shows whether or not the resolver should be
started according to the configuration.
+% BIND10_CONFIGURATOR_BUILD building plan '%1' -> '%2'
+A debug message. This indicates that the configurator is building a plan
+how to change configuration from the older one to newer one. This does no
+real work yet, it just does the planning what needs to be done.
+
+% BIND10_CONFIGURATOR_RECONFIGURE reconfiguring running components
+A different configuration of which components should be running is being
+installed. All components that are no longer needed will be stopped and
+newly introduced ones started. This happens at startup, when the configuration
+is read the first time, or when an operator changes configuration of the boss.
+
+% BIND10_CONFIGURATOR_RUN running plan of %1 tasks
+A debug message. The configurator is about to execute a plan of actions it
+computed previously.
+
+% BIND10_CONFIGURATOR_START bind10 component configurator is starting up
+The part that cares about starting and stopping the right component from the
+boss process is starting up. This happens only once at the startup of the
+boss process. It will start the basic set of processes now (the ones boss
+needs to read the configuration), the rest will be started after the
+configuration is known.
+
+% BIND10_CONFIGURATOR_STOP bind10 component configurator is shutting down
+The part that cares about starting and stopping processes in the boss is
+shutting down. All started components will be shut down now (more precisely,
+asked to terminate by their own, if they fail to comply, other parts of
+the boss process will try to force them).
+
+% BIND10_CONFIGURATOR_TASK performing task %1 on %2
+A debug message. The configurator is about to perform one task of the plan it
+is currently executing on the named component.
+
% BIND10_INVALID_USER invalid user: %1
The boss process was started with the -u option, to drop root privileges
and continue running as the specified user, but the user is unknown.
diff --git a/src/bin/bind10/bind10_src.py.in b/src/bin/bind10/bind10_src.py.in
index d09ad5b..19e1b14 100755
--- a/src/bin/bind10/bind10_src.py.in
+++ b/src/bin/bind10/bind10_src.py.in
@@ -67,7 +67,6 @@ import isc.util.process
import isc.net.parse
import isc.log
from isc.log_messages.bind10_messages import *
-import isc.bind10.sockcreator
import isc.bind10.component
isc.log.init("b10-boss")
@@ -143,7 +142,7 @@ class ProcessInfo:
def __init__(self, name, args, env={}, dev_null_stdout=False,
dev_null_stderr=False, uid=None, username=None):
- self.name = name
+ self.name = name
self.args = args
self.env = env
self.dev_null_stdout = dev_null_stdout
@@ -211,13 +210,13 @@ class CChannelConnectError(Exception): pass
class BoB:
"""Boss of BIND class."""
-
+
def __init__(self, msgq_socket_file=None, data_path=None,
config_filename=None, nocache=False, verbose=False, setuid=None,
username=None, cmdctl_port=None, brittle=False):
"""
Initialize the Boss of BIND. This is a singleton (only one can run).
-
+
The msgq_socket_file specifies the UNIX domain socket file that the
msgq process listens on. If verbose is True, then the boss reports
what it is doing.
@@ -243,8 +242,7 @@ class BoB:
self.config_filename = config_filename
self.cmdctl_port = cmdctl_port
self.brittle = brittle
- self.sockcreator = None
- self.__component_configurator = isc.bind10.component.Configurator(self)
+ self._component_configurator = isc.bind10.component.Configurator(self)
self.__core_components = {
'sockcreator': {
'kind': 'core',
@@ -262,6 +260,20 @@ class BoB:
'priority': 198
}
}
+ self.__started = False
+ self.__stopping = False
+ self.exitcode = 0
+
+ def __propagate_component_config(self, config):
+ comps = dict(config)
+ # Fill in the core components, so they stay alive
+ for comp in self.__core_components:
+ if comp in comps:
+ raise Exception(comp + " is core component managed by " +
+ "bind10 boss, do not set it")
+ comps[comp] = self.__core_components[comp]
+ # Update the configuration
+ self._component_configurator.reconfigure(comps)
def config_handler(self, new_config):
# If this is initial update, don't do anything now, leave it to startup
@@ -269,17 +281,17 @@ class BoB:
return
logger.debug(DBG_COMMANDS, BIND10_RECEIVED_NEW_CONFIGURATION,
new_config)
- # TODO: Propagate the config to the configurator
+ if 'components' in new_config:
+ self.__propagate_component_config(new_config['components'])
answer = isc.config.ccsession.create_answer(0)
return answer
def get_processes(self):
- # FIXME: This is all wrong.
pids = list(self.processes.keys())
pids.sort()
process_list = [ ]
for pid in pids:
- process_list.append([pid, self.processes[pid].name])
+ process_list.append([pid, self.processes[pid].name()])
return process_list
def _get_stats_data(self):
@@ -319,11 +331,6 @@ class BoB:
"Unknown command")
return answer
- def start_creator(self):
- self.curproc = 'b10-sockcreator'
- self.sockcreator = isc.bind10.sockcreator.Creator("@@LIBEXECDIR@@:" +
- os.environ['PATH'])
-
def kill_started_processes(self):
"""
Called as part of the exception handling when a process fails to
@@ -332,14 +339,12 @@ class BoB:
"""
logger.info(BIND10_KILLING_ALL_PROCESSES)
- self.stop_creator(True)
-
for pid in self.processes:
logger.info(BIND10_KILL_PROCESS, self.processes[pid].name)
self.processes[pid].process.kill()
self.processes = {}
- if self.__component_configurator.running():
- self.__component_configurator.shutdown()
+ if self._component_configurator.running():
+ self._component_configurator.shutdown()
def read_bind10_config(self):
"""
@@ -354,6 +359,7 @@ class BoB:
logger.info(BIND10_READING_BOSS_CONFIGURATION)
config_data = self.ccs.get_full_config()
+ self.__propagate_component_config(config_data['components'])
# Propagate the config to the config manager, first reconfigure
def log_starting(self, process, port = None, address = None):
@@ -450,8 +456,8 @@ class BoB:
The argument c_channel_env is unused but is supplied to keep the
argument list the same for all start_xxx methods.
"""
- self.log_starting("ccsession")
- self.ccs = isc.config.ModuleCCSession(SPECFILE_LOCATION,
+ self.log_starting("ccsession") #FIXME This is not a process, can't tell a process is starting
+ self.ccs = isc.config.ModuleCCSession(SPECFILE_LOCATION,
self.config_handler,
self.command_handler)
self.ccs.start()
@@ -544,7 +550,7 @@ class BoB:
Starts up all the processes. Any exception generated during the
starting of the processes is handled by the caller.
"""
- self.__component_configurator.startup(self.__core_components)
+ self._component_configurator.startup(self.__core_components)
# TODO: Once everything uses the socket creator, we can drop root
# privileges right now
@@ -588,13 +594,12 @@ class BoB:
# Started successfully
self.runnable = True
+ self.__started = True
return None
def stop_all_processes(self):
"""Stop all processes."""
- cmd = { "command": ['shutdown']}
-
- self.__component_configurator.shutdown()
+ self._component_configurator.shutdown()
def stop_process(self, process, recipient):
"""
@@ -606,11 +611,23 @@ class BoB:
self.cc_session.group_sendmsg({'command': ['shutdown']}, recipient,
recipient)
- def shutdown(self, exitcode=0):
+ def component_shutdown(self, exitcode=0):
+ """
+ Stop the Boss instance from a components' request. The exitcode
+ indicates the desired exit code.
+ """
+ if self.__stopping:
+ return
+ self.exitcode = exitcode
+ if not self.__started:
+ raise Exception("Component failed during startup");
+ else:
+ self.runnable = False
+
+ def shutdown(self):
"""Stop the BoB instance."""
- # FIXME: This is called from various places.
- # Do something about it, as each shutdown is different one
logger.info(BIND10_SHUTDOWN)
+ self.__stopping = True
# first try using the BIND 10 request to stop
try:
self.stop_all_processes()
@@ -623,12 +640,11 @@ class BoB:
self.reap_children()
# next try sending a SIGTERM
processes_to_stop = list(self.processes.values())
- for proc_info in processes_to_stop:
- logger.info(BIND10_SEND_SIGTERM, proc_info.name,
- proc_info.pid)
+ for component in processes_to_stop:
+ logger.info(BIND10_SEND_SIGTERM, component.name(),
+ component.pid())
try:
- # FIXME: This won't work. We replaced them with something else
- proc_info.process.terminate()
+ os.kill(component.pid(), signal.SIGTERM)
except OSError:
# ignore these (usually ESRCH because the child
# finally exited)
@@ -639,11 +655,11 @@ class BoB:
time.sleep(0.1)
self.reap_children()
processes_to_stop = list(self.processes.values())
- for proc_info in processes_to_stop:
- logger.info(BIND10_SEND_SIGKILL, proc_info.name,
- proc_info.pid)
+ for component in processes_to_stop:
+ logger.info(BIND10_SEND_SIGKILL, component.name(),
+ component.pid())
try:
- proc_info.process.kill()
+ os.kill(component.pid(), signal.SIGKILL)
except OSError:
# ignore these (usually ESRCH because the child
# finally exited)
@@ -927,7 +943,7 @@ def main():
signal.signal(signal.SIGCHLD, signal.SIG_DFL)
boss_of_bind.shutdown()
unlink_pid_file(options.pid_file)
- sys.exit(0)
+ sys.exit(boss_of_bind.exitcode)
if __name__ == "__main__":
main()
diff --git a/src/bin/bind10/tests/bind10_test.py.in b/src/bin/bind10/tests/bind10_test.py.in
index 3c9b12e..5b84caa 100644
--- a/src/bin/bind10/tests/bind10_test.py.in
+++ b/src/bin/bind10/tests/bind10_test.py.in
@@ -110,11 +110,6 @@ class TestBoB(unittest.TestCase):
self.assertEqual(bob.uid, None)
self.assertEqual(bob.username, None)
self.assertEqual(bob.nocache, False)
- self.assertEqual(bob.cfg_start_auth, True)
- self.assertEqual(bob.cfg_start_resolver, False)
-
- self.assertEqual(bob.cfg_start_dhcp4, False)
- self.assertEqual(bob.cfg_start_dhcp6, False)
def test_init_alternate_socket(self):
bob = BoB("alt_socket_file")
@@ -128,10 +123,6 @@ class TestBoB(unittest.TestCase):
self.assertEqual(bob.uid, None)
self.assertEqual(bob.username, None)
self.assertEqual(bob.nocache, False)
- self.assertEqual(bob.cfg_start_auth, True)
- self.assertEqual(bob.cfg_start_resolver, False)
- self.assertEqual(bob.cfg_start_dhcp4, False)
- self.assertEqual(bob.cfg_start_dhcp6, False)
def test_command_handler(self):
class DummySession():
@@ -174,11 +165,19 @@ class TestBoB(unittest.TestCase):
self.assertEqual(bob.command_handler("__UNKNOWN__", None),
isc.config.ccsession.create_answer(1, "Unknown command"))
-class TestComponent(Component):
- def start_internal(self):
+class MockComponent:
+ def __init__(self, name, pid):
+ self.name = lambda: name
+ self.pid = lambda: pid
+
+class MockConfigurator:
+ def startup(self, config):
+ pass
+
+ def shutdown(self):
pass
- def stop_internal(self, Kill=False):
+ def reconfigure(self, config):
pass
# Class for testing the BoB without actually starting processes.
@@ -206,408 +205,129 @@ class MockBob(BoB):
self.stats_httpd = False
self.cmdctl = False
self.c_channel_env = {}
- self.processes = { }
- isc.bind10.component.specials['sockcreator'] = TestComponent
+ self.processes = {
+ 1: MockComponent('first', 1),
+ 2: MockComponent('second', 2)
+ }
+ self._component_configurator = MockConfigurator()
def read_bind10_config(self):
# Configuration options are set directly
pass
- def start_msgq(self, c_channel_env):
- self.msgq = True
- self.processes[2] = ProcessInfo('b10-msgq', ['/bin/false'])
- self.processes[2].pid = 2
-
- def start_cfgmgr(self, c_channel_env):
- self.cfgmgr = True
- self.processes[3] = ProcessInfo('b10-cfgmgr', ['/bin/false'])
- self.processes[3].pid = 3
-
def start_ccsession(self, c_channel_env):
- self.ccsession = True
- self.processes[4] = ProcessInfo('b10-ccsession', ['/bin/false'])
- self.processes[4].pid = 4
-
- def start_auth(self, c_channel_env):
- self.auth = True
- self.processes[5] = ProcessInfo('b10-auth', ['/bin/false'])
- self.processes[5].pid = 5
-
- def start_resolver(self, c_channel_env):
- self.resolver = True
- self.processes[6] = ProcessInfo('b10-resolver', ['/bin/false'])
- self.processes[6].pid = 6
-
- def start_xfrout(self, c_channel_env):
- self.xfrout = True
- self.processes[7] = ProcessInfo('b10-xfrout', ['/bin/false'])
- self.processes[7].pid = 7
-
- def start_xfrin(self, c_channel_env):
- self.xfrin = True
- self.processes[8] = ProcessInfo('b10-xfrin', ['/bin/false'])
- self.processes[8].pid = 8
-
- def start_zonemgr(self, c_channel_env):
- self.zonemgr = True
- self.processes[9] = ProcessInfo('b10-zonemgr', ['/bin/false'])
- self.processes[9].pid = 9
-
- def start_stats(self, c_channel_env):
- self.stats = True
- self.processes[10] = ProcessInfo('b10-stats', ['/bin/false'])
- self.processes[10].pid = 10
-
- def start_stats_httpd(self, c_channel_env):
- self.stats_httpd = True
- self.processes[11] = ProcessInfo('b10-stats-httpd', ['/bin/false'])
- self.processes[11].pid = 11
-
- def start_cmdctl(self, c_channel_env):
- self.cmdctl = True
- self.processes[12] = ProcessInfo('b10-cmdctl', ['/bin/false'])
- self.processes[12].pid = 12
-
- def start_dhcp6(self, c_channel_env):
- self.dhcp6 = True
- self.processes[13] = ProcessInfo('b10-dhcp6', ['/bin/false'])
- self.processes[13]
-
- def start_dhcp4(self, c_channel_env):
- self.dhcp4 = True
- self.processes[14] = ProcessInfo('b10-dhcp4', ['/bin/false'])
- self.processes[14]
-
- # We don't really use all of these stop_ methods. But it might turn out
- # someone would add some stop_ method to BoB and we want that one overriden
- # in case he forgets to update the tests.
- def stop_msgq(self):
- if self.msgq:
- del self.processes[2]
- self.msgq = False
-
- def stop_cfgmgr(self):
- if self.cfgmgr:
- del self.processes[3]
- self.cfgmgr = False
-
- def stop_ccsession(self):
- if self.ccssession:
- del self.processes[4]
- self.ccsession = False
-
- def stop_auth(self):
- if self.auth:
- del self.processes[5]
- self.auth = False
-
- def stop_resolver(self):
- if self.resolver:
- del self.processes[6]
- self.resolver = False
-
- def stop_xfrout(self):
- if self.xfrout:
- del self.processes[7]
- self.xfrout = False
-
- def stop_xfrin(self):
- if self.xfrin:
- del self.processes[8]
- self.xfrin = False
-
- def stop_zonemgr(self):
- if self.zonemgr:
- del self.processes[9]
- self.zonemgr = False
-
- def stop_stats(self):
- if self.stats:
- del self.processes[10]
- self.stats = False
-
- def stop_stats_httpd(self):
- if self.stats_httpd:
- del self.processes[11]
- self.stats_httpd = False
-
- def stop_cmdctl(self):
- if self.cmdctl:
- del self.processes[12]
- self.cmdctl = False
+ pass
-class TestStartStopProcessesBob(unittest.TestCase):
+class TestBossComponents(unittest.TestCase):
"""
- Check that the start_all_processes method starts the right combination
- of processes and that the right processes are started and stopped
- according to changes in configuration.
+ Test the boss propagates component configuration properly to the
+ component configurator and acts sane.
"""
- def check_started(self, bob, core, auth, resolver):
- """
- Check that the right sets of services are started. The ones that
- should be running are specified by the core, auth and resolver parameters
- (they are groups of processes, eg. auth means b10-auth, -xfrout, -xfrin
- and -zonemgr).
- """
- self.assertEqual(bob.msgq, core)
- self.assertEqual(bob.cfgmgr, core)
- self.assertEqual(bob.ccsession, core)
- self.assertEqual(bob.auth, auth)
- self.assertEqual(bob.resolver, resolver)
- self.assertEqual(bob.xfrout, auth)
- self.assertEqual(bob.xfrin, auth)
- self.assertEqual(bob.zonemgr, auth)
- self.assertEqual(bob.stats, core)
- self.assertEqual(bob.stats_httpd, core)
- self.assertEqual(bob.cmdctl, core)
-
- def check_preconditions(self, bob):
- self.check_started(bob, False, False, False)
-
- def check_started_none(self, bob):
- """
- Check that the situation is according to configuration where no servers
- should be started. Some processes still need to be running.
- """
- self.check_started(bob, True, False, False)
+ def setUp(self):
+ self.__param = None
+ self.__called = False
+ self.__compconfig = {
+ 'comp': {
+ 'kind': 'needed',
+ 'process': 'cat'
+ }
+ }
- def check_started_both(self, bob):
+ def __unary_hook(self, param):
"""
- Check the situation is according to configuration where both servers
- (auth and resolver) are enabled.
+ A hook function that stores the parameter for later examination.
"""
- self.check_started(bob, True, True, True)
+ self.__param = param
- def check_started_auth(self, bob):
+ def __nullary_hook(self):
"""
- Check the set of processes needed to run auth only is started.
+ A hook function that notes down it was called.
"""
- self.check_started(bob, True, True, False)
+ self.__called = True
- def check_started_resolver(self, bob):
+ def __check_core(self, config):
"""
- Check the set of processes needed to run resolver only is started.
+ A function checking that the config contains parts for the valid
+ core component configuration.
"""
- self.check_started(bob, True, False, True)
+ self.assertIsNotNone(config)
+ for component in ['sockcreator', 'msgq', 'cfgmgr']:
+ self.assertTrue(component in config)
+ self.assertEqual(component, config[component]['special'])
+ self.assertEqual('core', config[component]['kind'])
- def check_started_dhcp(self, bob, v4, v6):
+ def __check_extended(self, config):
"""
- Check if proper combinations of DHCPv4 and DHCpv6 can be started
+ This checks that the config contains the core and one more component.
"""
- v4found = 0
- v6found = 0
-
- for pid in bob.processes:
- if (bob.processes[pid].name == "b10-dhcp4"):
- v4found += 1
- if (bob.processes[pid].name == "b10-dhcp6"):
- v6found += 1
-
- # there should be exactly one DHCPv4 daemon (if v4==True)
- # there should be exactly one DHCPv6 daemon (if v6==True)
- self.assertEqual(v4==True, v4found==1)
- self.assertEqual(v6==True, v6found==1)
-
- # Checks the processes started when starting neither auth nor resolver
- # is specified.
- def test_start_none(self):
- # Create BoB and ensure correct initialization
- bob = MockBob()
- self.check_preconditions(bob)
-
- # Start processes and check what was started
- bob.cfg_start_auth = False
- bob.cfg_start_resolver = False
-
- bob.start_all_processes()
- self.check_started_none(bob)
-
- # Checks the processes started when starting only the auth process
- def test_start_auth(self):
- # Create BoB and ensure correct initialization
- bob = MockBob()
- self.check_preconditions(bob)
-
- # Start processes and check what was started
- bob.cfg_start_auth = True
- bob.cfg_start_resolver = False
-
- bob.start_all_processes()
-
- self.check_started_auth(bob)
-
- # Checks the processes started when starting only the resolver process
- def test_start_resolver(self):
- # Create BoB and ensure correct initialization
- bob = MockBob()
- self.check_preconditions(bob)
-
- # Start processes and check what was started
- bob.cfg_start_auth = False
- bob.cfg_start_resolver = True
-
- bob.start_all_processes()
+ self.__check_core(config)
+ self.assertTrue('comp' in config)
+ self.assertEqual('cat', config['comp']['process'])
+ self.assertEqual('needed', config['comp']['kind'])
+ self.assertEqual(4, len(config))
- self.check_started_resolver(bob)
-
- # Checks the processes started when starting both auth and resolver process
- def test_start_both(self):
- # Create BoB and ensure correct initialization
- bob = MockBob()
- self.check_preconditions(bob)
-
- # Start processes and check what was started
- bob.cfg_start_auth = True
- bob.cfg_start_resolver = True
-
- bob.start_all_processes()
-
- self.check_started_both(bob)
-
- def test_config_start(self):
+ def test_correct_run(self):
"""
- Test that the configuration starts and stops processes according
- to configuration changes.
+ Test the situation when we run in usual scenario, nothing fails,
+ we just start, reconfigure and then stop peacefully.
"""
-
- # Create BoB and ensure correct initialization
bob = MockBob()
- self.check_preconditions(bob)
-
- # Start processes (nothing much should be started, as in
- # test_start_none)
- bob.cfg_start_auth = False
- bob.cfg_start_resolver = False
-
+ # Start it
+ orig = bob._component_configurator.startup
+ bob._component_configurator.startup = self.__unary_hook
bob.start_all_processes()
+ bob._component_configurator.startup = orig
+ self.__check_core(self.__param)
+ self.assertEqual(3, len(self.__param))
+
+ # Reconfigure it
+ self.__param = None
+ orig = bob._component_configurator.reconfigure
+ bob._component_configurator.reconfigure = self.__unary_hook
+ # Otherwise it does not work
bob.runnable = True
- self.check_started_none(bob)
-
- # Enable both at once
- bob.config_handler({'start_auth': True, 'start_resolver': True})
- self.check_started_both(bob)
-
- # Not touched by empty change
+ bob.config_handler({'components': self.__compconfig})
+ self.__check_extended(self.__param)
+ currconfig = self.__param
+ # If we reconfigure it, but it does not contain the components part,
+ # nothing is called
bob.config_handler({})
- self.check_started_both(bob)
-
- # Not touched by change to the same configuration
- bob.config_handler({'start_auth': True, 'start_resolver': True})
- self.check_started_both(bob)
-
- # Turn them both off again
- bob.config_handler({'start_auth': False, 'start_resolver': False})
- self.check_started_none(bob)
-
- # Not touched by empty change
- bob.config_handler({})
- self.check_started_none(bob)
-
- # Not touched by change to the same configuration
- bob.config_handler({'start_auth': False, 'start_resolver': False})
- self.check_started_none(bob)
-
- # Start and stop auth separately
- bob.config_handler({'start_auth': True})
- self.check_started_auth(bob)
-
- bob.config_handler({'start_auth': False})
- self.check_started_none(bob)
-
- # Start and stop resolver separately
- bob.config_handler({'start_resolver': True})
- self.check_started_resolver(bob)
-
- bob.config_handler({'start_resolver': False})
- self.check_started_none(bob)
-
- # Alternate
- bob.config_handler({'start_auth': True})
- self.check_started_auth(bob)
-
- bob.config_handler({'start_auth': False, 'start_resolver': True})
- self.check_started_resolver(bob)
-
- bob.config_handler({'start_auth': True, 'start_resolver': False})
- self.check_started_auth(bob)
-
- def test_config_start_once(self):
- """
- Tests that a process is started only once.
- """
- # Create BoB and ensure correct initialization
- bob = MockBob()
- self.check_preconditions(bob)
-
- # Start processes (both)
- bob.cfg_start_auth = True
- bob.cfg_start_resolver = True
-
- bob.start_all_processes()
- bob.runnable = True
- self.check_started_both(bob)
-
- bob.start_auth = lambda: self.fail("Started auth again")
- bob.start_xfrout = lambda: self.fail("Started xfrout again")
- bob.start_xfrin = lambda: self.fail("Started xfrin again")
- bob.start_zonemgr = lambda: self.fail("Started zonemgr again")
- bob.start_resolver = lambda: self.fail("Started resolver again")
-
- # Send again we want to start them. Should not do it, as they are.
- bob.config_handler({'start_auth': True})
- bob.config_handler({'start_resolver': True})
-
- def test_config_not_started_early(self):
- """
- Test that processes are not started by the config handler before
- startup.
+ self.assertEqual(self.__param, currconfig)
+ self.__param = None
+ bob._component_configurator.reconfigure = orig
+ # Check a configuration that messes up the core components is rejected.
+ compconf = dict(self.__compconfig)
+ compconf['msgq'] = { 'process': 'echo' }
+ # Is it OK to raise, or should it catch also and convert to error
+ # answer?
+ self.assertRaises(Exception, bob.config_handler,
+ {'components': compconf})
+
+ # Stop it
+ orig = bob._component_configurator.shutdown
+ bob._component_configurator.shutdown = self.__nullary_hook
+ self.__called = False
+ # We can't call shutdown, that one relies on the stuff in main
+ bob.stop_all_processes()
+ bob._component_configurator.shutdown = orig
+ self.assertTrue(self.__called)
+
+ def test_init_config(self):
+ """
+ Test initial configuration is loaded.
"""
bob = MockBob()
- self.check_preconditions(bob)
-
- bob.start_auth = lambda: self.fail("Started auth again")
- bob.start_xfrout = lambda: self.fail("Started xfrout again")
- bob.start_xfrin = lambda: self.fail("Started xfrin again")
- bob.start_zonemgr = lambda: self.fail("Started zonemgr again")
- bob.start_resolver = lambda: self.fail("Started resolver again")
-
- bob.config_handler({'start_auth': True, 'start_resolver': True})
-
- # Checks that DHCP (v4 and v6) processes are started when expected
- def test_start_dhcp(self):
-
- # Create BoB and ensure correct initialization
- bob = MockBob()
- self.check_preconditions(bob)
-
- # don't care about DNS stuff
- bob.cfg_start_auth = False
- bob.cfg_start_resolver = False
-
- # v4 and v6 disabled
- bob.cfg_start_dhcp6 = False
- bob.cfg_start_dhcp4 = False
- bob.start_all_processes()
- self.check_started_dhcp(bob, False, False)
-
- # v6 only enabled
- bob = MockBob()
- self.check_preconditions(bob)
- bob.cfg_start_dhcp6 = True
- bob.cfg_start_dhcp4 = False
+ # Start it
+ bob._component_configurator.reconfigure = self.__unary_hook
+ # We need to return the original read_bind10_config
+ bob.read_bind10_config = lambda: BoB.read_bind10_config(bob)
+ # And provide a session to read the data from
+ class CC:
+ pass
+ bob.ccs = CC()
+ bob.ccs.get_full_config = lambda: {'components': self.__compconfig}
bob.start_all_processes()
- self.check_started_dhcp(bob, False, True)
-
- # uncomment when dhcpv4 becomes implemented
- # v4 only enabled
- #bob.cfg_start_dhcp6 = False
- #bob.cfg_start_dhcp4 = True
- #self.check_started_dhcp(bob, True, False)
-
- # both v4 and v6 enabled
- #bob.cfg_start_dhcp6 = True
- #bob.cfg_start_dhcp4 = True
- #self.check_started_dhcp(bob, True, True)
+ self.__check_extended(self.__param)
class TestBossCmd(unittest.TestCase):
def test_ping(self):
@@ -623,6 +343,7 @@ class TestBossCmd(unittest.TestCase):
Confirm getting a list of processes works.
"""
bob = MockBob()
+ bob.processes = {}
answer = bob.command_handler("show_processes", None)
self.assertEqual(answer, {'result': [0, []]})
@@ -633,16 +354,8 @@ class TestBossCmd(unittest.TestCase):
bob = MockBob()
bob.start_all_processes()
answer = bob.command_handler("show_processes", None)
- processes = [[2, 'b10-msgq'],
- [3, 'b10-cfgmgr'],
- [4, 'b10-ccsession'],
- [5, 'b10-auth'],
- [7, 'b10-xfrout'],
- [8, 'b10-xfrin'],
- [9, 'b10-zonemgr'],
- [10, 'b10-stats'],
- [11, 'b10-stats-httpd'],
- [12, 'b10-cmdctl']]
+ processes = [[1, 'first'],
+ [2, 'second']]
self.assertEqual(answer, {'result': [0, processes]})
class TestParseArgs(unittest.TestCase):
@@ -752,6 +465,8 @@ class TestPIDFile(unittest.TestCase):
self.assertRaises(IOError, dump_pid,
'nonexistent_dir' + os.sep + 'bind10.pid')
+# TODO: Do we want brittle mode? Probably yes. So we need to re-enable to after that.
+ at unittest.skip("Brittle mode temporarily broken")
class TestBrittle(unittest.TestCase):
def test_brittle_disabled(self):
bob = MockBob()
diff --git a/src/lib/python/isc/bind10/component.py b/src/lib/python/isc/bind10/component.py
index 208b3d4..c0f2ba4 100644
--- a/src/lib/python/isc/bind10/component.py
+++ b/src/lib/python/isc/bind10/component.py
@@ -14,12 +14,15 @@
# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
import isc.bind10.sockcreator
+import isc.log
from isc.log_messages.bind10_messages import *
import time
from bind10_config import LIBEXECDIR
import os
logger = isc.log.Logger("boss")
+DBG_TRACE_DATA = 20
+DBG_TRACE_DETAILED = 80
"""
Module for managing components (abstraction of process). It allows starting
@@ -58,6 +61,12 @@ class Component:
* 'dispensable' means the component should be running, but if it
doesn't start or crashes for some reason, the system simply tries
to restart it and keeps running.
+ - `address` is the address on message bus. It is used to ask it to
+ shut down at the end. If you specialize the class for a component
+ that is shut down differently, it might be None.
+ - `params` is a list of parameters to pass to the process when it
+ starts. It is currently unused and this support is left out for
+ now.
"""
if kind not in ['core', 'needed', 'dispensable']:
raise ValueError('Component kind can not be ' + kind)
@@ -83,11 +92,13 @@ class Component:
raise ValueError("Can't resurrect already dead component")
if self.running():
raise ValueError("Can't start already running component")
+ logger.info(BIND10_COMPONENT_START, self.name())
self.__running = True
self.__start_time = time.time()
try:
self.start_internal()
- except:
+ except Exception as e:
+ logger.error(BIND10_COMPONENT_START_EXCEPTION, self.name(), e)
self.failed()
raise
@@ -96,15 +107,20 @@ class Component:
This method does the actual starting of a process. If you need to
change the way the component is started, replace this method.
"""
+ # This one is not tested. For one, it starts a real process
+ # which is out of scope of unit tests, for another, it just
+ # delegates the starting to other function in boss (if a derived
+ # class does not provide an override function), which is tested
+ # by use.
if self._start_func is not None:
procinfo = self._start_func()
else:
# TODO Handle params, etc
procinfo = self._boss.start_simple(self._process)
self._procinfo = procinfo
- self._boss.register_process(procinfo.pid, self)
+ self._boss.register_process(self.pid(), self)
- def stop(self, kill=False):
+ def stop(self):
"""
Stop the component. If you need to modify the way a component is
stopped, do not replace this method, but stop_internal. This one
@@ -113,12 +129,15 @@ class Component:
If you try to stop a component that is not running, it raises
ValueError.
"""
+ # This is not tested. It talks with the outher world, which is out
+ # of scope of unittests.
if not self.running():
raise ValueError("Can't stop a component which is not running")
- self.stop_internal(kill)
+ logger.info(BIND10_COMPONENT_STOP, self.name())
+ self.stop_internal()
self.__running = False
- def stop_internal(self, kill=False):
+ def stop_internal(self):
"""
This is the method that does the actual stopping of a component.
You can replace this method if you want a different way to do it.
@@ -134,16 +153,18 @@ class Component:
"""
if not self.running():
raise ValueError("Can't fail component that isn't running")
- self.failed_internal()
self.__running = False
+ self.failed_internal()
# If it is a core component or the needed component failed to start
# (including it stopped really soon)
if self.__kind == 'core' or \
(self.__kind == 'needed' and time.time() - 10 < self.__start_time):
self.__dead = True
- self._boss.shutdown(1)
+ logger.fatal(BIND10_COMPONENT_UNSATISFIED, self.name())
+ self._boss.component_shutdown(1)
# This means we want to restart
else:
+ logger.warn(BIND10_COMPONENT_RESTART, self.name())
self.start()
def failed_internal(self):
@@ -165,25 +186,57 @@ class Component:
"""
return self.__running
+ def name(self):
+ """
+ Returns human-readable name of the component. This is usually the
+ name of the executable, but it might be something different in a
+ derived class.
+ """
+ return self._process
+
+ def pid(self):
+ """
+ Provides a PID of a process, if the component is real running process.
+ This implementation expects it to be a real process, but derived class
+ may return None in case the component is something else.
+ """
+ return self._procinfo.pid
+
+# These are specialized components. Some of them are components which need
+# special care (like the message queue or socket creator) or they need
+# some parameters constructed from Boss's command line. They are not tested
+# currently, because it is not clear what to test on them anyway and they just
+# delegate the work for the boss
class SockCreator(Component):
+ """
+ The socket creator component. Will start and stop the socket creator
+ accordingly.
+ """
def start_internal(self):
self._boss.curproc = 'b10-sockcreator'
- print( LIBEXECDIR)
self.__creator = isc.bind10.sockcreator.Creator(LIBEXECDIR + ':' +
os.environ['PATH'])
- self._boss.register_process(self.__creator.pid(), self)
+ self._boss.register_process(self.pid(), self)
- def stop_internal(self, kill=False):
+ def stop_internal(self):
if self.__creator is None:
return
- if kill:
- self.__creator.kill()
- else:
- self.sockcreator.terminate()
+ self.__creator.terminate()
self.__creator = None
+ def pid(self):
+ """
+ Pid of the socket creator. It is provided differently from a usual
+ component.
+ """
+ return self.__creator.pid()
+
class Msgq(Component):
- def __init__(self, process, boss, kind):
+ """
+ The message queue. Starting is passed to boss, stopping is not supported
+ and we leave the boss kill it by signal.
+ """
+ def __init__(self, process, boss, kind, address, params):
Component.__init__(self, process, boss, kind)
self._start_func = boss.start_msgq
@@ -191,25 +244,25 @@ class Msgq(Component):
pass # Wait for the boss to actually kill it. There's no stop command.
class CfgMgr(Component):
- def __init__(self, process, boss, kind):
+ def __init__(self, process, boss, kind, address, params):
Component.__init__(self, process, boss, kind)
self._start_func = boss.start_cfgmgr
self._address = 'ConfigManager'
class Auth(Component):
- def __init__(self, process, boss, kind):
+ def __init__(self, process, boss, kind, address, params):
Component.__init__(self, process, boss, kind)
self._start_func = boss.start_auth
self._address = 'Auth'
class Resolver(Component):
- def __init__(self, process, boss, kind):
+ def __init__(self, process, boss, kind, address, params):
Component.__init__(self, process, boss, kind)
self._start_func = boss.start_resolver
self._address = 'Resolver'
class CmdCtl(Component):
- def __init__(self, process, boss, kind):
+ def __init__(self, process, boss, kind, address, params):
Component.__init__(self, process, boss, kind)
self._start_func = boss.start_cmdctl
self._address = 'Cmdctl'
@@ -269,6 +322,7 @@ class Configurator:
if self._running:
raise ValueError("Trying to start the component configurator " +
"twice")
+ logger.info(BIND10_CONFIGURATOR_START)
self.__reconfigure_internal({}, configuration)
self._running = True
@@ -279,6 +333,7 @@ class Configurator:
if not self._running:
raise ValueError("Trying to shutdown the component " +
"configurator while it's not yet running")
+ logger.info(BIND10_CONFIGURATOR_STOP)
self.__reconfigure_internal(self._old_config, {})
self._running = False
@@ -290,6 +345,7 @@ class Configurator:
if not self._running:
raise ValueError("Trying to reconfigure the component " +
"configurator while it's not yet running")
+ logger.info(BIND10_CONFIGURATOR_RECONFIGURE)
self.__reconfigure_internal(self._old_config, configuration)
def _build_plan(self, old, new):
@@ -303,6 +359,7 @@ class Configurator:
Any configuration problems are expected to be handled here, so the
plan is not yet run.
"""
+ logger.debug(DBG_TRACE_DATA, BIND10_CONFIGURATOR_BUILD, old, new)
plan = []
# Handle removals of old components
for cname in old.keys():
@@ -334,13 +391,14 @@ class Configurator:
# TODO: Better error handling
creator = specials[params['special']]
component = creator(params.get('process', cname), self.__boss,
- params['kind'])
+ params['kind'], params.get('address'),
+ params.get('params'))
priority = params.get('priority', 0)
# We store tuples, priority first, so we can easily sort
plan_add.append((priority, {
'component': component,
'command': 'start',
- 'name': cname
+ 'name': cname,
}))
# Push the starts there sorted by priority
plan.extend([command for (_, command) in sorted(plan_add,
@@ -361,9 +419,12 @@ class Configurator:
* start
* stop
"""
+ logger.debug(DBG_TRACE_DATA, BIND10_CONFIGURATOR_RUN, len(plan))
for task in plan:
component = task['component']
command = task['command']
+ logger.debug(DBG_TRACE_DETAILED, BIND10_CONFIGURATOR_TASK, command,
+ component.name())
if command == 'start':
component.start()
self._components[task['name']] = component
diff --git a/src/lib/python/isc/bind10/tests/component_test.py b/src/lib/python/isc/bind10/tests/component_test.py
index bb96937..7b4d72d 100644
--- a/src/lib/python/isc/bind10/tests/component_test.py
+++ b/src/lib/python/isc/bind10/tests/component_test.py
@@ -51,7 +51,7 @@ class BossUtils:
# Return the original time function
isc.bind10.component.time.time = self.__orig_time
- def shutdown(self, exitcode=0):
+ def component_shutdown(self, exitcode=0):
"""
Mock function to shut down. We just note we were asked to do so.
"""
@@ -89,7 +89,7 @@ class ComponentTests(BossUtils, unittest.TestCase):
"""
self.__start_called = True
- def __stop(self, kill=False):
+ def __stop(self):
"""
Mock function, installed into the component into stop_internal.
This only notes the component was "stopped".
@@ -124,12 +124,30 @@ class ComponentTests(BossUtils, unittest.TestCase):
The process used is some nonsense, as this isn't used in this
kind of tests and we pretend to be the boss.
"""
- component = Component('No process', self, kind)
+ component = Component('No process', self, kind, 'homeless', [])
component.start_internal = self.__start
component.stop_internal = self.__stop
component.failed_internal = self.__fail
return component
+ def test_name(self):
+ """
+ Test the name provides whatever we passed to the constructor as process.
+ """
+ component = self.__create_component('core')
+ self.assertEqual('No process', component.name())
+
+ def test_guts(self):
+ """
+ Test the correct data are stored inside the component.
+ """
+ component = self.__create_component('core')
+ self.assertEqual(self, component._boss)
+ self.assertEqual("No process", component._process)
+ self.assertEqual(None, component._start_func)
+ self.assertEqual("homeless", component._address)
+ self.assertEqual([], component._params)
+
def __check_startup(self, component):
"""
Check that nothing was called yet. A newly created component should
@@ -225,11 +243,13 @@ class ComponentTests(BossUtils, unittest.TestCase):
A start-stop test for core component. See do_start_stop.
"""
self.__do_start_stop('core')
+
def test_start_stop_needed(self):
"""
A start-stop test for needed component. See do_start_stop.
"""
self.__do_start_stop('needed')
+
def test_start_stop_dispensable(self):
"""
A start-stop test for dispensable component. See do_start_stop.
@@ -373,14 +393,14 @@ class TestComponent(Component):
A test component. It does not start any processes or so, it just logs
information about what happens.
"""
- def __init__(self, owner, name, kind):
+ def __init__(self, owner, name, kind, address=None, params=None):
"""
Initializes the component. The owner is the test that started the
component. The logging will happen into it.
The process is used as a name for the logging.
"""
- Component.__init__(self, name, owner, kind)
+ Component.__init__(self, name, owner, kind, address, params)
self.__owner = owner
self.__name = name
self.log('init')
@@ -396,7 +416,7 @@ class TestComponent(Component):
def start_internal(self):
self.log('start')
- def stop_internal(self, kill=False):
+ def stop_internal(self):
self.log('stop')
def failed_internal(self):
@@ -464,12 +484,12 @@ class ConfiguratorTest(BossUtils, unittest.TestCase):
BossUtils.tearDown(self)
specials = self.__orig_specials
- def __component_test(self, process, boss, kind):
+ def __component_test(self, process, boss, kind, address=None, params=None):
"""
Create a test component. It will log events to us.
"""
self.assertEqual(self, boss)
- return TestComponent(self, process, kind)
+ return TestComponent(self, process, kind, address, params)
def test_init(self):
"""
@@ -539,6 +559,9 @@ class ConfiguratorTest(BossUtils, unittest.TestCase):
self.assertTrue('component' in task)
self.assertEqual('start', task['command'])
self.assertEqual(name, task['name'])
+ component = task['component']
+ self.assertIsNone(component._address)
+ self.assertIsNone(component._params)
# A plan to go from older state to newer one containing more components
bigger = copy.copy(self.__core)
@@ -592,6 +615,25 @@ class ConfiguratorTest(BossUtils, unittest.TestCase):
self.assertEqual('another', plan[1]['name'])
self.assertTrue('component' in plan[1])
+ # Some slightly insane plans, like missing process, having parameters,
+ # no special, etc
+ plan = configurator._build_plan({}, {
+ 'component': {
+ 'kind': 'needed',
+ 'params': [1, 2],
+ 'address': 'address'
+ }
+ })
+ self.assertEqual(1, len(plan))
+ self.assertEqual('start', plan[0]['command'])
+ self.assertEqual('component', plan[0]['name'])
+ component = plan[0]['component']
+ self.assertEqual('component', component.name())
+ self.assertEqual([1, 2], component._params)
+ self.assertEqual('address', component._address)
+ # We don't use isinstance on purpose, it would allow a descendand
+ self.assertTrue(type(component) is Component)
+
def __do_switch(self, option, value):
"""
Start it with some component and then switch the configuration of the
@@ -654,6 +696,7 @@ class ConfiguratorTest(BossUtils, unittest.TestCase):
# Can't reconfigure nor stop yet
self.assertRaises(ValueError, configurator.reconfigure, self.__core)
self.assertRaises(ValueError, configurator.shutdown)
+ self.assertFalse(configurator.running())
# Start it
configurator.startup(self.__core)
self.assertEqual(self.__core_log, self.log)
@@ -661,6 +704,7 @@ class ConfiguratorTest(BossUtils, unittest.TestCase):
self.assertTrue(core in configurator._components)
self.assertEqual(self.__core, configurator._old_config)
self.assertTrue(configurator._running)
+ self.assertTrue(configurator.running())
# It can't be started twice
self.assertRaises(ValueError, configurator.startup, self.__core)
@@ -687,6 +731,7 @@ class ConfiguratorTest(BossUtils, unittest.TestCase):
self.assertEqual({}, configurator._components)
self.assertEqual({}, configurator._old_config)
self.assertFalse(configurator._running)
+ self.assertFalse(configurator.running())
self.__check_shutdown_log()
# It can't be stopped twice
More information about the bind10-changes
mailing list