BIND 10 trac2380, updated. 663bd7310eadfe3164d7504c1eceef716eb1086d [2380] cleanup: removed redundant white space
BIND 10 source code commits
bind10-changes at lists.isc.org
Sat Dec 15 23:49:48 UTC 2012
The branch, trac2380 has been updated
via 663bd7310eadfe3164d7504c1eceef716eb1086d (commit)
via 7898eed5a8cf9888d02dea2e3a7519b6148a8efe (commit)
via c2d65ae44cf2c167de0b6892359b72e54d8b81ba (commit)
via e3b18efc1d76c65fe1bdec7d6eb4821f3745cd52 (commit)
via 8b805bcd111bfeca94c98f60bd629387fbc7d342 (commit)
via 935c918578e428b5d24718e2e113dc706ac952da (commit)
via 0785c84f4f92d3aba3be9ee88a431ec7173929fa (commit)
via 9d1e869ba98529030a65359ecaeb2273cd2cb14f (commit)
via 0f4a4a3b12d7a1730bbf79079495eb397a72de08 (commit)
via 9e7a7d316106c60fdd3e3ec4ff85f7d3a10035a1 (commit)
via 605e81588c922ad5e93f87b56da722d79075993c (commit)
via d08ad5667f5b4fe540a1a00921e8a10d706cd88f (commit)
via 790d0b5527a8fd832f7236e17b4ac1274b865901 (commit)
from 52f3401fb59545555189a456f98fddbb2001b66d (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 663bd7310eadfe3164d7504c1eceef716eb1086d
Author: JINMEI Tatuya <jinmei at isc.org>
Date: Sat Dec 15 15:49:47 2012 -0800
[2380] cleanup: removed redundant white space
commit 7898eed5a8cf9888d02dea2e3a7519b6148a8efe
Author: JINMEI Tatuya <jinmei at isc.org>
Date: Sat Dec 15 15:47:55 2012 -0800
[2380] reordered log messages
commit c2d65ae44cf2c167de0b6892359b72e54d8b81ba
Author: JINMEI Tatuya <jinmei at isc.org>
Date: Sat Dec 15 15:47:11 2012 -0800
[2380] filled in log message descriptions
commit e3b18efc1d76c65fe1bdec7d6eb4821f3745cd52
Author: JINMEI Tatuya <jinmei at isc.org>
Date: Sat Dec 15 15:33:50 2012 -0800
[2380] updated TODO, removing things now supported (or soon to be supported)
commit 8b805bcd111bfeca94c98f60bd629387fbc7d342
Author: JINMEI Tatuya <jinmei at isc.org>
Date: Sat Dec 15 15:32:28 2012 -0800
[2380] adjusted bind10-guide so it matches the new b10-loadzone
commit 935c918578e428b5d24718e2e113dc706ac952da
Author: JINMEI Tatuya <jinmei at isc.org>
Date: Sat Dec 15 15:26:57 2012 -0800
[2380] updated man page.
commit 0785c84f4f92d3aba3be9ee88a431ec7173929fa
Author: JINMEI Tatuya <jinmei at isc.org>
Date: Sat Dec 15 14:22:23 2012 -0800
[2380] handle signals
commit 9d1e869ba98529030a65359ecaeb2273cd2cb14f
Author: JINMEI Tatuya <jinmei at isc.org>
Date: Sat Dec 15 13:31:42 2012 -0800
[2380] always use incremental load so we can handle emergency exit (eg signals)
still support no report mode.
commit 0f4a4a3b12d7a1730bbf79079495eb397a72de08
Author: JINMEI Tatuya <jinmei at isc.org>
Date: Sat Dec 15 13:19:24 2012 -0800
[2380] dump progress reports "on the same line", just like the old loadzone
commit 9e7a7d316106c60fdd3e3ec4ff85f7d3a10035a1
Author: JINMEI Tatuya <jinmei at isc.org>
Date: Sat Dec 15 13:08:28 2012 -0800
[2380] tighten option checks; make report interval configurable.
commit 605e81588c922ad5e93f87b56da722d79075993c
Author: JINMEI Tatuya <jinmei at isc.org>
Date: Sat Dec 15 12:24:37 2012 -0800
[2380] changed the default log output to stderr so we can separate them
commit d08ad5667f5b4fe540a1a00921e8a10d706cd88f
Author: JINMEI Tatuya <jinmei at isc.org>
Date: Sat Dec 15 12:12:03 2012 -0800
[2380] added minimal post-load zone checks
commit 790d0b5527a8fd832f7236e17b4ac1274b865901
Author: JINMEI Tatuya <jinmei at isc.org>
Date: Sat Dec 15 11:14:45 2012 -0800
[2380] added -d option to control logging level
-----------------------------------------------------------------------
Summary of changes:
doc/guide/bind10-guide.xml | 11 +-
src/bin/loadzone/TODO | 13 --
src/bin/loadzone/b10-loadzone.xml | 138 +++++++++++++++++---
src/bin/loadzone/loadzone.py.in | 138 ++++++++++++++++++--
src/bin/loadzone/loadzone_messages.mes | 43 +++++-
src/bin/loadzone/tests/Makefile.am | 2 +
src/bin/loadzone/tests/loadzone_test.py | 136 ++++++++++++++++++-
.../{example.org.zone => example-nons.org.zone} | 2 +-
.../loadzone/tests/testdata/example-nosoa.org.zone | 3 +
9 files changed, 423 insertions(+), 63 deletions(-)
copy src/bin/loadzone/tests/testdata/{example.org.zone => example-nons.org.zone} (73%)
create mode 100644 src/bin/loadzone/tests/testdata/example-nosoa.org.zone
-----------------------------------------------------------------------
diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml
index 61a9ee4..72892d9 100644
--- a/doc/guide/bind10-guide.xml
+++ b/doc/guide/bind10-guide.xml
@@ -449,7 +449,7 @@ var/
<listitem>
<para>Load desired zone file(s), for example:
- <screen>$ <userinput>b10-loadzone <replaceable>your.zone.example.org</replaceable></userinput></screen>
+ <screen>$ <userinput>b10-loadzone <replaceable>-c '{"database_file": "/usr/local/var/bind10/zone.sqlite3"}'</replaceable> <replaceable>your.zone.example.org</replaceable> <replaceable>your.zone.file</replaceable></userinput></screen>
</para>
</listitem>
@@ -2636,19 +2636,10 @@ can use various data source backends.
</para>
- <para>
- The <option>-o</option> argument may be used to define the
- default origin for loaded zone file records.
- </para>
-
<note>
<para>
In the current release, only the SQLite3 back
end is used by <command>b10-loadzone</command>.
- By default, it stores the zone data in
- <filename>/usr/local/var/bind10-devel/zone.sqlite3</filename>
- unless the <option>-d</option> switch is used to set the
- database filename.
Multiple zones are stored in a single SQLite3 zone database.
</para>
</note>
diff --git a/src/bin/loadzone/TODO b/src/bin/loadzone/TODO
index d8d5f24..a33385d 100644
--- a/src/bin/loadzone/TODO
+++ b/src/bin/loadzone/TODO
@@ -1,16 +1,3 @@
-Support optional origin in $INCLUDE:
-$INCLUDE filename origin
-
-Support optional comment in $INCLUDE:
-$INCLUDE filename origin comment
-
-Support optional comment in $TTL (RFC 2308):
-$TTL number comment
-
-Do not assume "." is origin if origin is not set and sees a @ or
-a label without a ".". It should probably fail. (Don't assume a
-mistake means it is a root level label.)
-
Add verbose option to show what it is adding, not necessarily
in master file format, but in the context of the data source.
diff --git a/src/bin/loadzone/b10-loadzone.xml b/src/bin/loadzone/b10-loadzone.xml
index 8c41e54..5174f44 100644
--- a/src/bin/loadzone/b10-loadzone.xml
+++ b/src/bin/loadzone/b10-loadzone.xml
@@ -2,7 +2,7 @@
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
[<!ENTITY mdash "—">]>
<!--
- - Copyright (C) 2010 Internet Systems Consortium, Inc. ("ISC")
+ - Copyright (C) 2012 Internet Systems Consortium, Inc. ("ISC")
-
- Permission to use, copy, modify, and/or distribute this software for any
- purpose with or without fee is hereby granted, provided that the above
@@ -20,7 +20,7 @@
<refentry>
<refentryinfo>
- <date>March 26, 2012</date>
+ <date>December 15, 2012</date>
</refentryinfo>
<refmeta>
@@ -36,7 +36,7 @@
<docinfo>
<copyright>
- <year>2010</year>
+ <year>2012</year>
<holder>Internet Systems Consortium, Inc. ("ISC")</holder>
</copyright>
</docinfo>
@@ -44,9 +44,13 @@
<refsynopsisdiv>
<cmdsynopsis>
<command>b10-loadzone</command>
- <arg><option>-d <replaceable class="parameter">database</replaceable></option></arg>
- <arg><option>-o <replaceable class="parameter">origin</replaceable></option></arg>
- <arg choice="req">filename</arg>
+ <arg><option>-d <replaceable class="parameter">debug_level</replaceable></option></arg>
+ <arg><option>-i <replaceable class="parameter">report_interval</replaceable></option></arg>
+ <arg><option>-t <replaceable class="parameter">datasrc_type</replaceable></option></arg>
+ <arg><option>-C <replaceable class="parameter">zone_class</replaceable></option></arg>
+ <arg choice="req">-c <replaceable class="parameter">datasrc_config</replaceable></arg>
+ <arg choice="req">zone name</arg>
+ <arg choice="req">zone file</arg>
</cmdsynopsis>
</refsynopsisdiv>
@@ -66,8 +70,6 @@
$ORIGIN is followed by a domain name, and sets the the origin
that will be used for relative domain names in subsequent records.
$INCLUDE is followed by a filename to load.
-<!-- TODO: and optionally a
- domain name used to set the relative domain name origin. -->
The previous origin is restored after the file is included.
<!-- the current domain name is also restored -->
$TTL is followed by a time-to-live value which is used
@@ -75,11 +77,31 @@
</para>
<para>
+ If the specified zone does not exist in the specified data
+ source, <command>b10-loadzone</command> will first create a
+ new empty zone in the data source, then fill it with the RRs
+ given in the specified master zone file. In this case, if
+ loading fails for some reason, the creation of the new zone
+ is also canceled.
+ <note><simpara>
+ Due to an implementation limitation, the current version
+ does not make the zone creation and subsequent loading an
+ atomic operation; an empty zone will be visible and used by
+ other application (e.g., the <command>b10-auth</command>
+ authoritative server) while loading. If this is an issue,
+ make sure the initial loading of a new zone is done before
+ starting other BIND 10 applications.
+ </simpara></note>
+ </para>
+
+ <para>
When re-loading an existing zone, the prior version is completely
removed. While the new version of the zone is being loaded, the old
version remains accessible to queries. After the new version is
completely loaded, the old version is swapped out and replaced
- with the new one in a single operation.
+ with the new one in a single operation. If loading fails for
+ some reason, the loaded RRs will be effectively deleted, and the
+ old version will still remain accessible for other applications.
</para>
</refsect1>
@@ -90,19 +112,77 @@
<variablelist>
<varlistentry>
- <term>-d <replaceable class="parameter">database</replaceable> </term>
+ <term>-d <replaceable class="parameter">debug_level</replaceable> </term>
<listitem><para>
- Defines the filename for the database.
- The default is
- <filename>/usr/local/var/bind10-devel/zone.sqlite3</filename>.
-<!-- TODO: fix filename -->
+ Enable dumping debug level logging with the specified
+ level. By default, only log messages at the severity of
+ informational or higher levels will be produced.
</para></listitem>
</varlistentry>
<varlistentry>
- <term>-o <replaceable class="parameter">origin</replaceable></term>
+ <term>-i <replaceable class="parameter">report_interval</replaceable></term>
<listitem><para>
- Defines the default origin for the zone file records.
+ Specifies the interval of status update by the number of RRs
+ loaded in the interval.
+ The <command>b10-loadzone</command> tool periodically
+ reports the progress of loading with the total number of
+ loaded RRs and elapsed time. This option specifies the
+ interval of the reports. If set to 0, status reports will
+ be suppressed. The default is 10,000.
+ </para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-t <replaceable class="parameter">datasrc_type</replaceable></term>
+ <listitem><para>
+ Specifies the type of data source to store the zone.
+ Currently, only the "sqlite3" type is supported (which is
+ the default of this option), which means the SQLite3 data
+ source.
+ </para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-C <replaceable class="parameter">zone_class</replaceable></term>
+ <listitem><para>
+ Specifies the RR class of the zone.
+ Currently, only class IN is supported (which is the default
+ of this option) due to limitation of the underlying data
+ source implementation.
+ </para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-c <replaceable class="parameter">datasrc_config</replaceable></term>
+ <listitem><para>
+ Specifies configuration of the data source in the JSON
+ format. The configuration contents depend on the type of
+ the data source, and that's the same as what would be
+ specified for the BIND 10 servers (see the data source
+ configuration section of the BIND 10 guide). For example,
+ for an SQLite3 data source, it would look like
+ '{"database_file": "path-to-sqlite3-db-file"}'.
+ <note>
+ <simpara>This option currently cannot be omitted. In a future
+ version it will be possible to retrieve the configuration from
+ the BIND 10 server configuration (if it exists).
+ </simpara></note>
+ </para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">zone name</replaceable></term>
+ <listitem><para>
+ The name of the zone to create or update. This must be a valid DNS
+ domain name.
+ </para></listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">zone file</replaceable></term>
+ <listitem><para>
+ A path to the master zone file to be loaded.
</para></listitem>
</varlistentry>
@@ -131,8 +211,30 @@
<refsect1>
<title>AUTHORS</title>
<para>
- The <command>b10-loadzone</command> tool was initial written
- by Evan Hunt of ISC.
+ A prior version of the <command>b10-loadzone</command> tool was
+ written by Evan Hunt of ISC.
+ The new version that this manual refers to was rewritten from
+ the scratch by the BIND 10 development team in around December 2012.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>BUGS</title>
+ <para>
+ As of the initial implementation, the underlying library that
+ this tool uses does not fully validate the loaded zone; for
+ example, loading will succeed even if it doesn't have the SOA or
+ NS record at its origin name. Such checks will be implemented
+ in a near future version, but until then, the
+ <command>b10-loadzone</command> performs the existence of the
+ SOA and NS records by itself. However, <command>b10-loadzone</command>
+ only warns about it, and does not cancel the load itself.
+ If this warning message is produced, it's the user's
+ responsibility to fix the errors and reload it. When the
+ library is updated with the post load checks, it will be more
+ sophisticated and the such zone won't be successfully loaded.
+
+ There are some other issues noted in the DESCRIPTION section.
</para>
</refsect1>
</refentry><!--
diff --git a/src/bin/loadzone/loadzone.py.in b/src/bin/loadzone/loadzone.py.in
index c6c846c..5b8f69d 100755
--- a/src/bin/loadzone/loadzone.py.in
+++ b/src/bin/loadzone/loadzone.py.in
@@ -17,15 +17,28 @@
import sys
sys.path.append('@@PYTHONPATH@@')
+import time
+import signal
from optparse import OptionParser
from isc.dns import *
from isc.datasrc import *
import isc.log
from isc.log_messages.loadzone_messages import *
-isc.log.init("b10-loadzone", buffer=False)
+# These are needed for logger settings
+import bind10_config
+import json
+from isc.config import module_spec_from_file
+from isc.config.ccsession import path_search
+
+isc.log.init("b10-loadzone")
logger = isc.log.Logger("loadzone")
+# The default value for the interval of progress report in terms of the
+# number of RRs loaded in that interval. Arbitrary choice, but intended to
+# be reasonably small to handle emergency exit.
+LOAD_INTERVAL_DEFAULT = 10000
+
class BadArgument(Exception):
'''An exception indicating an error in command line argument.
@@ -47,12 +60,22 @@ def set_cmd_options(parser):
the zone in. Example:
'{"database_file": "/path/to/dbfile/db.sqlite3"}'""",
metavar='CONFIG')
+ parser.add_option("-d", "--debug", dest="debug_level",
+ type='int', action="store", default=None,
+ help="enable debug logs with the specified level")
+ parser.add_option("-i", "--report-interval", dest="report_interval",
+ type='int', action="store",
+ default=LOAD_INTERVAL_DEFAULT,
+ help="""report logs progress per specified number of RRs
+(specify 0 to suppress report) [default: %default]""")
parser.add_option("-t", "--datasrc-type", dest="datasrc_type",
action="store", default='sqlite3',
- help="type of data source (e.g., 'sqlite3')")
+ help="""type of data source (e.g., 'sqlite3')\n
+[default: %default]""")
parser.add_option("-C", "--class", dest="zone_class", action="store",
default='IN',
- help="RR class of the zone; currently must be 'IN'")
+ help="""RR class of the zone; currently must be 'IN'
+[default: %default]""")
class LoadZoneRunner:
'''Main logic for the loadzone.
@@ -62,8 +85,21 @@ class LoadZoneRunner:
'''
def __init__(self, command_args):
self.__command_args = command_args
- self.__load_iteration_limit = 100000 # arbitrary choice for now
self.__loaded_rrs = 0
+ self.__interrupted = False # will be set to True on receiving signal
+
+ # system-wide log configuration. We need to configure logging this
+ # way so that the logging policy applies to underlying libraries, too.
+ self.__log_spec = json.dumps(isc.config.module_spec_from_file(
+ path_search('logging.spec', bind10_config.PLUGIN_PATHS)).
+ get_full_spec())
+ # "severity" and "debuglevel" are the tunable parameters, which will
+ # be set in _config_log().
+ self.__log_conf_base = {"loggers":
+ [{"name": "*",
+ "output_options":
+ [{"output": "stderr",
+ "destination": "console"}]}]}
# These are essentially private, and defined as "protected" for the
# convenience of tests inspecting them
@@ -72,6 +108,22 @@ class LoadZoneRunner:
self._zone_file = None
self._datasrc_config = None
self._datasrc_type = None
+ self._log_severity = 'INFO'
+ self._log_debuglevel = 0
+ self._load_iteration_limit = LOAD_INTERVAL_DEFAULT
+
+ self._config_log()
+
+ def _config_log(self):
+ '''Configure logging policy.
+
+ This is essentially private, but defined as "protected" for tests.
+
+ '''
+ self.__log_conf_base['loggers'][0]['severity'] = self._log_severity
+ self.__log_conf_base['loggers'][0]['debuglevel'] = self._log_debuglevel
+ isc.log.log_config_update(json.dumps(self.__log_conf_base),
+ self.__log_spec)
def _parse_args(self):
'''Parse command line options and other arguments.
@@ -80,11 +132,23 @@ class LoadZoneRunner:
'''
- usage_txt = 'usage: %prog [options] zonename zonefile'
+ usage_txt = \
+ 'usage: %prog [options] -c datasrc_config zonename zonefile'
parser = OptionParser(usage=usage_txt)
set_cmd_options(parser)
(options, args) = parser.parse_args(args=self.__command_args)
+ # Configure logging policy as early as possible
+ if options.debug_level is not None:
+ self._log_severity = 'DEBUG'
+ # optparse performs type check
+ self._log_debuglevel = int(options.debug_level)
+ if self._log_debuglevel < 0:
+ raise BadArgument(
+ 'Invalid debug level (must be non negative): %d' %
+ self._log_debuglevel)
+ self._config_log()
+
if options.conf is None:
raise BadArgument('data source config option cannot be omitted')
self._datasrc_config = options.conf
@@ -97,6 +161,12 @@ class LoadZoneRunner:
raise BadArgument("RR class is not supported: " +
str(self._zone_class))
+ self._load_iteration_limit = int(options.report_interval)
+ if self._load_iteration_limit < 0:
+ raise BadArgument(
+ 'Invalid report interval (must be non negative): %d' %
+ self._load_iteration_limit)
+
if len(args) != 2:
raise BadArgument('Unexpected number of arguments: %d (must be 2)'
% (len(args)))
@@ -130,6 +200,17 @@ class LoadZoneRunner:
cur.execute("DELETE FROM zones WHERE name = ?",
[self._zone_name.to_text()])
+ def _report_progress(self, loaded_rrs):
+ '''Dump the current progress report to stdout.
+
+ This is essentially private, but defined as "protected" for tests.
+
+ '''
+ elapsed = time.time() - self.__start_time
+ sys.stdout.write("\r" + (80 * " "))
+ sys.stdout.write("\r%d RRs loaded in %.2f seconds" %
+ (loaded_rrs, elapsed))
+
def _do_load(self):
'''Main part of the load logic.
@@ -146,10 +227,18 @@ class LoadZoneRunner:
self._zone_class)
loader = ZoneLoader(datasrc_client, self._zone_name,
self._zone_file)
- while not loader.load_incremental(self.__load_iteration_limit):
- self.__loaded_rrs += self.__load_iteration_limit
- logger.info(LOADZONE_LOADING, self.__loaded_rrs,
- self._zone_name, self._zone_class)
+ self.__start_time = time.time()
+ if self._load_iteration_limit > 0:
+ limit = self._load_iteration_limit
+ else:
+ limit = LOAD_INTERVAL_DEFAULT
+ while (not self.__interrupted and
+ not loader.load_incremental(limit)):
+ self.__loaded_rrs += self._load_iteration_limit
+ if self._load_iteration_limit > 0:
+ self._report_progress(self.__loaded_rrs)
+ if self.__interrupted:
+ raise LoadFailure('loading interrupted by signal')
except Exception as ex:
# release any remaining lock held in the client/loader
loader, datasrc_client = None, None
@@ -159,13 +248,44 @@ class LoadZoneRunner:
self._zone_class)
raise LoadFailure(str(ex))
+ def _post_load_checks(self):
+ '''Perform minimal validity checks on the loaded zone.
+
+ We do this ourselves because the underlying library currently
+ doesn't do any checks. Once the library support post-load validation
+ this check should be removed.
+
+ '''
+ datasrc_client = DataSourceClient(self._datasrc_type,
+ self._datasrc_config)
+ _, finder = datasrc_client.find_zone(self._zone_name) # should succeed
+ result = finder.find(self._zone_name, RRType.SOA())[0]
+ if result is not finder.SUCCESS:
+ self._post_load_warning('zone has no SOA')
+ result = finder.find(self._zone_name, RRType.NS())[0]
+ if result is not finder.SUCCESS:
+ self._post_load_warning('zone has no NS')
+
+ def _post_load_warning(self, msg):
+ logger.warn(LOADZONE_POSTLOAD_ISSUE, self._zone_name,
+ self._zone_class, msg)
+
+ def _set_signal_handlers(self):
+ signal.signal(signal.SIGINT, self._interrupt_handler)
+ signal.signal(signal.SIGTERM, self._interrupt_handler)
+
+ def _interrupt_handler(self, signal, frame):
+ self.__interrupted = True
+
def run(self):
'''Top-level method, simply calling other helpers'''
try:
+ self._set_signal_handlers()
self._parse_args()
self._do_load()
logger.info(LOADZONE_DONE, self._zone_name, self._zone_class)
+ self._post_load_checks()
return 0
except BadArgument as ex:
logger.error(LOADZONE_ARGUMENT_ERROR, ex)
diff --git a/src/bin/loadzone/loadzone_messages.mes b/src/bin/loadzone/loadzone_messages.mes
index 64a742f..3cbd6fd 100644
--- a/src/bin/loadzone/loadzone_messages.mes
+++ b/src/bin/loadzone/loadzone_messages.mes
@@ -17,15 +17,44 @@
# messages are in the correct order.
% LOADZONE_ARGUMENT_ERROR Error in command line arguments: %1
-
-% LOADZONE_ZONE_CREATED Zone %1/%2 does not exist in the data source, newly created
-
-% LOADZONE_LOAD_ERROR Failed to load zone %1/%2: %3
+Some semantics error in command line arguments or options to b10-loadzone
+is detected. b10-loadzone does effectively nothing and immediately
+terminates.
% LOADZONE_CANCEL_CREATE_ZONE Creation of new zone %1/%2 was canceled
+b10-loadzone has created a new zone in the data source (see
+LOADZONE_ZONE_CREATED), but the loading operation has subsequently
+failed. The newly created zone has been removed from the data source,
+so that the data source will go back to the original state.
-% LOADZONE_UNEXPECTED_FAILURE Unexpected exception: ex
+% LOADZONE_DONE Load zone %1/%2 completed
+b10-loadzone has successfully loaded the specified zone. If there was
+an old version of the zone in the data source, it is now deleted.
-% LOADZONE_LOADING Loaded %1 RRs into %2/%3, continued
+% LOADZONE_LOAD_ERROR Failed to load zone %1/%2: %3
+Loading a zone by b10-loadzone fails for some reason in the middle of
+the loading. This is most likely due to an error in the specified
+arguments to b10-loadzone (such as non-existent zone file) or an error
+in the zone file. When this happens, the RRs loaded so far are
+effectively deleted from the zone, and the old version (if exists)
+will still remain valid for operations.
+
+% LOADZONE_POSTLOAD_ISSUE New version of zone %1/%2 has an issue: %3
+b10-loadzone detected a problem after a successful load of zone:
+either or both of SOA and NS records are missing at the zone origin.
+In the current implementation the load will not be canceled for such
+problems. The operator will need to fix the issues and reload the
+zone; otherwise applications (such as b10-auth) that use this data
+source will not work as expected.
+
+% LOADZONE_UNEXPECTED_FAILURE Unexpected exception: %1
+b10-loadzone encounters an unexpected failure and terminates itself.
+This is generally a bug of b10-loadzone itself or the underlying
+data source library, so it's advisable to submit a bug report if
+this message is logged. The incomplete attempt of loading should
+have been cleanly canceled in this case, too.
-% LOADZONE_DONE Load zone %1/%2 completed
+% LOADZONE_ZONE_CREATED Zone %1/%2 does not exist in the data source, newly created
+The specified zone to b10-loadzone to load does not exist in the
+specified data source. b10-loadzone has created a new empty zone
+in the data source.
diff --git a/src/bin/loadzone/tests/Makefile.am b/src/bin/loadzone/tests/Makefile.am
index 641bc6e..1918bb5 100644
--- a/src/bin/loadzone/tests/Makefile.am
+++ b/src/bin/loadzone/tests/Makefile.am
@@ -4,6 +4,8 @@ PYTESTS = loadzone_test.py
EXTRA_DIST = $(PYTESTS)
EXTRA_DIST += testdata/example.org.zone
EXTRA_DIST += testdata/broken-example.org.zone
+EXTRA_DIST += testdata/example-nosoa.org.zone
+EXTRA_DIST += testdata/example-nons.org.zone
# If necessary (rare cases), explicitly specify paths to dynamic libraries
# required by loadable python modules.
diff --git a/src/bin/loadzone/tests/loadzone_test.py b/src/bin/loadzone/tests/loadzone_test.py
index 82c27bc..c9e258c 100755
--- a/src/bin/loadzone/tests/loadzone_test.py
+++ b/src/bin/loadzone/tests/loadzone_test.py
@@ -60,11 +60,16 @@ class TestLoadZoneRunner(unittest.TestCase):
def test_init(self):
'''
- Test the old socket file is removed (if any) and a new socket
- is created when the ddns server is created.
+ Checks initial class attributes
'''
self.assertIsNone(self.__runner._zone_class)
self.assertIsNone(self.__runner._zone_name)
+ self.assertIsNone(self.__runner._zone_file)
+ self.assertIsNone(self.__runner._datasrc_config)
+ self.assertIsNone(self.__runner._datasrc_type)
+ self.assertEqual(10000, self.__runner._load_iteration_limit)
+ self.assertEqual('INFO', self.__runner._log_severity)
+ self.assertEqual(0, self.__runner._log_debuglevel)
def test_parse_args(self):
self.__runner._parse_args()
@@ -72,7 +77,16 @@ class TestLoadZoneRunner(unittest.TestCase):
self.assertEqual(NEW_ZONE_TXT_FILE, self.__runner._zone_file)
self.assertEqual(DATASRC_CONFIG, self.__runner._datasrc_config)
self.assertEqual('sqlite3', self.__runner._datasrc_type) # default
+ self.assertEqual(10000, self.__runner._load_iteration_limit) # default
self.assertEqual(RRClass.IN(), self.__runner._zone_class) # default
+ self.assertEqual('INFO', self.__runner._log_severity) # default
+ self.assertEqual(0, self.__runner._log_debuglevel)
+
+ def test_set_loglevel(self):
+ runner = LoadZoneRunner(['-d', '1'] + self.__args)
+ runner._parse_args()
+ self.assertEqual('DEBUG', runner._log_severity)
+ self.assertEqual(1, runner._log_debuglevel)
def test_parse_bad_args(self):
# -c cannot be omitted (right now)
@@ -80,8 +94,9 @@ class TestLoadZoneRunner(unittest.TestCase):
LoadZoneRunner(['example', 'example.zone']).
_parse_args)
+ copt = ['-c', '0'] # template for the mandatory -c option
+
# There must be exactly 2 non-option arguments: zone name and zone file
- copt = ['-c', '0']
self.assertRaises(BadArgument, LoadZoneRunner(copt)._parse_args)
self.assertRaises(BadArgument, LoadZoneRunner(copt + ['example']).
_parse_args)
@@ -102,17 +117,31 @@ class TestLoadZoneRunner(unittest.TestCase):
LoadZoneRunner(copt + ['-C', 'CH']).
_parse_args)
+ # bad debug level
+ args = copt + ['example.org', 'example.zone'] # otherwise valid args
+ self.assertRaises(BadArgument,
+ LoadZoneRunner(['-d', '-10'] + args)._parse_args)
+
+ # bad report interval
+ self.assertRaises(BadArgument,
+ LoadZoneRunner(['-i', '-5'] + args)._parse_args)
+
def __common_load_setup(self):
self.__runner._zone_class = RRClass.IN()
self.__runner._zone_name = TEST_ZONE_NAME
self.__runner._zone_file = NEW_ZONE_TXT_FILE
self.__runner._datasrc_type = 'sqlite3'
self.__runner._datasrc_config = DATASRC_CONFIG
+ self.__runner._load_iteration_limit = 1
+ self.__reports = []
+ self.__runner._report_progress = lambda x: self.__reports.append(x)
def __check_zone_soa(self, soa_txt, zone_name=TEST_ZONE_NAME):
"""Check that the given SOA RR exists and matches the expected string
If soa_txt is None, the zone is expected to be non-existent.
+ Otherwise, if soa_txt is False, the zone should exist but SOA is
+ expected to be missing.
"""
@@ -123,16 +152,37 @@ class TestLoadZoneRunner(unittest.TestCase):
return
self.assertEqual(client.SUCCESS, result)
result, rrset, _ = finder.find(zone_name, RRType.SOA())
- self.assertEqual(finder.SUCCESS, result)
- self.assertEqual(soa_txt, rrset.to_text())
+ if soa_txt:
+ self.assertEqual(finder.SUCCESS, result)
+ self.assertEqual(soa_txt, rrset.to_text())
+ else:
+ self.assertEqual(finder.NXRRSET, result)
def test_load_update(self):
'''successful case to loading new contents to an existing zone.'''
self.__common_load_setup()
self.__check_zone_soa(ORIG_SOA_TXT)
self.__runner._do_load()
+ # In this test setup every loaded RR will be reported, and there will
+ # be 3 RRs
+ self.assertEqual([1, 2, 3], self.__reports)
self.__check_zone_soa(NEW_SOA_TXT)
+ def test_load_update_skipped_report(self):
+ '''successful loading, with reports for every 2 RRs'''
+ self.__common_load_setup()
+ self.__runner._load_iteration_limit = 2
+ self.__runner._do_load()
+ self.assertEqual([2], self.__reports)
+
+ def test_load_update_no_report(self):
+ '''successful loading, without progress reports'''
+ self.__common_load_setup()
+ self.__runner._load_iteration_limit = 0
+ self.__runner._do_load()
+ self.assertEqual([], self.__reports) # no report
+ self.__check_zone_soa(NEW_SOA_TXT) # but load is completed
+
def test_create_and_load(self):
'''successful case to loading contents to a new zone (created).'''
self.__common_load_setup()
@@ -178,6 +228,76 @@ class TestLoadZoneRunner(unittest.TestCase):
# _do_load() should have once created the zone but then canceled it.
self.__check_zone_soa(None, zone_name=Name('example.com'))
+ def __common_post_load_setup(self, zone_file):
+ '''Common setup procedure for post load tests.'''
+ # replace the LoadZoneRunner's original _post_load_warning() for
+ # inspection
+ self.__warnings = []
+ self.__runner._post_load_warning = \
+ lambda msg: self.__warnings.append(msg)
+
+ # perform load and invoke checks
+ self.__common_load_setup()
+ self.__runner._zone_file = zone_file
+ self.__check_zone_soa(ORIG_SOA_TXT)
+ self.__runner._do_load()
+ self.__runner._post_load_checks()
+
+ def test_load_post_check_fail_soa(self):
+ '''Load succeeds but warns about missing SOA, should cause warn'''
+ self.__common_load_setup()
+ self.__common_post_load_setup(LOCAL_TESTDATA_PATH +
+ '/example-nosoa.org.zone')
+ self.__check_zone_soa(False)
+ self.assertEqual(1, len(self.__warnings))
+ self.assertEqual('zone has no SOA', self.__warnings[0])
+
+ def test_load_post_check_fail_ns(self):
+ '''Load succeeds but warns about missing NS, should cause warn'''
+ self.__common_load_setup()
+ self.__common_post_load_setup(LOCAL_TESTDATA_PATH +
+ '/example-nons.org.zone')
+ self.__check_zone_soa(NEW_SOA_TXT)
+ self.assertEqual(1, len(self.__warnings))
+ self.assertEqual('zone has no NS', self.__warnings[0])
+
+ def __interrupt_progress(self, loaded_rrs):
+ '''A helper emulating a signal in the middle of loading.
+
+ On the second progress report, it internally invokes the signal
+ handler to see if it stops the loading.
+
+ '''
+ self.__reports.append(loaded_rrs)
+ if len(self.__reports) == 2:
+ self.__runner._interrupt_handler()
+
+ def test_load_interrupted(self):
+ '''Load attempt fails due to signal interruption'''
+ self.__common_load_setup()
+ self.__runner._report_progress = lambda x: self.__interrupt_progress(x)
+ # The interrupting _report_progress() will terminate the loading
+ # in the middle. the number of reports is smaller, and the zone
+ # won't be changed.
+ self.assertRaises(LoadFailure, self.__runner._do_load)
+ self.assertEqual([1, 2], self.__reports)
+ self.__check_zone_soa(ORIG_SOA_TXT)
+
+ def test_load_interrupted_create_cancel(self):
+ '''Load attempt for a new zone fails due to signal interruption
+
+ It cancels the zone creation.
+
+ '''
+ self.__common_load_setup()
+ self.__runner._report_progress = lambda x: self.__interrupt_progress(x)
+ self.__runner._zone_name = Name('example.com')
+ self.__runner._zone_file = ALT_NEW_ZONE_TXT_FILE
+ self.__check_zone_soa(None, zone_name=Name('example.com'))
+ self.assertRaises(LoadFailure, self.__runner._do_load)
+ self.assertEqual([1, 2], self.__reports)
+ self.__check_zone_soa(None, zone_name=Name('example.com'))
+
def test_run_success(self):
'''Check for the top-level method.
@@ -205,4 +325,10 @@ class TestLoadZoneRunner(unittest.TestCase):
if __name__== "__main__":
isc.log.resetUnitTestRootLogger()
+ # Disable the internal logging setup so the test output won't be too
+ # verbose by default.
+ LoadZoneRunner._config_log = lambda x: None
+
+ # Cancel signal handlers so we can stop tests when they hang
+ LoadZoneRunner._set_signal_handlers = lambda x: None
unittest.main()
diff --git a/src/bin/loadzone/tests/testdata/example-nons.org.zone b/src/bin/loadzone/tests/testdata/example-nons.org.zone
new file mode 100644
index 0000000..f37fa28
--- /dev/null
+++ b/src/bin/loadzone/tests/testdata/example-nons.org.zone
@@ -0,0 +1,10 @@
+;; Intentionally missing SOA for testing post-load checks
+example.org. 3600 IN SOA (
+ ns.example.org.
+ admin.example.org.
+ 1235
+ 3600 ;1H
+ 1800 ;30M
+ 2419200
+ 7200)
+ns.example.org. 3600 IN A 192.0.2.1
diff --git a/src/bin/loadzone/tests/testdata/example-nosoa.org.zone b/src/bin/loadzone/tests/testdata/example-nosoa.org.zone
new file mode 100644
index 0000000..3bf0d0e
--- /dev/null
+++ b/src/bin/loadzone/tests/testdata/example-nosoa.org.zone
@@ -0,0 +1,3 @@
+;; Intentionally missing SOA for testing post-load checks
+example.org. 3600 IN NS ns.example.org.
+ns.example.org. 3600 IN A 192.0.2.1
More information about the bind10-changes
mailing list