BIND 10 trac2980, updated. d63e340ac318b68e1a04458ba9669f159d6eb508 [2980] Merge branch 'trac2980' of ssh://git.bind10.isc.org/var/bind10/git/bind10 into trac2980
BIND 10 source code commits
bind10-changes at lists.isc.org
Tue Jun 25 13:27:58 UTC 2013
The branch, trac2980 has been updated
via d63e340ac318b68e1a04458ba9669f159d6eb508 (commit)
via 1954874770a322999e04315397c25a56cd1f1b3a (commit)
from c77a043a8a6f6216499cc9697d5881c65bd9ad2d (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit d63e340ac318b68e1a04458ba9669f159d6eb508
Merge: 1954874 c77a043
Author: Stephen Morris <stephen at isc.org>
Date: Tue Jun 25 14:20:42 2013 +0100
[2980] Merge branch 'trac2980' of ssh://git.bind10.isc.org/var/bind10/git/bind10 into trac2980
commit 1954874770a322999e04315397c25a56cd1f1b3a
Author: Stephen Morris <stephen at isc.org>
Date: Tue Jun 25 14:19:49 2013 +0100
[2980] Add Component Developer documentation
-----------------------------------------------------------------------
Summary of changes:
doc/devel/mainpage.dox | 45 ++-
src/lib/hooks/callout_handle.cc | 18 +-
src/lib/hooks/hooks_component_developer.dox | 427 +++++++++++++++++++++++++++
3 files changed, 474 insertions(+), 16 deletions(-)
create mode 100644 src/lib/hooks/hooks_component_developer.dox
-----------------------------------------------------------------------
diff --git a/doc/devel/mainpage.dox b/doc/devel/mainpage.dox
index 92f86eb..5c8efa2 100644
--- a/doc/devel/mainpage.dox
+++ b/doc/devel/mainpage.dox
@@ -1,11 +1,33 @@
+// Copyright (C) 2012-2013 Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
/**
- *
* @mainpage BIND10 Developer's Guide
*
* Welcome to BIND10 Developer's Guide. This documentation is addressed
- * at existing and prospecting developers and programmers, who would like
- * to gain insight into internal workings of BIND 10. It could also be useful
- * for existing and prospective contributors.
+ * at existing and prospecting developers and programmers and provides
+ * information needed to both extend and maintain BIND 10.
+ *
+ * If you wish to write "hook" code - code that is loaded by BIND 10 at
+ * run-time and modifies its behavior you should read the section
+ * @ref hookDevelopersGuide.
+ *
+ * BIND 10 maintanenace information is divided into a number of sections
+ * depending on focus. DNS-specific issues are covered in the
+ * @ref dnsMaintenanceGuide while information on DHCP-specific topics can
+ * be found in the @ref dhcpMaintenanceGuide. General BIND 10 topics, not
+ * specific to any protocol, are discussed in @ref miscellaneousTopics.
*
* If you are a user or system administrator, rather than software engineer,
* you should read <a href="http://bind10.isc.org/docs/bind10-guide.html">BIND10
@@ -13,13 +35,15 @@
*
* Regardless of your field of expertise, you are encouraged to visit
* <a href="http://bind10.isc.org/">BIND10 webpage (http://bind10.isc.org)</a>
+ * @section hooksFramework Hooks Framework
+ * - @subpage hooksComponentDeveloperGuide
*
- * @section DNS
+ * @section dnsMaintenanceGuide DNS Maintenance Guide
* - Authoritative DNS (todo)
* - Recursive resolver (todo)
* - @subpage DataScrubbing
*
- * @section DHCP
+ * @section dhcpMaintenanceGuide DHCP Maintenance Guide
* - @subpage dhcp4
* - @subpage dhcpv4Session
* - @subpage dhcpv4ConfigParser
@@ -39,7 +63,7 @@
* - @subpage dhcpDatabaseBackends
* - @subpage perfdhcpInternals
*
- * @section misc Miscellaneous topics
+ * @section miscellaneousTopics Miscellaneous topics
* - @subpage LoggingApi
* - @subpage LoggingApiOverview
* - @subpage LoggingApiLoggerNames
@@ -47,7 +71,10 @@
* - @subpage SocketSessionUtility
* - <a href="./doxygen-error.log">Documentation warnings and errors</a>
*
- * @todo: Move this logo to the right (and possibly up). Not sure what
- * is the best way to do it in Doxygen, without using CSS hacks.
* @image html isc-logo.png
*/
+/*
+ * @todo: Move the logo to the right (and possibly up). Not sure what
+ * is the best way to do it in Doxygen, without using CSS hacks.
+ */
+
diff --git a/src/lib/hooks/callout_handle.cc b/src/lib/hooks/callout_handle.cc
index d101965..60d4a1d 100644
--- a/src/lib/hooks/callout_handle.cc
+++ b/src/lib/hooks/callout_handle.cc
@@ -52,13 +52,17 @@ CalloutHandle::~CalloutHandle() {
context_collection_.clear();
// Normal destruction of the remaining variables will include the
- // decrementing of the reference count on the library manager collection
- // (which holds the libraries that could have allocated memory in the
- // argument and context members). When that goes to zero, the libraries
- // will be unloaded: however, at that point nothing in the hooks framework
- // will be accessing memory in the libraries' address space. It is possible // that some other data structure in the server (the program using the hooks
- // library) still references the address space, but that is outside the
- // scope of this framework.
+ // destruction of lm_collection_, wn action that will decrement the
+ // reference count on the library manager collection (which holds the
+ // libraries that could have allocated memory in the argument and context
+ // members). When that goes to zero, the libraries will be unloaded:
+ // at that point nothing in the hooks framework will be pointing to memory
+ // in the libraries' address space.
+ //
+ // It is possible that some other data structure in the server (the program
+ // using the hooks library) still references the address space and attempts
+ // to access it causing a segmentation fault. That issue is outside the
+ // scope of this framework and is not addressed by it.
}
// Return the name of all argument items.
diff --git a/src/lib/hooks/hooks_component_developer.dox b/src/lib/hooks/hooks_component_developer.dox
new file mode 100644
index 0000000..244e86b
--- /dev/null
+++ b/src/lib/hooks/hooks_component_developer.dox
@@ -0,0 +1,427 @@
+// Copyright (C) 2013 Internet Systems Consortium, Inc. ("ISC")
+//
+// Permission to use, copy, modify, and/or distribute this software for any
+// purpose with or without fee is hereby granted, provided that the above
+// copyright notice and this permission notice appear in all copies.
+//
+// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
+// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
+// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+// PERFORMANCE OF THIS SOFTWARE.
+
+/**
+ at page hooksComponentDeveloperGuide Guide to Hooks for the BIND 10 Component Developer
+
+ at section hooksComponentIntroduction Introduction
+
+The hooks framework is a BIND 10 library that simplifies the way that
+users can write code to modify the behavior of BIND 10. Instead of
+altering the BIND 10 source code, they write functions that are compiled
+and linked into a shared library. The library is specified in the BIND 10
+configuration database and run time, BIND 10 dynamically loads the library
+into its address space. At various points in the processing, the server
+"calls out" to functions in the library, passing to them the data is it
+currently working on. They can examine and modify the data as required.
+
+The document @ref hooksDevelopersGuide describes how to write a library
+that interfaces into a BIND 10 component. This guide describes how to
+write or modify a BIND 10 component so that it can load a shared library
+and call out to functions in it.
+
+ at subsection hooksComponentTerminology Terminology
+
+In the remainder of this guide, the following terminology is used:
+
+- Hook/Hook Point - used interchageably, this is a point in the code at
+which a call to user-written functions is made. Each hook has a name and
+each hook can have any number (including 0) of user-written functions
+attached to it.
+
+- Callout - a user-written function called by the server at a hook
+point. This is so-named because the server "calls out" to the library
+to execute a user-written function.
+
+- Component - a BIND 10 process, e.g. the authoritative server or the
+DHCPv4 server.
+
+- User code/user library - non-BIND 10 code that is compiled into a
+shared library and loaded by BIND 10 into its address space. Multiple
+user libraries can be loaded at the same time, each containing callouts for
+the same hooks. The hooks framework calls these libraries one after the
+other. (See the document @hooksDevelopersGuide for more details.)
+
+ at subsection hooksComponentLanguages Languages
+
+The core of BIND 10 is written in C++ with some parts in Python. While it is
+the intention to provide the hooks framework for all languages, the initial
+versions are for C++. All examples in this guide are in that language.
+
+ at section hooksComponentBasicIdeas Basic Ideas
+
+From the point of view of the component author, the basic ideas of the hooks
+framework are quite simple:
+
+- The hook points need to be defined.
+
+- At each hook point, the component needs to:
+ - copy data into the object used to pass information to the callout.
+ - call the callout.
+ - copy data back from the object used to exchange information.
+ - take action based on information returned.
+
+Of course, to set up the system the libraries need to be loaded in the first
+place. The component also needs to:
+
+- Define the configuration item that specifies the user libraries for this
+component.
+
+- Handle configuration changes and load/unload the user libraries.
+
+The following sections will describe these tasks in more detail.
+
+ at section hooksComponentDefinition Defining the Hook Points
+
+Before any other action takes place, the hook points in the code need to be
+defined. Each hook point has a name that must be unique amongst all hook
+points for the server, and the first step is to register those names. The
+naming is done using the static method isc::hooks::HooksManager::registerHook():
+
+ at code
+
+#include <hooks/hooks_manager.h>
+ :
+ int example_index = HooksManager::registerHook("manager");
+ at endcode
+
+The name of the hooks is passed as the sole argument to the
+HooksManager::registerHook() method. The value returned is the index of that
+hook point and should be retained - it is needed to call the hook.
+
+All hooks used by the component must be registered before the component
+starts operations.
+
+ at subsection hooksComponentHookNames Hook Names
+
+Hook names are strings and in principle, any string can be used as the
+name of a hook, even one containing spaces and non-printable characters.
+However, the following guidelines should be observed:
+
+- The names <b>context_create</b> and <b>context_destroy</b> are reserved to
+the hooks system and are automatically registered: an attempt to register
+one of these will lead to a isc::hooks::DuplicateHook exception being thrown.
+
+- The hook name should be a valid function name in C. If a user gives a
+callout the same name as one of the hooks, the hooks framework will
+automatically load that callout and attach it to the hook: the user does not
+have to explicitly register it. <b>TBD: do we still want this given
+the possibility of confusion with functions in system libraries?</b>
+
+- The hook name should not conflict with the name of a function in any of
+the system libraries (e.g. naming a hook "sqrt" could lead to the
+square-root function in the system's maths library being attached to the hook
+as a callout).
+
+- Although hook names can be in any case (including mixed case), the BIND 10
+convention is that they are lower-case.
+
+ at subsection hooksComponentAutomaticRegistration Automatic Registration of Hooks
+
+In some components, it may be convenient to set up a separate
+initialization function that registers all hooks. For others, it may
+be more convenient for each module within the component to perform its
+own initialization. Since the HooksManager object is a singleton and
+is created when first requested, a useful trick is to automatically
+register the hooks when the module is loaded.
+
+This technique involves declaring an object outside of any execution
+unit in the module. When the module is loaded, the object's constructor
+is run. By placing the hook registration calls in the constructor,
+the hooks in the module are defined at load time, before any function in
+the module is run. The code for such an initialization sequence would
+be similar to:
+
+ at code
+#include <hooks/hooks_manager.h>
+
+namespace {
+
+// Declare structure to perform initialization and store the hook indexes.
+//
+struct MyHooks {
+ int pkt_rcvd; // Packet received
+ int pkt_sent; // Packet sent
+
+ // Constructor
+ MyHooks() {
+ pkt_rcvd = HooksManager::registerHook("pkt_rcvd");
+ pkt_sent = HooksManager::registerHook("pkt_sent");
+ }
+};
+
+// Instantiate a "Hooks" object. The constructor is run when the module is
+// loaded and so the hook indexes will be defined before any method in this
+// module is called.
+Hooks hooks;
+
+} // Anonymous namespace
+
+void Someclass::someFunction() {
+ :
+ // Check if any callouts are defined on the pkt_rcvd hook.
+ if (HooksManager::calloutPresent(hooks.pkt_rcvd)) {
+ :
+ }
+ :
+}
+ at endcode
+
+ at section hooksComponentCallingCallouts Calling Callouts on a Hook
+
+ at subsection hooksComponentArgument The Callout Handle
+
+Before describing how to call user code at a hook point, we must first consider
+how to pass data to it.
+
+Each user callout has the signature:
+ at code
+int callout_name(CalloutHandle& handle);
+ at endcode
+
+The isc::hooks::CalloutHandle object is the object used to pass data to
+and from the callout. This holds the data as a set of name/value pairs,
+each pair being considered an argument to the callout.
+
+Two methods are provided to get and set the arguments passed to
+the callout called (naturally enough) getArgument and SetArgument.
+Their usage is illustrated by the following code snippets.
+
+ at code
+ int count = 10;
+ boost::shared_ptr<Pkt4> pktptr = ... // Set to appropriate value
+
+ // Assume that "handle" has been created
+ handle.setArgument("data_count", count);
+ handle.setArgument("inpacket", pktptr);
+
+ // Call the hook code...
+ ...
+
+ // Retrieve the modified values
+ handle.getArgument("data_count", count);
+ handle.getArgument("inpacket", pktptr);
+ at endcode
+
+As can be seen "getArgument" is used to retrieve data from the
+isc::hooks::CalloutHandle, and setArgument used to put data into it.
+If a callout wishes to alter data and pass it back to the server,
+it should retrieve the data with getArgument, modify it, and call
+setArgument to send it back.
+
+There are a couple points to be aware of:
+
+- The data type of the variable in the call to getArgument must
+match the data type of the variable passed to the corresponding
+setArgument <B>exactly</B>: using what would normally be considered
+to be a "compatible" type is not enough. For example, if the callout
+passed an argument back to the component as an "int" and the component
+attempted to retrieve it as a "long", an exception would be thrown even
+though any value that can be stored in an "int" will fit into a "long".
+This restriction also applies the "const" attribute but only as applied to
+data pointed to by pointers, e.g. if an argument is defined as a "char*",
+an exception will be thrown if an attempt is made to retrieve it into
+a variable of type "const char*". (However, if an argument is set as a
+"const int", it can be retrieved into an "int".) The documentation of
+each hook point should detail the exact data type of each argument.
+
+- If a pointer to an object is passed to a callout (either a "raw"
+pointer, or a boost smart pointer (as in the example above), and the
+underlying object is altered through that pointer, the change will be
+reflected in the component even if the callout makes no call to setArgument.
+This can be avoided by passing a pointer to a "const" object.
+
+ at subsection hooksComponentGettingHandle Getting the Callout Handle
+
+The CalloutHandle object is linked to the loaded libraries
+for lifetime reasons (described below). Components
+should retrieve a isc::hooks::CalloutHandle using
+isc::hooks::HooksManager::createCalloutHandle():
+ at code
+ CalloutHandlePtr handle = HooksManager::createCalloutHandle();
+ at endcode
+(isc::hooks::CalloutHandlePtr is a typedef for a boost shared pointer to a
+CalloutHandle.) The CalloutHandle so retrieved may be used for as
+long as the libraries are loaded.
+ at code
+ handle.reset();
+ at endcode
+... or by letting the handle object go out of scope. The actual deletion
+occurs when the CallHandle's reference count goes to zero. (The
+current version of the hooks framework does not maintain any other
+pointer to the returned CalloutHandle, so it gets destroyed when the
+shared pointer to it is cleared or destroyed. However, this may change
+in a future version.)
+
+ at subsection hooksComponentCallingCallout Calling the Callout
+
+Calling the callout is a simple matter of executing the
+isc::hooks::HooksManager::callCallouts() method for the hook index in
+question. For example, with the hook index pkt_sent defined as above,
+the hook can be executed by:
+ at code
+ HooksManager::callCallouts(pkt_rcvd, *handle);
+ at endcode
+... where "*handle" is a reference (note: not a pointer) to the
+isc::hooks::CalloutHandle object holding the arguments. No status code
+is returned. If a component needs to get data returned, it should define
+an argument through which the callout can do so.
+
+Actually, the statement "no status code is returned" is not strictly true. At
+many hooks, the following logic applies:
+ at code
+call hook_code
+if (hook_code has not performed an action) {
+ perform the action
+}
+ at endcode
+For example, in a DHCP server an address should be allocated for a client.
+The DHCP server does that by default, but the hook code may want to override
+it in some situations.
+
+As this logic is so common, the CalloutHandle includes a "skip" flag. This
+is a boolean flag that can be set by the callout to pass a basic yes/no
+response to the component. Its use is illustrated by the following code
+snippet:
+ at code
+// Set up arguments for lease assignment
+handle->setArgument("query", query);
+handle->setArgument("response", response);
+HooksManager::callCallouts(lease_hook_index, *handle);
+if (! handle->getSkip()) {
+ // Skip flag not set, do the address allocation
+ :
+}
+ at endcode
+<b>SHOULD WE GET RID OF THE SKIP FLAG AND WHERE APPROPRIATE, SIGNAL SUCH
+PROCESSING THROUGH AN ARGUMENT?</b>
+
+ at subsubsection hooksComponentConditionalCallout Conditionally Calling Hook Callouts
+
+Most hooks in a server will not have callouts attached to them. To avoid the
+overhead of setting up arguments in the CalloutHandle, a component can
+check for callouts before doing that processing. The
+isc::hooks::HooksManager::calloutsPresent() method performs this check.
+Taking the index of a hook as its sole argument, it returns true if there
+are any callouts attached to the hook and false otherwise.
+
+With this check, the above example can be modified to:
+ at code
+if (HooksManager::calloutsPresent(lease_hook_index)) {
+ // Set up arguments for lease assignment
+ handle->setArgument("query", query);
+ handle->setArgument("response", response);
+ HooksManager::callCallouts(lease_hook_index, *handle);
+ if (! handle->getSkip()) {
+ // Skip flag not set, do the address allocation
+ :
+ }
+}
+ at endcode
+
+ at section hooksComponentLoadLibraries Loading the User Libraries
+
+Once hooks are defined, all the hooks code describe above will
+work, even if no libraries are loaded (and even if the library
+loading method is not called). The CalloutHandle returned by
+isc::hooks::HooksManager::createCalloutHandle() will be valid,
+isc::hooks::HooksManager::calloutsPresent() will return false for every
+index, and isc::hooks::HooksManager::callCallouts() will be a no-op.
+
+However, if user libraries are specified in the BIND 10 configuration,
+the component should load them. (Note the term "libraries": the hooks
+framework allows multiple user libraries to be loaded.) This should take
+place after the component's configuration has been read, and is achieved
+by the isc::hooks::HooksManager::loadLibraries() method. The method is
+passed a vector of strings, each giving the full file specification of
+a user library:
+ at code
+ std::vector<std::string> libraries = ... // Get array of libraries
+ bool success = HooksManager::loadLibraries(libraries);
+ at endcode
+loadLibraries() returns a boolean status which is true if all libraries
+loaded successfully or false if one or more failed to load. Appropriate
+error messages will have been logged in the latter case, the status
+being more to allow the developer to decide whether the execution
+should proceed in such circumstances.
+
+If loadLibraries() is called a second or subsequent time (as a result
+of a reconfiguration), all existing libraries are unloaded and a new
+set loaded. Libraries can be explicitly unloaded either by calling
+isc::hooks::HooksManager::unloadLibraries() or by calling
+loadLibraries() with an empty vector as an argument.
+
+ at subsection hooksComponentUnloadIssues Unload and Reload Issues
+
+Unloading a shared library works by unmapping the part of the process's
+virtual address space in which the library lies. This may lead to problems
+consequences if there are still references to that address space elsewhere
+in the process.
+
+In many operating systems, heap storage allowed by a shared library will
+lie in the virtual address allocated to the library. This has implications
+in the hooks framework because:
+
+- Argument information stored in a CalloutHandle by a callout in a library
+may lie in the library's address space.
+- Data modified in objects passed as arguments may lie in the address
+space. For example, it is common for a DHCP callout to add "options" to
+a packet: the memory allocated for those options will like in library address
+space.
+
+The problem really arises because of the extensive use by BIND 10 of boost
+smart pointers. When the pointer is destroyed, the pointed-to memory is
+deallocated. If the pointer points to address space that is unmapped because
+a library has been unloaded, the deletion causes a segmentation fault.
+
+The hooks framework addresses the issue for CalloutHandles by keeping
+in that object a shared pointer to the object controlling library
+unloading. Although a library can be unloaded at any time, it is only when
+all CalloutHandles that could possibly reference address space in the
+library have been deleted that the library will be unloaded and the address
+space unmapped.
+
+The hooks framework cannot solve the second issue as the objects in
+question are under control of the component. It is up to the component
+developer to ensure that all such objects have been destroyed before
+libraries are reloaded. In extreme cases this may mean the component
+suspending all processing of incoming requests until all currently
+executing requests have completed and data object destroyed, reloading
+the libraries, then resuming processing.
+
+ at section hooksComponentCallouts Component-Defined Callouts
+
+Previous sections have discussed callout registration by user libraries.
+It is possible for a component to register its own functions (i.e. within
+its own address space) as hook callouts. These functions are called
+in eactly the same way as user callouts, being passed their arguments
+though a CalloutHandle object. (Guidelines for writing callouts can be
+found in @ref hooksDevelopersGuide.)
+
+A component can associate with a hook callouts that run either before
+user-registered callouts or after them. Registration is done via a
+isc::hooks::LibraryHandle object, a reference to one being obtained
+through the methods isc::hooks::HooksManager::preCalloutLibraryHandle()
+(for a handle to register callouts to run before the user library
+callouts) or isc::hooks::HooksManager::postCalloutLibraryHandle() (for
+a handle to register callouts to run after the user callouts). Use of
+the LibraryHandle to register and deregister callouts is described in
+ at ref hooksLibraryHandle.
+
+Finally, it should be noted that callouts registered in this way only
+remain registered until the next call to isc::hooks::loadLibraries().
+It is up to the server to re-register the callouts after this
+method has been called.
+
+*/
More information about the bind10-changes
mailing list