[Kea-announce] Kea 1.7.5, a new development release of Kea, is now available

[Michael McNally]

Internet Systems Consortium is pleased to announce the release of Kea 1.7.5.

Kea releases are divided into two branch sequences:

1)  A current stable branch (currently 1.6.x) which receives only important
     bug fixes and is otherwise feature-stable, and

2)  An experimental development branch (currently 1.7.x) where new features
     are added and made available for testing during development, before eventually
     being included in the next stable release branch.

Kea 1.7.5 is the latest release in the experimental development branch.


Michael McNally
ISC Support

-----------------------

# Kea 1.7.5, February 26th 2020, Release Notes

Welcome to Kea 1.7.5, a monthly development release of Kea. This release
is the next step towards having fully multi-threaded DHCP servers, a
feature that will be available in the next major release, 1.8.0. Kea
1.7.5 is a development release: use with caution - development releases
are not recommended for production use.

Changes introduced in this version:

1. **Multi-threading work**. While multi-threading is not yet available,
significant progress has been made with enabling Kea to use more than
one core. This month we managed to multi-thread the allocation engine
(#890) and statistics manager (#891), added a mechanism for hooks to
report whether they are multi-threading compatible, integrated the
thread pool with the server's packet processing (#892), and implemented
a connection pool for the PostgreSQL host backend (#1074). We should be
starting our internal tests in the coming month. Functionality-wise, the
only missing piece of code is a configuration option to enable
mutli-threading. We expect to add this once our internal tests starts
producing sufficiently stable results.

2. **Pkt4_send hook point can now drop packets**. Hooks can set a
next-step field to indicate what Kea should do with a packet once it
regains control from the Hook code. The Pkt4_send hook point used to
treat DROP and SKIP the same way - as an instruction to skip packing the
packet. The DROP status now instructs Kea to drop the packet. #1090

3. **Options with embedded NULL characters**. Previously, Kea did not
handle correctly an uncommon situation where a DHCP option contained an
embedded NULL (0) character. The bug has been fixed (#1048).

4. **Perfdhcp uniqueness checks**. Perfdhcp, our performance testing
tool, just got an upgrade. It now checks if the addresses assigned are
all unique. Up to now, Kea has been assigning addresses one by one.
Although very unlikely, it is theoretically possible that with
multi-threading enabled, a bug could exist in the code that would cause
Kea to assign the same address twice. We are now able to check for that
possibility in our performance tests. #1021

5. **Compilation fix for Ubuntu 19.10 and MySQL/MariaDB**. An elusive
issue was reported some time ago regarding the compilation of Kea on
Ubuntu 19.10. The problem affected only specific versions of MySQL and
MariaDB and was not reproducible on all Ubuntu 19.10 systems. It is now
fixed. It is possible that the fix helps with compilation on systems
other than Ubuntu. #1071

6. **Better recovery after database connection failure**. Kea has a
mechanism to reconnect to a database if the connection was lost. A
problem existed that after several unsuccessful attempts, Kea would give
up and terminate abruptly without cleaning up properly (e.g. not
removing UNIX sockets, etc). This caused the next Kea restart attempt to
fail under certain circumstances. In version 1.7.5 the
Kea server, upon encountering this problem, now calls its shutdown
routines to clean up orphaned resources before exiting.  However, this
has the inadvertent side effect of changing the exit code for the server
process, which now exits with a 0 return value (indicating no error)
upon completion of the server's shutdown function. Beginning in
version 1.7.6, it is planned to extend the shutdown code, permitting
the server to pass along a non-zero exit code when a failure is detected
while still performing cleanup. Operators are therefore cautioned not to
rely on the temporary return value behavior unique to this release version.
#1097, #1108

7. **Running Kea from unprivileged account**. In principle, Kea requires
root access to open raw sockets and UDP sockets with ports lower than
1024. However, it is possible to run Kea under a regular account on
Linux using the capabilities mechanism. There is a new section in the
Kea ARM that explains how to do this. #160

## Changes to Release Model
The Kea project has been in development for several years now, and it
has a significant production deployment base with users who are looking
for stability, rather than a constant stream of new "bleeding-edge"
features. At the same time, we want to continue developing the software
and add some new powerful, but difficult-to-implement, features. As a
result, we decided to change the release cycle. Starting from 1.6.0,
there are two series of releases: stable and development.

Stable releases are what you would expect: stable, released
infrequently, without new features or significant changes, very
well-tested. These can be identified by the middle version number being
even. The current stable release is 1.6.2. If we discover important bugs
that require fixing, we may release 1.6.3, but that will be determined
on a case-by-case basis. The next major stable version will be 1.8.0,
followed by 2.0.0 in the future.

Our team continues development of new features. In particular, we're
tackling the difficult problem of being able to use all available CPU
cores simultaneously. The multi-threading implementation is a complex
task and it is unknown how long it will take before the solution is
stable and ready for a production environment. At the same time, we
continue to receive a stream of requests for small features and bug
fixes. We don't want to force users to wait half a year or more for the
fixes and features that are already done. Therefore, we have decided to
start issuing development releases on a monthly basis. Those are
slightly less well-tested and may have features that are not complete.
For example, it is possible that one of the next releases will provide a
configuration knob to specify the number of threads in multi-threading,
but the actual code that spawns those threads will not yet
have been added.

The development releases can be easily identified by the middle version
number being odd: for example, 1.7.5 is a development release. In March
2020 we will release 1.7.6, the next development version. Once 1.8.0 is
out, we will continue our development work with 1.9.0, then 1.9.1, and
so on.

Our goal is to make the development release available on the last
Wednesday of each month. There may be exceptions (such as during
holidays), but that's the general plan.

We encourage users to test the development releases and report back
their findings.

For more details on the plan, see ISC's Software Support Policy at
https://kb.isc.org/v1/docs/aa-00896.

## Kea overview
Kea is a DHCP implementation developed by Internet Systems Consortium,
Inc. that features fully functional DHCPv4 and DHCPv6 servers, a dynamic
DNS update daemon, a Control Agent (CA) that provides a REST API to
control the DHCP and DNS update servers, an example shell client to
connect to the CA, a daemon that is able to retrieve YANG configuration
and updates from Sysrepo, and a DHCP performance-measurement tool. Both
DHCP servers fully support server discovery, address assignment,
renewal, rebinding, release, decline, information request, DNS updates,
client classification, and host reservations. The DHCPv6 server also
supports prefix delegation. Lease information is stored in a CSV file by
default; it can optionally be stored in a MySQL, PostgreSQL, or
Cassandra database instead. Host reservations can be stored in a
configuration file, or in a MySQL, PostgreSQL, or Cassandra database.
They can also be retrieved from a RADIUS server, although this
functionality is somewhat limited. Kea DHCPv4 and DHCPv6 daemons provide
support for YANG models, which are stored in a Sysrepo datastore and can
be configured via the NETCONF protocol.

This text references issue numbers. For more details, visit the Kea
GitLab page at https://gitlab.isc.org/isc-projects/kea/issues.

## License
This version of Kea is released under the Mozilla Public License,
version 2.0.

    https://www.mozilla.org/en-US/MPL/2.0

The premium and subscriber-only hook libraries are provided in source
code form, under the terms of an End User License Agreement (you will
get the source code that you can modify freely, but you are not
permitted to redistribute it).

## Download
Pre-built ISC packages for current versions of the most popular Linux
operating systems are available at:

    https://cloudsmith.io/~isc/repos/

The Kea source and PGP signature for this release may be downloaded from:

    https://www.isc.org/download

The signature was generated with the ISC code signing key which is
available at:

    https://www.isc.org/pgpkey

ISC provides detailed documentation, including installation instructions
and usage tutorials, in the Kea Administrator Reference Manual.
Documentation is included with the installation or via
https://kb.isc.org/docs/kea-administrator-reference-manual in HTML,
plain text, or PDF formats. ISC maintains a public open source code
tree, a wiki, an issue tracking system, milestone planning, and a
roadmap at https://gitlab.isc.org//isc-projects/kea.

Limitations and known issues with this release can be found at
https://gitlab.isc.org/isc-projects/kea/wikis/known-issues-list.

We ask users of this software to please let us know how it worked for
you and what operating system you tested on. Feel free to share your
feedback on the Kea Users mailing list
(https://lists.isc.org/mailman/listinfo/kea-users).  Also we would like
to hear whether the documentation is adequate and accurate. Please open
tickets in the Kea GitLab project for bugs, documentation omissions and
errors, and enhancement requests. We want to hear from you even if
everything worked.

## Support
Professional support for Kea is available from ISC. We encourage all
professional users to consider this option; Kea development and
maintenance are funded with support subscriptions. For more information
on ISC's Kea and DHCP software support see https://www.isc.org/support/.

Free best-effort support is provided by our user community via a mailing
list. Information on all public email lists is available at
https://www.isc.org/community/mailing-list. If you have any comments or
questions about working with Kea, please share them to the Kea Users
List (https://lists.isc.org/mailman/listinfo/kea-users). Bugs and
feature requests may be submitted via GitLab at
https://gitlab.isc.org/isc-projects/kea/issues.

## Changes
The following summarizes changes and important upgrade notes since the
previous release (1.7.4).

1725.	[func]		wlodek
	Added perfdhcp command option to enable checking address
	and/or prefix uniqueness.
	(Gitlab #1021)

1724.	[func]		razvan
	Added packet thread pool in dhcpv4 and dhcpv6 servers. It adds
	the ability to handle processPacket function on multiple threads.
	When enabled, parked packets are also added to the queue of items
	to be handles by the thread pool. The packet thread pool is
	currently disabled and there is no option to enable it yet.
	(Gitlab #892)

1723.	[func]		tmark
	Hostname sanitizing is now globally enabled by default.  Prior
	to this it was disabled.
	(Gitlab #1048)

1722.	[bug]		razvan
	Compilation fix for MySQL/MariaDB on Ubuntu 19.10.
	(Gitlab #1071)

1721.	[func]*		fdupont
	Added multi_threading_compatible function in hooks to retrieve
	the compatibility with multi-threading. This function is called
	when the library is loaded and if it reports 0 (incompatible) and
	multi-threading is enabled, then the configuration will be
	rejected. The absence of this function from the library defaults
	to incompatibility with multi-threading. All hooks and premium
	hooks must be updated to be able to be used with multi-threading.
	However, not all libraries are (or will ever be) compatible.
	(Gitlab #947)

1720.	[func]*		tomek
	The next step status in pkt4_send hook point now is able to
	distinguish between SKIP (Kea will skip packing the packet with
	the assumption that a hook already did that) and DROP (Kea will
	drop the packet). Previously both SKIP and DROP were interpreted
	as skip packing.
	(Gitlab #1090)

1719.	[func]		razvan
	Prepared CalloutManager to be used in multi-threading by moving
	the current library index and hook index to the CallbackHandle.
	This change removes the CallountHandle::getLibraryHandle, along
	with the possibility of dynamically registering/deregistering
	callouts inside callouts.
	(Gitlab #957)

1718.	[bug]		tmark
	kea-dhcp4 and kea-dhcp6 now shutdown gracefully by executing
	the shutdown command, if connectivity with a backend database
	has been lost and retries are either disabled or have been
	exhausted. Prior to this they simply invoked exit() which
	could orphan control socket files or cause segfaults unloading
	the CB Cmds hook library.
	(Gitlab #1108)

1717.	[func]		razvan
	Prepared PgSqlHostMgr to be used with multi-threading by using a
	connection pool with thread context.
	(Gitlab #1074)

1716.	[func]		razvan
	Implemented thread safe stats manager and stats context.
	(Gitlab #891)

1715.	[bug]		tmark
	Kea servers now detect and remove orphaned control channel
	sockets.  This corrects a failure of the servers to restart
	with an error of "address already in use" following a fatal
	loss of database connecivity.
	(Gitlab #1097)

Thank you again to everyone who assisted us in making this release
possible.

We look forward to receiving your feedback.