Operational Notification: synth-from-dnssec may cause slow resolution on resolvers under certain cache conditions

[Brian Conry]

To our customers and partners --

Today ISC is issuing an operational notification concerning a new
feature that was introduced in BIND 9.12 that we have discovered may
have a significant performance impact on some servers, depending on
the size and composition of their caches.

This feature (synth-from-dnssec) is enabled by default in BIND 9.12,
9.13 (both now EOL) and in 9.14 and 9.15.

Later this month we will be issuing new maintenance releases of BIND
9.14 and 9.15 that include a change to reverse the default so that
this synth-from-dnssec is inactive unless explicitly enabled.

We plan to improve this feature in future maintenance releases, but
until then, our recommendation is to disable it unless you are certain
that it is providing a clear benefit in your production environment.

Servers with large caches and which respond with a high proportion of
NXDOMAINs are most likely to be affected.

If you are experiencing intermittent periods of poor resolver
responsiveness and/or unexplained spikes in CPU consumption that do
not correspond with a similar increase in client traffic, it is
possible that synth-from-dnssec processing is the underlying root
cause.

Cathy Almond ISC Support


-----

Posting date:        11 November 2019

Program Impacted:    BIND

Versions affected:   9.12.0 -> 9.12.4-P2, 9.14.0 -> 9.14.7.
                     Also versions 9.13.0 -> 9.13.7 of the 9.13
                     development branch and versions 9.15.0 -> 9.15.5
                     of the 9.15 development branch.

Description:

   "synth-from-dnssec" is a feature, introduced in BIND 9.12.0,
   intended to mitigate the impact of "random subdomain" attacks on
   recursive resolvers by allowing them to avoid recursive lookups
   when NSEC information already present in the resolver's cache would
   be sufficient to conclude the nonexistence of a requested record.

   Unfortunately, it has been discovered that on resolvers that have
   large caches with a high proportion of negative answers,
   synth-from- dnssec can be noticeably slower than simply performing
   recursion to obtain an answer from an authoritative server. Because
   of the computational expense of traversing a large number of cache
   data structures in order to reach a conclusion, using
   synth-from-dnssec on a large cache can cause high CPU consumption
   and delays while the system tries to determine whether a
   synthesized answer can be provided.  While the intent of the
   feature is to modestly improve performance, under some cache
   conditions performance can instead be significantly reduced;
   whether or not the feature improves performance as intended depends
   considerably on the size and contents of the cache.

Impact:

   Under certain cache circumstances synth-from-dnssec may
   significantly slow performance of recursive resolution,
   irrespective of whether or not the servers have enabled
   DNSSEC-validation.

Workarounds:

   Until replacement versions of BIND are made available which contain
   an improvement to the synth-from-dnssec feature, operators can
   choose to disable it by including "synth-from-dnssec no;" in the
   global options section of named.conf. Synth-from-dnssec is
   currently enabled by default in versions which support the feature
   so it must be turned off explicitly if operators wish to avoid its
   use. Beginning with November 2019 maintenance releases the default
   behavior will be reversed (so that synth-from-dnssect defaults to
   off) until further notice.

Solution:

   ISC plans to improve the behavior of the synth-from-dnssec feature
   in future maintenance releases. Until then, we will be changing the
   default behavior, beginning with the November 2019 maintenance
   releases.

   However, until the feature has been corrected to prevent potential
   significant performance impact we recommend disabling the feature
   (using the configuration syntax provided in the "Workarounds"
   section above) if you suspect that it is negatively affecting
   resolver performance.

Do you still have questions? Questions regarding this advisory should
go to security-officer at isc.org. To report a new issue, please encrypt
your message using security-officer at isc.org's PGP key which can be
found here:
   https://www.isc.org/downloads/software-support-policy/openpgp-key/.
If you are unable to use encrypted email, you may also report new
issues at: https://www.isc.org/community/report-bug/.

Note:

   ISC patches only currently supported versions. When possible we
   indicate EOL versions affected. (For current information on which
   versions are actively supported, please see
   https://www.isc.org/downloads/).

ISC Security Vulnerability Disclosure Policy:

   Details of our current security advisory policy and practice can be
   found here: https://kb.isc.org/docs/aa-00861

This Knowledgebase article is the complete and official operational
notification document:


https://kb.isc.org/docs/operational-notification-synth-from-dnssec-may-cause-slow-resolution-on-resolvers-under-certain-cache-conditions

Legal Disclaimer:

   Internet Systems Consortium (ISC) is providing this notice on an
   "AS IS" basis. No warranty or guarantee of any kind is expressed in
   this notice and none should be implied. ISC expressly excludes and
   disclaims any warranties regarding this notice or materials
   referred to in this notice, including, without limitation, any
   implied warranty of merchantability, fitness for a particular
   purpose, absence of hidden defects, or of non-infringement. Your
   use or reliance on this notice or materials referred to in this
   notice is at your own risk. ISC may change this notice at any time.
   A stand-alone copy or paraphrase of the text of this document that
   omits the document URL is an uncontrolled copy.  Uncontrolled
   copies may lack important information, be out of date, or contain
   factual errors.