From bconry at isc.org Mon Nov 11 17:20:40 2019 From: bconry at isc.org (Brian Conry) Date: Mon, 11 Nov 2019 11:20:40 -0600 Subject: Operational Notification: synth-from-dnssec may cause slow resolution on resolvers under certain cache conditions Message-ID: To our customers and partners -- Today ISC is issuing an operational notification concerning a new feature that was introduced in BIND 9.12 that we have discovered may have a significant performance impact on some servers, depending on the size and composition of their caches. This feature (synth-from-dnssec) is enabled by default in BIND 9.12, 9.13 (both now EOL) and in 9.14 and 9.15. Later this month we will be issuing new maintenance releases of BIND 9.14 and 9.15 that include a change to reverse the default so that this synth-from-dnssec is inactive unless explicitly enabled. We plan to improve this feature in future maintenance releases, but until then, our recommendation is to disable it unless you are certain that it is providing a clear benefit in your production environment. Servers with large caches and which respond with a high proportion of NXDOMAINs are most likely to be affected. If you are experiencing intermittent periods of poor resolver responsiveness and/or unexplained spikes in CPU consumption that do not correspond with a similar increase in client traffic, it is possible that synth-from-dnssec processing is the underlying root cause. Cathy Almond ISC Support ----- Posting date: 11 November 2019 Program Impacted: BIND Versions affected: 9.12.0 -> 9.12.4-P2, 9.14.0 -> 9.14.7. Also versions 9.13.0 -> 9.13.7 of the 9.13 development branch and versions 9.15.0 -> 9.15.5 of the 9.15 development branch. Description: "synth-from-dnssec" is a feature, introduced in BIND 9.12.0, intended to mitigate the impact of "random subdomain" attacks on recursive resolvers by allowing them to avoid recursive lookups when NSEC information already present in the resolver's cache would be sufficient to conclude the nonexistence of a requested record. Unfortunately, it has been discovered that on resolvers that have large caches with a high proportion of negative answers, synth-from- dnssec can be noticeably slower than simply performing recursion to obtain an answer from an authoritative server. Because of the computational expense of traversing a large number of cache data structures in order to reach a conclusion, using synth-from-dnssec on a large cache can cause high CPU consumption and delays while the system tries to determine whether a synthesized answer can be provided. While the intent of the feature is to modestly improve performance, under some cache conditions performance can instead be significantly reduced; whether or not the feature improves performance as intended depends considerably on the size and contents of the cache. Impact: Under certain cache circumstances synth-from-dnssec may significantly slow performance of recursive resolution, irrespective of whether or not the servers have enabled DNSSEC-validation. Workarounds: Until replacement versions of BIND are made available which contain an improvement to the synth-from-dnssec feature, operators can choose to disable it by including "synth-from-dnssec no;" in the global options section of named.conf. Synth-from-dnssec is currently enabled by default in versions which support the feature so it must be turned off explicitly if operators wish to avoid its use. Beginning with November 2019 maintenance releases the default behavior will be reversed (so that synth-from-dnssect defaults to off) until further notice. Solution: ISC plans to improve the behavior of the synth-from-dnssec feature in future maintenance releases. Until then, we will be changing the default behavior, beginning with the November 2019 maintenance releases. However, until the feature has been corrected to prevent potential significant performance impact we recommend disabling the feature (using the configuration syntax provided in the "Workarounds" section above) if you suspect that it is negatively affecting resolver performance. Do you still have questions? Questions regarding this advisory should go to security-officer at isc.org. To report a new issue, please encrypt your message using security-officer at isc.org's PGP key which can be found here: https://www.isc.org/downloads/software-support-policy/openpgp-key/. If you are unable to use encrypted email, you may also report new issues at: https://www.isc.org/community/report-bug/. Note: ISC patches only currently supported versions. When possible we indicate EOL versions affected. (For current information on which versions are actively supported, please see https://www.isc.org/downloads/). ISC Security Vulnerability Disclosure Policy: Details of our current security advisory policy and practice can be found here: https://kb.isc.org/docs/aa-00861 This Knowledgebase article is the complete and official operational notification document: https://kb.isc.org/docs/operational-notification-synth-from-dnssec-may-cause-slow-resolution-on-resolvers-under-certain-cache-conditions Legal Disclaimer: Internet Systems Consortium (ISC) is providing this notice on an "AS IS" basis. No warranty or guarantee of any kind is expressed in this notice and none should be implied. ISC expressly excludes and disclaims any warranties regarding this notice or materials referred to in this notice, including, without limitation, any implied warranty of merchantability, fitness for a particular purpose, absence of hidden defects, or of non-infringement. Your use or reliance on this notice or materials referred to in this notice is at your own risk. ISC may change this notice at any time. A stand-alone copy or paraphrase of the text of this document that omits the document URL is an uncontrolled copy. Uncontrolled copies may lack important information, be out of date, or contain factual errors. From cathya at isc.org Wed Nov 20 19:49:10 2019 From: cathya at isc.org (Cathy Almond) Date: Wed, 20 Nov 2019 19:49:10 +0000 Subject: CVE-2019-6477: TCP-pipelined queries can bypass tcp-clients limit Message-ID: <5c12718d-79ce-8651-4bc3-0377b810962b@isc.org> CVE: CVE-2019-6477 Document version: 1.1 Posting date: 20 November 2019 Program impacted: BIND Versions affected: BIND 9.11.6-P1 -> 9.11.12, 9.12.4-P1 -> 9.12.4-P2, 9.14.1 -> 9.14.7, and versions 9.11.5-S6 -> 9.11.12-S1 of BIND 9 Supported Preview Edition. Versions 9.15.0 -> 9.15.5 of the BIND 9.15 development branch are also affected. Versions prior to BIND 9.11.0 have not been evaluated for vulnerability to CVE-2019-6477. Severity: Medium Exploitable: Remotely Description: By design, BIND is intended to limit the number of TCP clients that can be connected at any given time. The update to this functionality introduced by CVE-2018-5743 changed how BIND calculates the number of concurrent TCP clients from counting the outstanding TCP queries to counting the TCP client connections. On a server with TCP-pipelining capability, it is possible for one TCP client to send a large number of DNS requests over a single connection. Each outstanding query will be handled internally as an independent client request, thus bypassing the new TCP clients limit. Impact: With pipelining enabled each incoming query on a TCP connection requires a similar resource allocation to a query received via UDP or via TCP without pipelining enabled. A client using a TCP-pipelined connection to a server could consume more resources than the server has been provisioned to handle. When a TCP connection with a large number of pipelined queries is closed, the load on the server releasing these multiple resources can cause it to become unresponsive, even for queries that can be answered authoritatively or from cache. (This is most likely to be perceived as an intermittent server problem). CVSS Score: 6.5 CVSS Vector: CVSS:3.1AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H/E:U/RL:O/RC:C For more information on the Common Vulnerability Scoring System and to obtain your specific environmental score please visit: https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator?vector=AV:N/AC:L/PR:N/UI: N/S:U/C:N/I:N/A:H/E:U/RL:O/RC:C&version=3.1 Workarounds: The vulnerability can be avoided by disabling server TCP-pipelining: keep-response-order { any; }; and then restarting BIND. The server restart is necessary because neither a 'reload' nor a 'reconfig' operation will properly reset currently pipelining TCP clients. Active exploits: We are not aware of any active exploits but we have received reports of servers accidentally affected by high-query-volume clients using TCP-pipelining. Solution: Upgrade to the patched release most closely related to your current version of BIND: * BIND 9.11.13 * BIND 9.14.8 * BIND 9.15.6 BIND Supported Preview Edition is a special feature preview branch of BIND provided to eligible ISC support customers. * BIND 9.11.13-S1 Note that the fix for CVE-2019-6477 addresses only the server memory leak issue. TCP-pipelining may still malfunction by dropping some responses on a TCP connection where a client query pattern generates excessive outstanding queries, but the malfunction will affect that TCP connection alone and will not cause any degradation of service to other clients. An affected client connection might also appear to hang, but will clear when either the client or the server initiates a close or reset and will not remain in that state indefinitely. Disabling TCP-pipelining entirely is completely effective at mitigating the vulnerability with minimal impact to clients that use pipelined TCP connections and with no impact to clients that do not support TCP- pipelining. The majority of Internet client DNS queries are transported over UDP or TCP without use of TCP-pipelining. Document revision history: 1.0 Early Notification, 13 November 2019 1.1 Updated Solution, 19 November 2019 Related documents: See our BIND 9 Security Vulnerability Matrix ( https://kb.isc.org/docs/aa-00913 ) for a complete listing of security vulnerabilities and versions affected. Do you still have questions? Questions regarding this advisory should go to security-officer at isc.org. To report a new issue, please encrypt your message using security-officer at isc.org's PGP key which can be found here: https://www.isc.org/pgpkey/ If you are unable to use encrypted email, you may also report new issues at: https://www.isc.org/reportbug/ . Note: ISC patches only currently supported versions. When possible we indicate EOL versions affected. (For current information on which versions are actively supported, please see https://www.isc.org/download/ .) ISC Security Vulnerability Disclosure Policy: Details of our current security advisory policy and practice can be found in the ISC Software Defect and Security Vulnerability Disclosure Policy at https://kb.isc.org/docs/aa-00861 . Legal Disclaimer: Internet Systems Consortium (ISC) is providing this notice on an "AS IS" basis. No warranty or guarantee of any kind is expressed in this notice and none should be implied. ISC expressly excludes and disclaims any warranties regarding this notice or materials referred to in this notice, including, without limitation, any implied warranty of merchantability, fitness for a particular purpose, absence of hidden defects, or of non-infringement. Your use or reliance on this notice or materials referred to in this notice is at your own risk. ISC may change this notice at any time. A stand-alone copy or paraphrase of the text of this document that omits the document URL is an uncontrolled copy. Uncontrolled copies may lack important information, be out of date, or contain factual errors. From cathya at isc.org Wed Nov 20 19:49:37 2019 From: cathya at isc.org (Cathy Almond) Date: Wed, 20 Nov 2019 19:49:37 +0000 Subject: New security release versions of BIND are available: 9.11.13, 9.14.8 and 9.15.6 Message-ID: New security releases of BIND are available which contain fixes for the CVEs disclosed today. The new versions of BIND are available for download from the ISC website's downloads page -- https://www.isc.org/download/ Release notes can be found via these links. Stable release branches: 9.11.13: https://downloads.isc.org/isc/bind9/9.11.13/RELEASE-NOTES-bind-9.11.13.html 9.14.8: https://downloads.isc.org/isc/bind9/9.14.8/RELEASE-NOTES-bind-9.14.8.html Experimental development branch 9.15.6: https://downloads.isc.org/isc/bind9/9.15.6/RELEASE-NOTES-bind-9.15.6.html Cathy Almond ISC Support