<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
The only dns request my server are handling now is just some monitoring dns request.. It's just a few dns request / min, not much. </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Even the 'rndc' command cannot get any answer from the named process <span id="😕">
😕- It looks like named don't even handle the incoming traffic from rndc command, since my revc-q increase for every time i use the rndc command. (I'm running rndc local)</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span><br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span><span>[root@ns-2d ~]# ss -lnt<br>
</span>
<div>State      Recv-Q Send-Q                                                                          Local Address:Port                                                                                         Peer Address:Port<br>
</div>
<span>LISTEN     10     128                                                                                 127.0.0.1:953                                                                                                     *:*</span><br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
And once again:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span>[root@ns-2d ~]# ss -lnt<br>
</span>
<div>State      Recv-Q Send-Q                                                                          Local Address:Port                                                                                         Peer Address:Port<br>
</div>
<span>LISTEN     11     128                                                                                 127.0.0.1:953                                                                                                     *:*</span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span><br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Outgoing network traffic is working just fine. I've checked serval dns servers with dig..  So i don't think this is the problem..</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Do you guys have any other suggestions for my problem?</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
/Søren </div>
<div>
<div id="Signature">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px"></div>
</div>
</div>
</div>
</div>
</div>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> bind-users <bind-users-bounces@lists.isc.org> on behalf of Frey, Rick E <Rick.Frey@windstream.com><br>
<b>Sent:</b> Monday, April 27, 2020 15:11<br>
<b>To:</b> bind-users@lists.isc.org <bind-users@lists.isc.org><br>
<b>Subject:</b> Re: Bind suddenly starts responding clients with servfail</font>
<div> </div>
</div>
<style>
<!--
@font-face
        {font-family:"Cambria Math"}
@font-face
        {font-family:Calibri}
@font-face
        {font-family:"Segoe UI"}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:11.0pt;
        font-family:"Calibri",sans-serif}
a:link, span.x_MsoHyperlink
        {color:blue;
        text-decoration:underline}
span.x_EmailStyle19
        {font-family:"Calibri",sans-serif;
        color:windowtext}
.x_MsoChpDefault
        {font-size:10.0pt}
@page WordSection1
        {margin:1.0in 1.0in 1.0in 1.0in}
div.x_WordSection1
        {}
-->
</style>
<div lang="EN-US" link="blue" vlink="purple"><span style="background:#FFEB9C; color:#9C6500">[EXTERNAL MAIL]</span><br>
<br>
<div>
<div class="x_WordSection1">
<p class="x_MsoNormal">Recursive clients are lookups/clients on your nameserver on behalf of a query received.  If you are seeing that your nameserver is running out of recursive clients after removing â€œall” traffic, it would indicate something is still querying
 your nameserver as BIND won’t spontaneously create recursive lookups.  Perhaps something local on the server is generating queries?</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">A dump of existing recursive clients can be performed using â€œrndc recursing”.   Output is normally â€œnamed.recursing” in your data directory. 
</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">I would suspect that your server may be unable to make outbound connections to authoritative servers.  This could cause high number of recursive clients.  Note that behavior of BIND is to start dropping older outstanding recursive lookups
 once 90% of recursive clients is reached (900 recursive clients in your case).  Thus, a high number of recursive clients in itself normally doesn’t result in SERVFAIL for queries.
</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">Not sure why you’re unable to run rndc commands (local or remote?).   Perhaps you are out of file descriptors as well?</p>
<p class="x_MsoNormal"> </p>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="x_MsoNormal"><b><span style="font-size:12.0pt; color:black">From: </span>
</b><span style="font-size:12.0pt; color:black">bind-users <bind-users-bounces@lists.isc.org> on behalf of Søren Andersen <SOAN@stofa.dk><br>
<b>Date: </b>Monday, April 27, 2020 at 4:00 AM<br>
<b>To: </b>"bind-users@lists.isc.org" <bind-users@lists.isc.org><br>
<b>Subject: </b>Bind suddenly starts responding clients with servfail</span></p>
</div>
<div>
<p class="x_MsoNormal"> </p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130; background:white">Hello List,</span><span style="font-size:12.0pt; color:black"></span></p>
</div>
<div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130">I'm running a few BIND servers, but lately one of my servers suddenly starts responding to clients with servfail for every request from the
 clients, and BIND doesn't respond to the rndc or statistics interface anymore. </span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130">My logs for client-channel show me this: </span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130">25-Apr-2020 21:52:04.501 client @XX XX.37#2921 (<a href="https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgoogle.dk%2F&data=02%7C01%7Crick.frey%40windstream.com%7C088f1237535d4029ded408d7ea896aa1%7C2567b4c1b0ed40f5aee358d7c5f3e2b2%7C1%7C1%7C637235748246317789&sdata=nIEJu8WpBU%2FecqbCjax4pFS2QQDrgCntDc761goKcY4%3D&reserved=0" target="_blank">google.dk</a>):
 no more recursive clients (1000/900/1000): quota reached</span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130">I've removed all the dns traffic from the server, and the quota is still reached after 6+ hours?</span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130">Do you guys have some clue what all this is about? - Or any suggestions where to look for any further information?</span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130">I'm running BIND 9.16.1 on CentOS 7:</span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130">named -V<br>
BIND 9.16.1 (Stable Release) <id:d497c32><br>
running on Linux x86_64 3.10.0-1062.el7.x86_64 #1 SMP Wed Aug 7 18:08:02 UTC 2019<br>
built by make with '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '--prefix=/opt/isc/isc-bind/root/usr' '--exec-prefix=/opt/isc/isc-bind/root/usr' '--bindir=/opt/isc/isc-bind/root/usr/bin'
 '--sbindir=/opt/isc/isc-bind/root/usr/sbin' '--sysconfdir=/etc/opt/isc/isc-bind' '--datadir=/opt/isc/isc-bind/root/usr/share' '--includedir=/opt/isc/isc-bind/root/usr/include' '--libdir=/opt/isc/isc-bind/root/usr/lib64' '--libexecdir=/opt/isc/isc-bind/root/usr/libexec'
 '--localstatedir=/var/opt/isc/isc-bind' '--sharedstatedir=/var/opt/isc/isc-bind/lib' '--mandir=/opt/isc/isc-bind/root/usr/share/man' '--infodir=/opt/isc/isc-bind/root/usr/share/info' '--disable-static' '--enable-dnstap' '--with-pic' '--with-gssapi' '--with-json-c'
 '--with-libtool' '--with-libxml2' '--without-lmdb' '--with-docbook-xsl=/usr/share/sgml/docbook/xsl-stylesheets' '--with-python' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
 -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic' 'LDFLAGS= -L/opt/isc/isc-bind/root/usr/lib64' 'PKG_CONFIG_PATH=:/opt/isc/isc-bind/root/usr/lib64/pkgconfig:/opt/isc/isc-bind/root/usr/share/pkgconfig'<br>
compiled by GCC 4.8.5 20150623 (Red Hat 4.8.5-39)<br>
compiled with OpenSSL version: OpenSSL 1.0.2k-fips  26 Jan 2017<br>
linked to OpenSSL version: OpenSSL 1.0.2k-fips  26 Jan 2017<br>
compiled with libxml2 version: 2.9.1<br>
linked to libxml2 version: 20901<br>
compiled with json-c version: 0.11<br>
linked to json-c version: 0.11<br>
compiled with zlib version: 1.2.7<br>
linked to zlib version: 1.2.7<br>
compiled with protobuf-c version: 1.3.2<br>
linked to protobuf-c version: 1.3.2<br>
threads support is enabled</span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:11.5pt; font-family:"Segoe UI"; color:#323130">/Søren</span></p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>