<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:URWPalladioL;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:NimbusMonL;
panose-1:2 11 6 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.gmail-m-6196280132677980643p1, li.gmail-m-6196280132677980643p1, div.gmail-m-6196280132677980643p1
{mso-style-name:gmail-m_-6196280132677980643p1;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.gmail-m-6196280132677980643s1
{mso-style-name:gmail-m_-6196280132677980643s1;}
span.gmail-m-6196280132677980643apple-converted-space
{mso-style-name:gmail-m_-6196280132677980643apple-converted-space;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Greg, What other awesome stuff do you have on the top of your head? This makes sense as it’s running on EC2 @AWS (I.e. poor source of randomness on VM’s).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">And thanks to Grant for the haveged suggestion. Initial tests with haveged running seem to be positive.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’ll report back here if the problem continues.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks so much for your help!<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Greg Choules <gregchoules+bindusers@googlemail.com><br>
<b>Date: </b>Monday, August 1, 2022 at 6:21 PM<br>
<b>To: </b>White, Peter <pwhite@penguinrandomhouse.com><br>
<b>Cc: </b>bind-users@lists.isc.org <bind-users@lists.isc.org><br>
<b>Subject: </b>Re: Bind 9.11/RHEL7 Server Freezes FUTEX_WAKE_PRIVATE<o:p></o:p></span></p>
</div>
<div style="border:solid white 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="MsoNormal" style="line-height:12.0pt;background:#FF6600"><span style="font-size:10.0pt;color:white">CAUTION: This email originated from outside of Penguin Random House. Please be extra cautious when opening file attachments or clicking on links.<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Hi Peter.<o:p></o:p></p>
<div>
<p class="MsoNormal">Off the top of my head, could it be this?<o:p></o:p></p>
</div>
<div>
<div>
<div>
<div>
<p><b><span style="font-size:10.0pt;font-family:"URWPalladioL",serif">random-device
</span></b><o:p></o:p></p>
<p><span style="font-size:10.0pt;font-family:"URWPalladioL",serif">The source of entropy to be used by the server. Entropy is primarily needed for DNSSEC operations, such as TKEY transactions and dynamic update of signed zones. This options specifies the device
(or file) from which to read entropy. If this is a file, operations re- quiring entropy will fail when the file has been exhausted. If not specified, the default value is
</span><span style="font-size:10.0pt;font-family:"NimbusMonL",serif">/dev/random </span>
<span style="font-size:10.0pt;font-family:"URWPalladioL",serif">(or equivalent) when present, and none otherwise. The
<b>random- device </b>option takes effect during the initial configuration load at server startup time and is ignored on subsequent reloads. </span><o:p></o:p></p>
<p>BIND will need a good source of randomness for crypto operations.<o:p></o:p></p>
<p>Cheers, Greg<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Mon, 1 Aug 2022 at 23:08, White, Peter <<a href="mailto:pwhite@penguinrandomhouse.com">pwhite@penguinrandomhouse.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="gmail-m-6196280132677980643p1"><span style="font-size:12.0pt">I’m running
<span class="gmail-m-6196280132677980643s1">BIND 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.9 (Extended Support Version) on RHEL 7 in a chroot jail.</span></span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">As of late, at times running some rndc commands are causing my server to lock up. It’s usually an “rndc addzone” that triggers the issue. I’ll also mention that I have recently
started signing some domains with DNSSEC, so I suspect it may be somehow related.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Here is an example of a command that frequently triggers my issue, although it doesn’t trigger it every time.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">rndc addzone '"<a href="http://example.com" target="_blank">example.com</a>" in external {type master; file "dnssec/<a href="http://example.com" target="_blank">example.com</a>";key-directory
"keys"; auto-dnssec maintain; inline-signing yes;};'<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">During these times, named will not respond to any rndc commands, nothing is logged to the bind logs (I’m running trace level 3 ), and will not answer queries. Everything seems just
frozen in time. Waiting for a period of time, varying from a few seconds to many minutes, the server picks back up again and operates normally. The following are my observations to this point.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">CPU and memory show as being fine.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="gmail-m-6196280132677980643p1"><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">top - 17:57:37 up 33 min,</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><span style="font-size:12.0pt">
</span></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">3 users,</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><span style="font-size:12.0pt">
</span></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">load average: 0.00, 0.01, 0.05</span></span><o:p></o:p></p>
<p class="gmail-m-6196280132677980643p1"><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">Tasks:<b> 125
</b>total,<b> </b></span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">2
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">running,<b> 123
</b>sleeping,<b> </b></span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">0
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">stopped,<b>
</b></span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">0
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">zombie</span></span><o:p></o:p></p>
<p class="gmail-m-6196280132677980643p1"><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">%Cpu(s):</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">0.2
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">us,</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">0.3
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">sy,</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">0.0
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">ni,<b> 98.5
</b>id,</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">0.0
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">wa,</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">0.0
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">hi,</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">0.0
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">si,</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">1.0
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">s</span></span><o:p></o:p></p>
<p class="gmail-m-6196280132677980643p1"><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">KiB Mem :</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">1842956
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">total,<b>
</b></span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">439452
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">free,<b>
</b></span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">665760
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">used,<b>
</b></span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">737744
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">buff/cache</span></span><o:p></o:p></p>
<p class="gmail-m-6196280132677980643p1"><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">KiB Swap:</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">8384508
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">total,</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">8384508
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">free,</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">0
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">used.</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><b><span style="font-size:12.0pt">
</span></b></span><span class="gmail-m-6196280132677980643s1"><b><span style="font-size:12.0pt">1013652
</span></b></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">avail Mem</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><span style="font-size:12.0pt"> </span></span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Strace shows the following over and over again.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="gmail-m-6196280132677980643p1"><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">strace -p 1156 -f</span></span><o:p></o:p></p>
<p class="gmail-m-6196280132677980643p1"><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt"> </span></span><o:p></o:p></p>
<p class="gmail-m-6196280132677980643p1"><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">[pid</span></span><span class="gmail-m-6196280132677980643apple-converted-space"><span style="font-size:12.0pt">
</span></span><span class="gmail-m-6196280132677980643s1"><span style="font-size:12.0pt">1159] futex(0x7fc1c15a307c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 16657, {tv_sec=1659390139, tv_nsec=255860000}, 0xffffffff) = -1 ETIMEDOUT (Connection timed
out)</span></span><o:p></o:p></p>
<p class="gmail-m-6196280132677980643p1"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Any pointers here would be greatly appreciated. I’m about at my wits end with this one, and rebuilding this server on a newer build of RHEL or recompiling BIND is not a journey
that I would like to take at the moment.<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal">-- <br>
Visit <a href="https://lists.isc.org/mailman/listinfo/bind-users" target="_blank">
https://lists.isc.org/mailman/listinfo/bind-users</a> to unsubscribe from this list<br>
<br>
ISC funds the development of this software with paid support subscriptions. Contact us at
<a href="https://www.isc.org/contact/" target="_blank">
https://www.isc.org/contact/</a> for more information.<br>
<br>
<br>
bind-users mailing list<br>
<a href="mailto:bind-users@lists.isc.org" target="_blank">bind-users@lists.isc.org</a><br>
<a href="https://lists.isc.org/mailman/listinfo/bind-users" target="_blank">https://lists.isc.org/mailman/listinfo/bind-users</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</body>
</html>