Re: BIND 9.20.22 – Recurring SERVFAIL on Internal Recursive Resolvers
Amir Sarbaz Ahmed
amir.ahmed at sevennet.net
Sun Nov 8 08:50:43 UTC 2026
Hi Greg, Ondřej, and everyone,
Thank you for your earlier recommendations. We upgraded to *BIND 9.20.30*,
but the issue subsequently recurred. We have collected more specific
observations involving outgoing DNS COOKIEs.
The affected resolver, ns03, is currently resolving Yahoo again, so the
failure is no longer available for further testing. I am sharing the
available evidence for your assessment rather than claiming a confirmed
BIND defect. The attached PCAP is provided for reference; the summary below
also includes separately collected dig output and tcpdump excerpts.
1. Earlier reproduction on ns04 — November 3
On ns04 (192.168.41.253), an outgoing query to 68.142.255.16 contained this
complete COOKIE:
c1d6e41d3546e943010000006ae6d7e5edb59244a545f624
Interpreted using the version-1 server-cookie format, its embedded
timestamp was *November 1, 06:10:45 UTC*, approximately 55 hours before the
capture.
Direct queries from the same resolver host to that same authoritative
address produced:
- *Exact captured full cookie:* timeout.
- *Same client-cookie portion, without the server-cookie portion:*
NOERROR, approximately 80 ms.
- *No cookie:* NOERROR, approximately 81 ms.
Setting send-cookie no; and running rndc reload restored resolution.
However, because the configuration change and reload happened together, we
did not independently isolate their effects.
2. Reproduction on ns03 — November 8
On ns03 (192.168.41.153), recursive queries for yahoo.com A returned *SERVFAIL
/ EDE 22 (No Reachable Authority)* after approximately 10 seconds.
At approximately *07:01 UTC*, the outgoing queries contained server cookies
with these embedded timestamps:
Destination Embedded timestamp (UTC) Approximate age
27.123.42.42 Nov 6, 18:00:43 37 hours
202.165.97.53 Nov 5, 20:51:27 58 hours
68.180.131.16 Nov 5, 02:54:32 76 hours
98.138.11.157 Nov 4, 14:40:41 88 hours
The displayed capture excerpt contains outgoing queries to these four
addresses, without corresponding incoming responses.
Immediately beforehand, direct tests to *68.142.255.16* succeeded with no
cookie, a client-only cookie, and a recently issued full cookie. Five
additional replays of that full cookie also succeeded in *69–81 ms*.
Changing one digit in its server-cookie hash caused a timeout.
Those direct tests targeted *ns2*, whereas the failing recursive trace
showed the other four addresses. We did not complete matching replay tests
against those four destinations before ns03 recovered.
Questions
Does BIND 9.20.30’s expected server-cookie expiry and refresh behavior
explain why these days-old timestamps were present in outgoing queries? In
particular, what recovery should occur when requests carrying a remembered
server cookie repeatedly time out without a BADCOOKIE response?
Which targeted ADB or resolver diagnostics would be most useful during the
next recurrence to determine when these cookie values were learned and why
they continue to be selected?
We recognize that the timestamps do not establish how long BIND
continuously retained the cookies, and the missing responses do not
establish whether the authoritative servers or something in the network
path discarded the requests.
Thank you for your time and guidance.
Best regards,
Amir
On Wed, Oct 28, 2026 at 11:38 AM Greg <greg at isc.org> wrote:
> Hello.
> Curiosity is good. But I stand by needing to upgrade, now! Do that first
> and see if your issue still exists. But also see below.
> A few comments on your reply:
>
> - You have redacted a lot, which is generally not helpful to diagnosis.
> - Your tcpdump would have been better saved to disc, then the low
> level bits could be analysed offline in Wireshark. However...
> - You do say, twice, that no matching responses were observed from the
> Yahoo NS. I sent digs to both (see below) and received responses promptly,
> which I think supports my theory that you have network issues somewhere.
> Try manual testing from the command line of your recursive server out to
> various NS, not just Yahoo's (using their addresses, not names), log the
> results and talk to your network and firewall people.
> - Also check your server's hardware, interfaces, routeing table, what
> other process are running etc. Basic due diligence on the box itself.
> - Unless you have custom root hints in the file "named.root" you don't
> need a hint zone at all, as the Internet root hints have been built into
> BIND for a very long time.
> - Please consider disabling the stale cache, unless you really
> understand why you think you need it and how to use it. Here are some
> references:
> -
> 8. Configuration Reference — BIND 9 9.20.30 documentation
> <https://bind9.readthedocs.io/en/stable/reference.html#namedconf-statement-stale-answer-enable>
> bind9.readthedocs.io
> <https://bind9.readthedocs.io/en/stable/reference.html#namedconf-statement-stale-answer-enable>
> [image: favicon.ico]
> <https://bind9.readthedocs.io/en/stable/reference.html#namedconf-statement-stale-answer-enable>
> <https://bind9.readthedocs.io/en/stable/reference.html#namedconf-statement-stale-answer-enable>
> -
> [image: ISC-logo-rgb-2048x1149.png]
>
> Changes to BIND's serve-stale option stale-answer-client-timeout in
> BIND 9.18 and newer
> <https://kb.isc.org/docs/changes-to-serve-stale-option-stale-answer-client-timeout-in-bind-918-and-newer>
> kb.isc.org
> <https://kb.isc.org/docs/changes-to-serve-stale-option-stale-answer-client-timeout-in-bind-918-and-newer>
>
> <https://kb.isc.org/docs/changes-to-serve-stale-option-stale-answer-client-timeout-in-bind-918-and-newer>
> - https://kb.isc.org/docs/serve-stale-implementation-details
>
>
> My digs:
>
> % dig @27.123.42.42 yahoo.com
>
>
> ; <<>> DiG 9.20.24 <<>> @27.123.42.42 yahoo.com
>
> ; (1 server found)
>
> ;; global options: +cmd
>
> ;; Got answer:
>
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3646
>
> ;; flags: qr aa rd; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1
>
> ;; WARNING: recursion requested but not available
>
>
> ;; OPT PSEUDOSECTION:
>
> ; EDNS: version: 0, flags:; udp: 1272
>
> ; COOKIE: 5ff156dc3958b217010000006ae1aeaf6b65bb33e3c390f2 (good)
>
> ;; QUESTION SECTION:
>
> ;yahoo.com. IN A
>
>
> ;; ANSWER SECTION:
>
> yahoo.com. 1800 IN A 74.6.143.25
>
> yahoo.com. 1800 IN A 74.6.231.21
>
> yahoo.com. 1800 IN A 98.137.11.163
>
> yahoo.com. 1800 IN A 74.6.143.26
>
> yahoo.com. 1800 IN A 74.6.231.20
>
> yahoo.com. 1800 IN A 98.137.11.164
>
>
> ;; Query time: 206 msec
>
> ;; SERVER: 27.123.42.42#53(27.123.42.42) (UDP)
>
> ;; WHEN: Wed Oct 28 08:13:35 GMT 2026
>
> ;; MSG SIZE rcvd: 162
>
>
>
> % dig @68.142.255.16 yahoo.com
>
>
> ; <<>> DiG 9.20.24 <<>> @68.142.255.16 yahoo.com
>
> ; (1 server found)
>
> ;; global options: +cmd
>
> ;; Got answer:
>
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60657
>
> ;; flags: qr aa rd; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1
>
> ;; WARNING: recursion requested but not available
>
>
> ;; OPT PSEUDOSECTION:
>
> ; EDNS: version: 0, flags:; udp: 1272
>
> ; COOKIE: e0e11b4aac46fc8c010000006ae1aeb3064444069e6e622f (good)
>
> ;; QUESTION SECTION:
>
> ;yahoo.com. IN A
>
>
> ;; ANSWER SECTION:
>
> yahoo.com. 1800 IN A 98.137.11.163
>
> yahoo.com. 1800 IN A 74.6.143.26
>
> yahoo.com. 1800 IN A 74.6.231.20
>
> yahoo.com. 1800 IN A 74.6.143.25
>
> yahoo.com. 1800 IN A 74.6.231.21
>
> yahoo.com. 1800 IN A 98.137.11.164
>
>
> ;; Query time: 16 msec
>
> ;; SERVER: 68.142.255.16#53(68.142.255.16) (UDP)
>
> ;; WHEN: Wed Oct 28 08:13:39 GMT 2026
>
> ;; MSG SIZE rcvd: 162
>
> Cheers, Greg
>
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.isc.org/pipermail/bind-users/attachments/20261108/a47328f5/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ISC-logo-rgb-2048x1149.png
Type: image/png
Size: 512038 bytes
Desc: not available
URL: <https://lists.isc.org/pipermail/bind-users/attachments/20261108/a47328f5/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: yahoo-old-cookie-replay.pcap
Type: application/octet-stream
Size: 1090 bytes
Desc: not available
URL: <https://lists.isc.org/pipermail/bind-users/attachments/20261108/a47328f5/attachment-0001.obj>
More information about the bind-users
mailing list