Massive increase of SERVFAIL after April 28th 2025.

[Michael Richardson]

vincent at cojot.name wrote:
    > I've been running bind (since bind 4) in my home/lab for around 3 decades. I
    > went that route because I wanted: A) to be abstracted from the ISP's DNS
    > servers and B) to have a local DNS cache.

Ditto.

    > I run 'bind9.16' on RHEL/Linux using the default bundled root zone
    > (/var/named/named.ca) and the default root key (/etc/named.root.key).

And you have DNSSEC validation turned on?

    > This has worked well for years until a few days ago (April 28th?) when the
    > amount of SERVFAIL started going ballistic and started preventing the
    > resolution of a lot of DNS names on the internet to the point where DNS was
    > unusable..

    > Starting April 28th, I started seeing tons of things like this in the auth
    > log:
    > 28-Apr-2025 00:13:03.714 lame-servers: info: SERVFAIL unexpected RCODE
    > resolving 'github.com/A/IN': 198.51.44.8#53
    > 28-Apr-2025 00:13:03.720 lame-servers: info: SERVFAIL unexpected RCODE
    > resolving 'github.com/A/IN': 205.251.197.3#53
    > 28-Apr-2025 00:13:03.725 lame-servers: info: SERVFAIL unexpected RCODE
    > resolving 'github.com/A/IN': 198.51.45.8#53

All of these look like valid DNS servers for github.
Of course, AWS/GITHUB can't be bothered to do DNSSEC.
From my vantage point, they all seem to resolve github.com

My guess is that something is in the way, and it's probably trying to
attack you (or your ISP) with fake replies, but it's doing a bad job.

When I do:
     dig +short +nsid version.bind. txt ch dns4.p08.nsone.net

I get:
  "9.21.2-1+0~20241120.131+debian12~1.gbpa6576d-Debian"

If you get something different, then that would be consistent with something
else intercepting your traffic.

    > I struggled for a couple days to bring my DNS servers back into service and
    > the -ONLY- thing which worked was to declare some 'forwarders' (Google +
    > CloudFlare). Nothing else brought reliable DNS service back.

:-(
But that does suggest that something else is in the way.
Did you forward with Do53, or did you use DoT/DoH?
{No idea if bind can forward over DoH, I never tried}

    > - I tried to turn off dnssec completely but that barely made a difference:

    > dnssec-enable no;
    > dnssec-validation no;

Won't matter, since github doesn't do DNSSEC, so the NXDOMAINs can't be
validated (or rejected as invalid)

    > The only way to get back to a working state is to add back some forwarders.

    > Any ideas? Am I doing anything wrong? I'm attaching a sanitized copy of my
    > named.conf in case someone could spot something:

I think you did everything right.
I think talking to your upstream ISP is in order.

    > Thank you for your attention,

    > ,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,
    > Vincent S. Cojot, Computer Engineering. STEP project. _.,-*~'`^`'~*-,._.,-*~
    > Ecole Polytechnique de Montreal, Comite Micro-Informatique. _.,-*~'`^`'~*-,.

Bonjour!
Elbows Up.



--
Michael Richardson <mcr+IETF at sandelman.ca>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 511 bytes
Desc: not available
URL: <https://lists.isc.org/pipermail/bind-users/attachments/20250501/0bf7d553/attachment.sig>