BIND9 SERVFAIL Issue with Windows 2008 R2 DNS Server

[David Lam]

> This turned out to be happening because Windows DNS was actually sending its query as "dig badsign-A.test.dnssec-tools.org +dnssec +cdflag", in other words telling BIND not to perform DNSSEC validation.

Agreed. Looking at a Wireshark capture, it does look like this was the
case with the Windows 2008 R2 query:

    Flags: 0x0110 Standard query
        0... .... .... .... = Response: Message is a query
        .000 0... .... .... = Opcode: Standard query (0)
        .... ..0. .... .... = Truncated: Message is not truncated
        .... ...1 .... .... = Recursion desired: Do query recursively
        .... .... .0.. .... = Z: reserved (0)
        .... .... ...1 .... = Non-authenticated data: Acceptable <--
CD flag set here

with:

    Additional records
        <Root>: type OPT
            Name: <Root>
            Type: OPT (EDNS0 option)
            UDP payload size: 4000
            Higher bits in extended RCODE: 0x0
            EDNS0 version: 0
            Z: 0x8000
                Bit 0 (DO bit): 1 (Accepts DNSSEC security RRs) <-- DO
flag set here
                Bits 1-15: 0x0 (reserved)
            Data length: 0

returning this as a result (non-validated, even with DNSSEC validation
turned on at the BIND side):

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18664
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;badsign-A.test.dnssec-tools.org. IN    A

;; ANSWER SECTION:
badsign-A.test.dnssec-tools.org. 85968 IN A     75.119.216.33

whereas Windows 2012 does have the CD flag turned off by default:

    Flags: 0x0100 Standard query
        0... .... .... .... = Response: Message is a query
        .000 0... .... .... = Opcode: Standard query (0)
        .... ..0. .... .... = Truncated: Message is not truncated
        .... ...1 .... .... = Recursion desired: Do query recursively
        .... .... .0.. .... = Z: reserved (0)
        .... .... ...0 .... = Non-authenticated data: Unacceptable <--
CD flag is off

    Additional records
        <Root>: type OPT
            Name: <Root>
            Type: OPT (EDNS0 option)
            UDP payload size: 4000
            Higher bits in extended RCODE: 0x0
            EDNS0 version: 0
            Z: 0x8000
                Bit 0 (DO bit): 1 (Accepts DNSSEC security RRs) <-- DO
flag is on
                Bits 1-15: 0x0 (reserved)
            Data length: 0

and SERVFAIL returned:

; <<>> DiG 9.9.3-P1 <<>> @127.0.0.1 badsign-A.test.dnssec-tools.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 48614
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

> It turned out not to be possible to enable DNSSEC validation on the Windows domain controller itself, because the mechanism for entering the DNS root trust anchor was also broken.

Looks that way to me  as well. From what I can see in R2, the only
trust anchor algorithm that is supported is RSA/SHA-1, not compatible
with the root's algorithm 8 (RSA/SHA-256). Looking at the algorithm
list in 2012 though, it seems like many new algorithms are now
supported.

> Based on a Microsoft tech support case that I opened, the only way to fix this was to turn off EDNS ("dnscmd /config /EnableEDnsProbes 0").
> This also seems to have been fixed in Windows Server 2012.

What a bummer, this essentially stops anyone from using DNSSEC
validation correctly on R2. And while DNSSEC validation is a useful
utility, what concerns me more is the inability for other
organizations / entities to be able to look up our DNSSEC signed
zones, especially with the fact that IPv6 is enabled by default on R2,
causing unanticipated service failures for these organizations /
entities.

Taking comcast.com as an example again, if Exchange was to send mail
to a @comcast.com address, it will do the following lookups:

; <<>> DiG 9.9.3-P1 <<>> @127.0.0.1 comcast.com MX
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19571
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 5

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;comcast.com.                   IN      MX

;; ANSWER SECTION:
comcast.com.            600     IN      MX      5 mx1.comcast.com.
comcast.com.            600     IN      MX      5 mx4.comcast.com.
comcast.com.            600     IN      MX      5 mx2.comcast.com.
comcast.com.            600     IN      MX      5 mx3.comcast.com.

;; ADDITIONAL SECTION:
mx1.comcast.com.        3600    IN      A       76.96.32.247
mx4.comcast.com.        7200    IN      A       69.241.43.118
mx2.comcast.com.        7200    IN      A       76.96.32.252
mx3.comcast.com.        7200    IN      A       69.241.43.117

----

; <<>> DiG 9.9.3-P1 <<>> @127.0.0.1 mx1.comcast.com AAAA
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 13421
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;mx1.comcast.com.               IN      AAAA

----

And mail bounces in Exchange because DNS lookup for the mail server
supposedly "failed." (it never gets to try the A records themselves).
Unless the system administrator knows about this issue, I really doubt
they will go try disabling EDNS or IPv6. Instead I think our DNS
servers would be to blame for this issue (which I half agree, due to
the tampering of the order of the RRs by BIND9 returned).
Now just hoping that there is a directive that we can use to maintain
the authority section RRs' order.

-- 
David Lam
Security Administrator
Information Educational Technology
davlam at ucdavis.edu
(530) 752-6971