USADOTGOV.NET Root Problems?

[Warren Kumari]

On Jul 26, 2010, at 12:34 AM, Kevin Oberman wrote:

>> From: Warren Kumari <warren at kumari.net>
>> Date: Sun, 25 Jul 2010 11:22:46 +0200
>> Sender: bind-users-bounces+oberman=es.net at lists.isc.org
>> 
>> 
>> On Jul 25, 2010, at 4:33 AM, Danny Mayer wrote:
>> 
>>> On 7/24/2010 5:10 AM, Warren Kumari wrote:
>>>> 
>>>> On Jul 23, 2010, at 2:37 PM, Danny Mayer wrote:
>>>> 
>>>>> On 7/22/2010 11:08 PM, Merton Campbell Crockett wrote:
>>>>>> Thanks for the confirmation that the problem was related to DNSSEC.
>>>>>> 
>>>>>> I didn't see your message until I got home from work; however, I did
>>>>>> find the root of the problem late this afternoon.  At each of our
>>>>>> Internet egress and ingress points, we have Cisco ASA devices sitting in
>>>>>> front of a pair of redundant firewalls.  Each ASA is configured with the
>>>>>> default DNS inspect policy that doesn't accept fragmented UDP packets.
>>>>> 
>>>>> Why would any inspection policy not allow fragmented UDP packets?
>>>>> There's nothing wrong with that.
>>>> 
>>>> 
>>>> Because it's "hard".... The issue is that then you need to buffer
>>> fragments until you get a full packet -- which leaves you open to
>>> attacks that send a bunch of fragments but leave one of them out.
>>>> 
>>>> Vendors like to avoid reassembling fragments by default, because it
>>> makes their performance numbers better....
>>> 
>>> At the expense of correct behavior and loss of real performance.
>> 
>> Yes. 
>> 
>> Sorry, if I gave the impression that I was condoning this -- I'm not.
>> 
>> Vendors exist to sell boxen -- tuning for the test cases at the
>> expense of correctness often wins....
> 
> And, as tests start to include DNSSEC (and EDNS0) tests, the vendors will
> likely adjust defaults. Tests for DNSSEC are already appearing on
> federal systems (not a trivial part of the business) and will likely
> become general test in the procurement process in the next year. 
> 
> Of course, changing defaults will take longer to change.
> 
> Now to a more basic question...why the ^@#$ does everyone put STATEFUL
> firewalls in front of servers.

i suspect that this was intended as a rhetorical question / rant, but I'll ignore that and answer it anyway :-)

Folk seems to do this for 3 reasons:
1: They truly believe that a stateful firewall will protect their server, either from port 53 attacks, or attacks on other ports.
2: They went through some certification process that demands that all "services" live behind a "firewall"
3: (and this is the common one) The folk that run the DNS servers are not the "network security" folk. The network security folk believe that the sysadmin types are unable to run a secure service that can be exposed to the Internet. The DNS folk may / probably have tried explaining that this firewall causes more issues than it solves, but what management hear is "The DNS folk think that they can run a secure service, but network folk think that they need more security." and they err on the side of "caution"...



> They are a denial of service attack
> waiting to happen.

Yup.

> I don't know of any highly regarded security expert
> who recommends them and most object to them rather strongly.

Yup.


> 
> I will admit to once having stateful firewalls in front of my DNS
> servers, but after an unfortunate case of a badly written application
> DOSing ourselves, stateful firewalls have been removed. Yes, the software
> needed fixing, but the software was not enough to cause any problem for
> the servers...just the firewall. And, yes, we still have stateless
> firewalls in front of our DNS servers and other public servers as well
> as an aggressive IDS/IPS system.
> -- 
> R. Kevin Oberman, Network Engineer
> Energy Sciences Network (ESnet)
> Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab)
> E-mail: oberman at es.net			Phone: +1 510 486-8634
> Key fingerprint:059B 2DDF 031C 9BA3 14A4  EADA 927D EBB3 987B 3751