How Somebody Helped Kill dhcpd on Our Network

[Tim Peiffer]

I apologize for the mis-config.  Access_IN is our standard access list 
naming and I called the filter No_DHCP_SERVER to be more descriptive..

Tim

interface G1/0/1
  ip access-group No_DHCP_SERVER in
!
ip access-list extended No_DHCP_SERVER
  deny udp any eq bootps any log
  permit ip any any


Tim Peiffer wrote:
> We used to have a problem with rogue servers when the big marketing push 
> for NAT routers and wireless complete with DHCP began hitting the 
> street.   Since then, we have placed filters on the edge ports to deny 
> DHCP server traffic.  Now we only have an issue if a new server pops 
> up.  The clients can't get to their server until we remove the filter 
> from the server port.
>
> This isn't rate limiting, but it is quite effective at handling the 
> original source of the problem
>
>
> In Cisco parlance:
>
> interface G1/0/1
>   ip access-group No_DHCP_SERVER in
> !
> ip access-list extended Access_IN
>   deny udp any eq bootps any log
>   permit ip any any
>
> Tim Peiffer
>
> Darren wrote:
>   
>> Martin,
>>
>> I suggest Vlans.  Set up as many Vlans as you can and separate the user 
>> traffic.  The rogue DHCP server may not have been malicious in nature.  
>> All someone has to do is hook a linksys router up backwards and this 
>> will happen.  Its worse when all 10,000 users are in the same broadcast 
>> domain.  If it was one vlan of 253 users or something, the damage would 
>> have been minimal.  The rogue DHCP server would not have seen the vast 
>> majority of user traffic, and therefore would not have been able to 
>> respond to it with DHCPNAK.
>>
>>
>> Martin McCormick wrote:
>>   
>>     
>>> 	We recently had our dhcp V3.0.3 system crash or, I should
>>> say, crashed by a denial-of-service scheme in which the miscreant
>>> ran something on his/her computer that behaved like a dhcp server
>>> enough to send DHCPNAK's to every dhcp request it saw on the
>>> VLAN.  This made every client send more traffic to the real dhcp
>>> server and apparently caused the 1-gig processor with 1 gig of
>>> RAM to consume all available memory.  For 3 minutes, it generated
>>> about 2,500 "out of memory" messages and then dhcpd finally
>>> exited with a 11) SIGSEG and a core dump.  Good old FreeBSD unix
>>> is pretty bullet-proof, but this was more than the box could
>>> handle.  The platform continued to operate properly afterwards,
>>> but dhcpd had to be restarted again.
>>>
>>> 	We are in the process of instituting dhcp failover with a
>>> second server although I suspect that this situation would have
>>> added maybe another few minutes to the mayhem before it, to,
>>> succumbed to the hammering since failover isn't going to protect
>>> against external attacks that hit both servers equally.
>>>
>>> 	We are also installing switches with port snooping
>>> capabilities as funds permit to kill off anybody who aspires to
>>> run dhcpd on anything other than the proper dhcp server.  This
>>> brings me to my question.
>>>
>>> 	Is there any particular configuration parameter I can use
>>> in dhcpd to make it rate-limit itself since it would have been a
>>> much better outcome for it to come up for air, so to speak,
>>> rather than crash and have to be restarted.
>>>
>>> 	This particular event occurred on a Sunday afternoon on
>>> the weekend that classes finished for the Summer semester and
>>> campus activity was as near to dead as it ever gets around here
>>> so we didn't even know anything was wrong for a period of time.
>>>
>>> 	Our dhcp server gives service to around 10,000 clients of
>>> several types and the platform it is on hardly breaks a sweat
>>> even on the busiest days, but this was too much to handle.
>>>
>>> Martin McCormick WB5AGZ  Stillwater, OK 
>>> Systems Engineer
>>> OSU Information Technology Department Network Operations Group
>>>   
>>>     
>>>