Performance issue ( maybe )

[Bjarne Blichfeldt]

> -----Original Message-----
> From: Tom Schmitt
> Sent: 6. september 2010 09:01
> To: Users of ISC DHCP
> Subject: Re: Performance issue ( maybe )
> 
> 
> > Von: Bjarne Blichfeldt <bjb at jndata.dk>
> > Betreff: Performance issue ( maybe )
> 
> 
> > During clients startup, the servers takes a very long time  ~20-30
> > seconds> to answer a DISCOVER.
> 
> As a first step before you look for the source on the server, I would do a
> tcpdump on the server interface to make sure that the delay you are
> experiencing are really originate there and not somewhere else on the network.
Absolutely. Unfortunately, in the heat last Friday, the traces where not coordinated. If there comes a next time
the traces will get started asap.

> 
> 
> 
> 
> > è Cpu2  :  0.0%us,  0.3%sy,  0.0%ni, 1.3%id,  98%wa,  0.0%hi,  0.0%si,
> > 0.0%st
> 
> So regarding the wait I/O: first step is looging without sync, second step is
> logging only errors and higher third step is no logging at all (only for a
> testperiod to see if your problem is going away)
I have now changed to logging without sync.


> 
> > The only process really working is kjournald.
> 
> So are ther any other logmessages in high numbers beside the ones from dhcpd?
No, but I have noticed that the all the balancing lines seems to be duplicated :
Sep  3 06:53:40 b00011100859 dhcpd: balancing pool 9c8de78 10.229.16.0/24  total 248  free 124  backup 124  lts 0  max-own (+/-)25
Sep  3 06:53:40 b00011100859 dhcpd: balanced pool 9c8de78 10.229.16.0/24  total 248  free 124  backup 124  lts 0  max-misbal 37
Sep  3 06:53:40 b00011100859 dhcpd: balancing pool 9c822c8 10.229.15.0/24  total 248  free 124  backup 124  lts 0  max-own (+/-)25
Sep  3 06:53:40 b00011100859 dhcpd: balanced pool 9c822c8 10.229.15.0/24  total 248  free 124  backup 124  lts 0  max-misbal 37

And one (of many different) specific :
Sep  3 08:19:53 b00011100859 dhcpd: balanced pool 8a552c0 10.20.65.0/24  total 199  free 102  backup 88  lts 7  max-misbal 29
Sep  3 08:20:56 b00011100859 dhcpd: balanced pool 8a552c0 10.20.65.0/24  total 199  free 102  backup 88  lts 7  max-misbal 29
Sep  3 08:21:50 b00011100859 dhcpd: balanced pool 8a552c0 10.20.65.0/24  total 199  free 102  backup 88  lts 7  max-misbal 29
Sep  3 08:22:52 b00011100859 dhcpd: balanced pool 8a552c0 10.20.65.0/24  total 199  free 102  backup 88  lts 7  max-misbal 29
Sep  3 08:23:52 b00011100859 dhcpd: balanced pool 8a552c0 10.20.65.0/24  total 199  free 102  backup 88  lts 7  max-misbal 29
Sep  3 08:24:31 b00011100859 dhcpd: balanced pool 8a552c0 10.20.65.0/24  total 199  free 102  backup 88  lts 7  max-misbal 29
Sep  3 08:25:34 b00011100859 dhcpd: balanced pool 8a552c0 10.20.65.0/24  total 199  free 102  backup 88  lts 7  max-misbal 29
Sep  3 08:26:35 b00011100859 dhcpd: balanced pool 8a552c0 10.20.65.0/24  total 199  free 102  backup 88  lts 7  max-misbal 29
Sep  3 08:27:37 b00011100859 dhcpd: balanced pool 8a552c0 10.20.65.0/24  total 199  free 102  backup 88  lts 7  max-misbal 29
Sep  3 08:28:31 b00011100859 dhcpd: balanced pool 8a552c0 10.20.65.0/24  total 199  free 102  backup 88  lts 7  max-misbal 29
Sep  3 08:29:32 b00011100859 dhcpd: balanced pool 8a552c0 10.20.65.0/24  total 199  free 102  backup 88  lts 7  max-misbal 29
Sep  3 08:30:33 b00011100859 dhcpd: balanced pool 8a552c0 10.20.65.0/24  total 199  free 102  backup 88  lts 7  max-misbal 29
Sep  3 08:31:11 b00011100859 dhcpd: balanced pool 8a552c0 10.20.65.0/24  total 199  free 102  backup 88  lts 7  max-misbal 29

Also , why are there so many "balanced pool" messages ?. Is that a result of our high numbers of outstanding OFFERS ?


> 
> 
> 
> >
> > 1)      anybody seen something similar ?
> No.
> 
> >
> > 2)      Good ideas to further investigate ?  What about the network
> > topology ? Any gotcha's when sending DISCOVERY through two cisco routers ?
> Shouldn't be an isuue. I do the same without having problems.
> Or did you you just upgrade your IOS?
No new IOS

> 
> 
> Is the problem occuring on both on your dhcpd-servers at the same time? Or
> only on one of them?
Both at the same time.

> 
> And are there any messages in the log which could give you a hint?
Except for the very high number of "balanced pool", no.

> 
> 
> > Also, what would be the consensus of disabling pingcheck ?
> >
> > ping-check false;
> >
> > The ping adds at least one  second to every discovery/offer,
> 
> You would win one second, but haveing other problems. I won't recomment it.
> Better you find the real problem than working around it.
Yes, I kind of thought so. I will not change that for now.

> 
> 
> 
> 
> >         option option-150 10.11.75.10 ;
> >         filename "\\mboot.0<file:///\\mboot.0>" ;
> >         next-server 10.2.2.240 ;
> 
> Could it be, that not your server is the source of the problem, but the server
> 10.2.2.240?
No, this is just a small sample of the conf file, to show a typical example, we have 1352 subnets with different next-server statements.


Thanks for your input. It is really appreciated.

Regards,
Bjarne