Too many open files (not only BIND 9.5.0-P1)

[Chris Thompson]

On Jul 9 2008, JINMEI Tatuya / 神明達哉 wrote:

>At Wed, 09 Jul 2008 10:56:30 -0500,
>Walter Gould <gouldwp at auburn.edu> wrote:
>
>> I upgraded this morning from 9.5.0 to 9.5.0-P1 and shortly after began 
>> receiving the below errors.  Would anybody know why?  Also - this killed 
>> external name resolution for us.  Thanks in advance.
>> 
>> Jul  9 09:17:53 dns named: named startup succeeded
>> Jul  9 09:19:24 dns named[25109]: error: socket.c:2105: unexpected error:
>> Jul  9 09:19:24 dns named[25109]: error: internal_accept: fcntl() 
>> failed: Too many open files
>
>Can you figure out how many sockets were opened in that situation?  It
>will also help if you specify your OS and version.
>
>This situation could happen if named opens so many open sockets bound
>to random UDP ports simultaneously.  But as long as the query rate and
>cache hit rate are moderate, this should be a rare event in
>practice.  So my first guess is that the system default of the maximum
>allowable open sockets is too small.  Please check the value (e.g., by
>'ulimit -n' that works for many shells), and try a larger value if
>it's too small.
>
>If it's equal to or larger than 1024 and you still see this problem,
>your operational environment might be such a rare unlucky one.  In
>that case, I'd recommend you try 9.5.1b1 (or 9.4.3b1), which handles
>such cases much better.

I have observed two occurrences of this with BIND 9.4.2-P1 running under 
Solaris 10_x86 even though plimit(1) clearly shows the named process 
has both soft and hard limits on file descriptors set to 65536:

Jul  9 20:29:09 recdns0.csx.cam.ac.uk named[18018]: [ID 873579 local7.error] 
 general: error: socket: too many open file descriptors
Jul  9 20:29:25 recdns0.csx.cam.ac.uk last message repeated 758 times

# plimit 18018
18018:  /usr/local/BIND/bind/sbin/named -u namesrvr -c ../etc/named.conf
   resource              current         maximum
[...]
  nofiles(descriptors)  65536           65536

-- 
Chris Thompson
Email: cet1 at cam.ac.uk