strange core dump

[sthaug at nethelp.no]

> Today morning our watchogs complained that two of our DNS servers
> had died. I am still running 8.1.2 on both of them, but I am curious
> if anyone has seen this before, if it is kind of an exploit? What
> puzzles me is that they both died within 10 seconds.
...
> Sep 18 07:10:09 fano named[3867]: ns_main.c:672: INSIST(evRead(lev, rfd, &iov, 1, stream_getlen, sp, &sp->evID_r) != -1): Invalid argument failed.
> Sep 18 07:10:16 fano /kernel: pid 3867 (named), uid 0: exited on signal 6 (core dumped)

What OS the boxes running? Below is a message which may be relevant if
you're running Solaris. Note that this refers to 8.2.x, but earlier
messages in the same thread specifically mentioned that it had happened
with 8.1.2 also.

Steinar Haug, Nethelp consulting, sthaug at nethelp.no
----------------------------------------------------------------------
From: marka at isc.org
Message-Id: <199905240019.KAA05314 at bsdi.dv.isc.org>
To: Juergen Georgi <georgi at belwue.de>
Cc: bind-bugs at isc.org
Subject: Re: 8.2.1-T1A dies with INSIST(evRead ...) (was Re: [BIND-BUGS #23] ...) 
In-reply-to: Your message of "Sun, 23 May 1999 22:19:30 +0200."
             <19990523221930.52947 at belwue.de> 
Date: Mon, 24 May 1999 10:18:18 +1000


> 
> --G4iJoqBmSsgzjUCe
> Content-Type: text/plain; charset=us-ascii
> 
> Attached is another truss record of a dieing named.
> 
> May 23 21:09:01 noc2.BelWue.DE named[3539]: ns_main.c:693: INSIST(evRead(lev,
>  rfd, &iov, 1, stream_getlen, sp, &sp->evID_r) != -1): Invalid argument faile
> d.
> 
> -Juergen
> 
	Well someone has changed the default file limits on this
	box.  Solaris 2.6 by default has a soft limit of 64 and a
	hard limit of 1024.  From the trace below the limit has
	been set higher than this accept() returned 1024 and open()
	1025.  FD_SETSIZE on Soalris 2.6 is 1024 by default.  The
	event library will complain if it handed a file descriptor
	>= FD_SETSIZE.

	The limit could be being set by a startup script.

	Apply the patch, you should see named logging that it has
	lowered the file limit.  Then fix the startup script, any
	limits that may need adjusting can be done via named.conf.

	Mark

Index: src/bin/named/ns_config.c
===================================================================
RCS file: /proj/cvs/isc/bind/src/bin/named/ns_config.c,v
retrieving revision 8.71
diff -c -r8.71 ns_config.c
*** ns_config.c	1999/05/14 18:00:07	8.71
--- ns_config.c	1999/05/24 00:05:02
***************
*** 1243,1248 ****
--- 1243,1262 ----
  	if (getrlimit(RLIMIT_NOFILE, &initial_num_files) < 0)
  		ns_warning(ns_log_config, "getrlimit(NOFILE): %s",
  			   strerror(errno));
+ 	if (initial_num_files.rlim_cur > FD_SETSIZE) {
+ 		initial_num_files.rlim_cur = FD_SETSIZE;
+ 		if (initial_num_files.rlim_cur > initial_num_files.rlim_max)
+ 			initial_num_files.rlim_max = FD_SETSIZE;
+ 		if (setrlimit(RLIMIT_NOFILE, &initial_num_files) < 0) {
+ 			ns_warning(ns_log_config, "setrlimit(files): %s",
+ 				   strerror(errno));
+ 			return;
+ 		} else {
+ 			ns_warning(ns_log_config,
+ 				   "limit \"files\" lowered to FD_SETSIZE (%d)",
+ 				   FD_SETSIZE);
+ 		}
+ 	}
  # endif
  }
  
***************
*** 1298,1303 ****
--- 1312,1319 ----
  		if (value == 0)
  			limits = initial_num_files;
  		/* XXX check < FD_SETSIZE? */
+ 		if (value > FD_SETSIZE)
+ 			limits.rlim_cur = limits.rlim_max = value = FD_SETSIZE;
  		break;
  	default:


[ truss output deleted - jg ]

--
Mark Andrews, Internet Software Consortium
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: marka at isc.org