Bind named 9.7.2-P2 segfault and core dump when in debug mode
Mark Andrews
marka at isc.org
Wed Sep 29 23:45:08 UTC 2010
In message <62426.10.0.66.17.1285784847.squirrel at interact.purplecow.org>, Denni
s Clarke writes:
>
> I am trying to track down a bit of strange behavior. Not sure if anyone
> else sees this.
>
> I tend to run named in the foreground and in debug level 2 for a while
> after I compile it. If all looks good then I can run it as a service
> daemon in the usual way.
>
> This means I run it like so :
>
> bash-3.00# /opt/csw/sbin/named -4 -c /etc/opt/csw/named.conf \
> > -d 2 -f -g -n 1 -p 53 -u named
>
> Note the -d 2 there.
>
> 29-Sep-2010 17:31:43.715 starting BIND 9.7.2-P2 -4 -c
> /etc/opt/csw/named.conf -d 2 -f -g -n 1 -p 53 -u named
> .
> .
> .
>
> Everything seems to be fine until I saw this after 20 minutes or so :
>
> 29-Sep-2010 17:40:35.964 error (unexpected RCODE REFUSED) resolving
> '243.136.240.111.dun.dnsrbl.net/A/IN': 66.11.124.26#53
> 29-Sep-2010 17:40:35.965 client 66.225.151.243#45979: query failed
> (SERVFAIL) for 243.136.240.111.dun.dnsrbl.net/IN/A at query.c:4650
>
>
> At this point it is just hung.
>
>
> So I started it up again in the exact same way and then went to a client
> machine and issued this query via dig :
>
> $ /opt/csw/bin/dig +qr @ns1.blastwave.org 243.136.240.111.dun.dnsrbl.net
>
> ; <<>> DiG 9.7.2-P2 <<>> +qr @ns1.blastwave.org
> 243.136.240.111.dun.dnsrbl.net
> ; (1 server found)
> ;; global options: +cmd
> ;; Sending:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 430
> ;; flags: rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
>
> ;; QUESTION SECTION:
> ;243.136.240.111.dun.dnsrbl.net. IN A
>
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 430
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
>
> ;; QUESTION SECTION:
> ;243.136.240.111.dun.dnsrbl.net. IN A
>
> ;; Query time: 235 msec
> ;; SERVER: 66.225.151.252#53(66.225.151.252)
> ;; WHEN: Wed Sep 29 17:47:25 2010
> ;; MSG SIZE rcvd: 48
>
> So that response looks okay but
>
> QUERY, status: SERVFAIL, id: 430
>
> also the named process was hung again :
>
> 29-Sep-2010 17:47:24.850 createfetch: 243.136.240.111.dun.dnsrbl.net A
> 29-Sep-2010 17:47:24.986 error (unexpected RCODE REFUSED) resolving
> '243.136.240.111.dun.dnsrbl.net/A/IN': 66.11.124.26#53
> 29-Sep-2010 17:47:25.081 lame server resolving
> '243.136.240.111.dun.dnsrbl.net' (in 'dnsrbl.net'?): 66.11.124.30#53
> 29-Sep-2010 17:47:25.082 client 66.225.151.227#24722: query failed
> (SERVFAIL) for 243.136.240.111.dun.dnsrbl.net/IN/A at query.c:4650
> Killed
>
> So I figured I would run this in debug level 4 :
>
> bash-3.00# /opt/csw/sbin/named -4 -c /etc/opt/csw/named.conf -d 4 -f -g -n
> 1 -p 53 -u named
> 29-Sep-2010 17:48:56.400 starting BIND 9.7.2-P2 -4 -c
> /etc/opt/csw/named.conf -d 4 -f -g -n 1 -p 53 -u named
> .
> .
> .
> 29-Sep-2010 17:48:56.455 loading configuration from '/etc/opt/csw/named.conf'
> 29-Sep-2010 17:48:56.470 reading built-in trusted keys from file
> '/etc/opt/csw/bind.keys'
> Segmentation Fault (core dumped)
>
> bash-3.00# file core
> core: ELF 32-bit MSB core file SPARC Version 1, from 'named'
>
> wow .. that is really not good.
>
> What failed ?
>
> bash-3.00# mdb /opt/csw/sbin/sparcv8/named core
> Loading modules: [ libc.so.1 ld.so.1 ]
> > ::status
> debugging core file of named (32-bit) from callistoz
> file: /opt/csw/sbin/sparcv8/named
> initial argv:
> /opt/csw/sbin/named -4 -c /etc/opt/csw/named.conf -d 3 -f -g -n 1 -p 53 -u
> name
> threading model: multi-threaded
> status: process terminated by SIGSEGV (Segmentation Fault)
>
> > $c
> libc.so.1`strlen+0x18(cf73c, 2000, a7cb0, fe94fc00, cf720, fe8c0d60)
> libisc.so.62`isc_log_doit+0x794(cf700, fee27ab0, c4f44, 3, 0, 0)
> libisc.so.62`isc_log_write+0x60(cf700, fee27ab0, c4f44, 3, a7cb0, a7cd8)
> set_limit+0x210(a7cb0, a7cd8, a7cd8, 9, ffffffff, fffffffd)
> set_limits+0x64(fe94fdf8, d89e0, ff0719c0, fe94fe14, a8028, d89e0)
> load_configuration+0x7d4(ffbffe22, dc758, 1, 0, ea758, 5fc28)
> run_server+0x420(ea758, e89c8, fe935840, 0, fe7e0200, fe8c21f0)
> libisc.so.62`dispatch+0x7d8(d6758, 0, 0, 0, fe7e0200, 1)
> libisc.so.62`run+0x14(d6758, fe950000, 0, 0, fedde7f0, 0)
> libc.so.1`_lwp_start(0, 0, 0, 0, 0, 0)
>
> > ::stack
> libc.so.1`strlen+0x18(cf73c, 2000, a7cb0, fe94fc00, cf720, fe8c0d60)
> libisc.so.62`isc_log_doit+0x794(cf700, fee27ab0, c4f44, 3, 0, 0)
> libisc.so.62`isc_log_write+0x60(cf700, fee27ab0, c4f44, 3, a7cb0, a7cd8)
> set_limit+0x210(a7cb0, a7cd8, a7cd8, 9, ffffffff, fffffffd)
> set_limits+0x64(fe94fdf8, d89e0, ff0719c0, fe94fe14, a8028, d89e0)
> load_configuration+0x7d4(ffbffe22, dc758, 1, 0, ea758, 5fc28)
> run_server+0x420(ea758, e89c8, fe935840, 0, fe7e0200, fe8c21f0)
> libisc.so.62`dispatch+0x7d8(d6758, 0, 0, 0, fe7e0200, 1)
> libisc.so.62`run+0x14(d6758, fe950000, 0, 0, fedde7f0, 0)
> libc.so.1`_lwp_start(0, 0, 0, 0, 0, 0)
>
> bash-3.00# pstack core
> core 'core' of 1647: /opt/csw/sbin/named -4 -c /etc/opt/csw/named.conf
> -d 3 -f -g -n 1 -p 5
> ----------------- lwp# 1 / thread# 1 --------------------
> fe8cbc3c ___sigtimedwait (ffbffbb0, 0, 0, fefe2a00, 0, 1) + 8
> fe8b4158 __posix_sigwait (ffbffbb0, ffbffb2c, 0, 0, fefe2a00, 1) + 18
> fede4aac isc__app_ctxrun (fee27fb8, c5170, c4f34, fffffffe, a5574, a5728)
> + 45c
> fede4bd8 isc__app_run (c9be0, a56fc, 0, 6e616d65, 80808080, 1010101) + 28
> 000458dc main (e, ffbffd14, ffbffd50, c3800, fefe0100, 0) + 304
> 00029188 _start (0, 0, 0, 0, 0, 0) + 108
> ----------------- lwp# 2 / thread# 2 --------------------
> fe832198 strlen (cf73c, 2000, a7cb0, fe94fc00, cf720, fe8c0d60) + 18
> fedb70bc isc_log_doit (cf700, fee27ab0, c4f44, 3, 0, 0) + 794
> fedb4958 isc_log_write (cf700, fee27ab0, c4f44, 3, a7cb0, a7cd8) + 60
> 00060088 set_limit (a7cb0, a7cd8, a7cd8, 9, ffffffff, fffffffd) + 210
> 0006010c set_limits (fe94fdf8, d89e0, ff0719c0, fe94fe14, a8028, d89e0) + 64
> 00061bec load_configuration (ffbffe22, dc758, 1, 0, ea758, 5fc28) + 7d4
> 00065be0 run_server (ea758, e89c8, fe935840, 0, fe7e0200, fe8c21f0) + 420
> fedde308 dispatch (d6758, 0, 0, 0, fe7e0200, 1) + 7d8
> fedde804 run (d6758, fe950000, 0, 0, fedde7f0, 0) + 14
> fe8c8c9c _lwp_start (0, 0, 0, 0, 0, 0)
> ----------------- lwp# 3 / thread# 3 --------------------
> fe8c8d40 __lwp_park (0, d7768, 0, 0, 6d94c, 0) + 14
> fe8c2d50 cond_wait_queue (d7798, d7768, 0, 0, 1c00, 0) + 4c
> fe8c3298 cond_wait (d7798, d7768, e6ea1, 1c00, 0, fe7cff0c) + 10
> fe8c32d4 pthread_cond_wait (d7798, d7768, 0, 0, d7768, fe8c2438) + 8
> fede2fe0 run (d7758, fe7d0000, 0, 0, fede2e10, 0) + 1d0
> fe8c8c9c _lwp_start (0, 0, 0, 0, 0, 0)
> ----------------- lwp# 4 / thread# 4 --------------------
> fe8cc300 ioctl (d9758, fe7b0000, 0, 0, fedfd550, 0) + c
> fe8c8c9c _lwp_start (0, 0, 0, 0, 0, 0)
> bash-3.00#
>
>
> It looks like thread 2 blew up in strlen ?
>
> That can't be right.
>
> If I try to start named in the foreground with anything for -d debuglevel
> > 2 the process dumps core right away with a segfault.
>
> The whole service is up and running fine right now with debuglevel 1 and
> there has been no issue.
>
> I can go to another maching and use dig to trigger the same response over
> and over :
>
> 29-Sep-2010 18:14:46.174 client 66.225.151.227#19653: query failed
> (SERVFAIL) for 243.136.240.111.dun.dnsrbl.net/IN/A at query.c:4650
>
> This is very odd behavior.
>
> Anyone have any thoughts on this at all?
>
> Dennis
If tar up named, the core and all shared libraries involved (use
relative paths please) and put upload them to
ftp://ftp.isc.org/private/debug/XXXX.tar, then open a ticket with
bind-bugs at isc.org with this message and the name of the tarball
someone will look at it.
ftp://ftp.isc.org/private/debug is write only.
> _______________________________________________
> bind-users mailing list
> bind-users at lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users
--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: marka at isc.org
More information about the bind-users
mailing list