[bind10-dev] msgq crash and auth died in trac327
Jeremy C. Reed
jreed at isc.org
Thu Dec 9 15:25:11 UTC 2010
I am running trac327.
b10-recurse is enabled and bind10 continually restarting (I understand).
In bindctl I did:
config set listen_on [ { "address": "127.0.0.1", "port": 5301 } ]
config commit
[bind10] Resurrecting dead b10-recurse process...
(jreed: I understand this part because was trying to bind to default
port which was already in use. I also know that there isn't a way to
restart recurse to listen on new port.)
[bind10] Resurrected b10-recurse (PID 13851)
[b10-recurse] Command line: b10-recurse -v
[b10-recurse] Server created.
[b10-recurse] Query shutdown
[b10-recurse] Query setup
[b10-recurse] IOService created.
[b10-recurse] Configuration session channel created.
[b10-cmdctl] Finish send command 'set_config' to module 'ConfigManager'
/Recurse>
/Recurse> [b10-recurse] Server failed: Timeout while reading data from
cc session
[b10-recurse] Query shutdown
[b10-recurse] Deleting the Recursor
[bind10] Process b10-recurse (PID 13851) died.
Traceback (most recent call last):
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-msgq",
line 448, in <module>
msgq.run()
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-msgq",
line 364, in run
self.run_poller()
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-msgq",
line 382, in run_poller
self.process_socket(fd)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-msgq",
line 206, in process_socket
self.process_packet(fd, sock)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-msgq",
line 273, in process_packet
self.process_command(fd, sock, routingmsg, data)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-msgq",
line 283, in process_command
self.process_command_send(sock, routing, data)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-msgq",
line 344, in process_command_send
self.send_prepared_msg(socket, msg)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-msgq",
line 311, in send_prepared_msg
sock.send(msg)
socket.error: [Errno 32] Broken pipe
Traceback (most recent call last):
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-cfgmgr",
line 69, in <module>
[b10-auth] Server failed: asynchronous read failed
[bind10] msgq channel disappeared.
Traceback (most recent call last):
Traceback (most recent call last):
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-xfrout",
line 545, in <module>
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-zonemgr",
line 578, in <module>
[bind10] msgq channel disappeared.
[bind10] msgq channel disappeared.
Traceback (most recent call last):
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-stats",
line 416, in <module>
[bind10] msgq channel disappeared.
sys.exit(main())
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-cfgmgr",
line 54, in main
[bind10] msgq channel disappeared.
[bind10] msgq channel disappeared.
[bind10] Process b10-auth (PID 13815) died.
[bind10] Resurrecting dead b10-auth process...
cm.run()
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/config/cfgmgr.py",
line 410, in run
zonemgrd.run()
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-zonemgr",
line 552, in run
xfrout_server.run()
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-xfrout",
line 518, in run
main()
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-stats",
line 404, in main
[b10-xfrin] Read of 0 bytes: connection closed
msg, env = self.cc.group_recvmsg(False)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 247, in group_recvmsg
self._module_cc.check_command(False)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/config/ccsession.py",
line 192, in check_command
subject.check()
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-stats",
line 166, in check
self._cc.check_command(False)
env, msg = self.recvmsg(nonblock, seq)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/config/ccsession.py",
line 192, in check_command
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 108, in recvmsg
msg, env = self._session.group_recvmsg(nonblock)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 247, in group_recvmsg
self.notify('check')
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-stats",
line 128, in notify
data = self._receive_full_buffer(nonblock)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 184, in _receive_full_buffer
msg, env = self._session.group_recvmsg(nonblock)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 247, in group_recvmsg
env, msg = self.recvmsg(nonblock, seq)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 108, in recvmsg
listener.update(event)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-stats",
line 142, in update
self._receive_len_data()
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 144, in _receive_len_data
[bind10] Resurrected b10-auth (PID 13855)
data = self._receive_full_buffer(nonblock)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 184, in _receive_full_buffer
env, msg = self.recvmsg(nonblock, seq)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 108, in recvmsg
return callback()
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-stats",
line 92, in __call__
new_data = self._receive_bytes(self._recv_len_size)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 132, in _receive_bytes
[bind10] msgq channel disappeared.
self._receive_len_data()
data = self._receive_full_buffer(nonblock)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 144, in _receive_len_data
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 184, in _receive_full_buffer
return self.callback(*args, **kwargs)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-stats",
line 236, in check
[bind10] Process b10-msgq (PID 13813) died.
[bind10] The b10-msgq process died, shutting down.
raise ProtocolError("Read of 0 bytes: connection closed")
isc.cc.session.ProtocolError: Read of 0 bytes: connection closed
new_data = self._receive_bytes(self._recv_len_size)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 132, in _receive_bytes
self._receive_len_data()
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 144, in _receive_len_data
[bind10] msgq channel disappeared.
[bind10] Stopping the server.
return self.cc_session.check_command(False)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/config/ccsession.py",
line 192, in check_command
raise ProtocolError("Read of 0 bytes: connection closed")
new_data = self._receive_bytes(self._recv_len_size)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 132, in _receive_bytes
isc.cc.session.ProtocolError: Read of 0 bytes: connection closed
msg, env = self._session.group_recvmsg(nonblock)
raise ProtocolError("Read of 0 bytes: connection closed")
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 247, in group_recvmsg
isc.cc.session.ProtocolError: Read of 0 bytes: connection closed
env, msg = self.recvmsg(nonblock, seq)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 108, in recvmsg
data = self._receive_full_buffer(nonblock)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 184, in _receive_full_buffer
[b10-auth] Server created.
[b10-auth] IOService created.
[b10-auth] Configuration session channel created.
self._receive_len_data()
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 144, in _receive_len_data
[b10-auth] Server failed: Unable to connect to message queue: Connection
refused
new_data = self._receive_bytes(self._recv_len_size)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 132, in _receive_bytes
raise ProtocolError("Read of 0 bytes: connection closed")
isc.cc.session.ProtocolError: Read of 0 bytes: connection closed
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/jreed/pkg/lib/python3.1/threading.py", line 509, in
_bootstrap_inner
self.run()
File "/home/jreed/pkg/lib/python3.1/threading.py", line 462, in run
self._target(*self._args, **self._kwargs)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/libexec/bind10-devel/b10-cmdctl",
line 330, in _handle_msg_from_msgq
self._module_cc.check_command(False)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/config/ccsession.py",
line 192, in check_command
msg, env = self._session.group_recvmsg(nonblock)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 247, in group_recvmsg
env, msg = self.recvmsg(nonblock, seq)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 108, in recvmsg
data = self._receive_full_buffer(nonblock)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 211, in _receive_full_buffer
raise se
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 184, in _receive_full_buffer
self._receive_len_data()
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 144, in _receive_len_data
new_data = self._receive_bytes(self._recv_len_size)
File
"/home/jreed/dnsbenchsuite/work/branches/trac327-HEAD/20101209134603/install/lib/python3.1/site-packages/isc/cc/session.py",
line 130, in _receive_bytes
data = self._socket.recv(size)
socket.error: [Errno 104] Connection reset by peer
[bind10] Process b10-cfgmgr (PID 13814) died.
[bind10] Process b10-xfrin (PID 13818) died.
[bind10] Process b10-stats (PID 13821) died.
[bind10] Process b10-auth (PID 13855) died.
[bind10] Sending SIGTERM to b10-xfrout (PID 13817).
[bind10] Sending SIGTERM to b10-zonemgr (PID 13819).
[bind10] Sending SIGTERM to b10-cmdctl (PID 13822).
[bind10] Process b10-cmdctl (PID 13822) died.
[bind10] Sending SIGKILL to b10-xfrout (PID 13817).
[bind10] Sending SIGKILL to b10-zonemgr (PID 13819).
[bind10] Process b10-xfrout (PID 13817) died.
[bind10] Process b10-zonemgr (PID 13819) died.
[bind10] All processes ended, server done.
[bind10] BIND 10 exiting
[Errno 111] Connection refused
Failed to connect with b10-cmdctl module, is it running?
(This is on CentOS Linux 5.3.)
More information about the bind10-dev
mailing list