BIND 10 #2903: need more asio error handling in asiodns
BIND 10 Development
do-not-reply at isc.org
Fri Apr 19 17:14:56 UTC 2013
#2903: need more asio error handling in asiodns
-------------------------------------+-------------------------------------
Reporter: jinmei | Owner:
Type: defect | Status: new
Priority: very high | Milestone: Next-
Component: b10-auth | Sprint-Proposed
Keywords: | Resolution:
Sensitive: 0 | CVSS Scoring:
Sub-Project: DNS | Defect Severity: N/A
Estimated Difficulty: 0 | Feature Depending on Ticket:
Total Hours: 0 | Add Hours to Ticket: 0
| Internal?: 0
-------------------------------------+-------------------------------------
Comment (by jreed):
It crashed today. Output (that was sent to logger):
{{{
terminate called after throwing an instance of
'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<asio::system_error>
>'
what(): Connection reset by peer
}}}
The time stamp for that saved output is: Apr 19 13:10
Here are my logs:
{{{
2013-04-19 12:55:10.565 ERROR [b10-auth.cc/5156] CC_TIMEOUT timeout
reading data from command channel
2013-04-19 12:55:10.566 ERROR [b10-auth.cc/5152] CC_TIMEOUT timeout
reading data from command channel
2013-04-19 12:55:10.567 ERROR [b10-auth.auth/5156] AUTH_ZONEMGR_COMMS
error communicating with zone manager: Timeout while reading data from cc
session
2013-04-19 12:55:10.568 ERROR [b10-auth.cc/5156] CC_LENGTH_NOT_READY
length not ready
2013-04-19 12:55:10.568 ERROR [b10-auth.auth/5152] AUTH_ZONEMGR_COMMS
error communicating with zone manager: Timeout while reading data from cc
session
2013-04-19 12:55:10.568 ERROR [b10-auth.auth/5156] AUTH_ZONEMGR_COMMS
error communicating with zone manager: ASIO read: data length is not ready
2013-04-19 12:55:10.569 ERROR [b10-auth.cc/5152] CC_LENGTH_NOT_READY
length not ready
2013-04-19 12:55:10.570 ERROR [b10-auth.auth/5152] AUTH_ZONEMGR_COMMS
error communicating with zone manager: ASIO read: data length is not ready
2013-04-19 13:10:20.426 INFO [b10-init.init/5145]
BIND10_LOST_SOCKET_CONSUMER consumer 31 of sockets disconnected,
considering all its sockets closed
2013-04-19 13:10:20.541 INFO [b10-init.init/5145] BIND10_PROCESS_ENDED
process 5157 of b10-auth-4 ended with status 134
2013-04-19 13:10:20.615 ERROR [b10-init.init/5145] BIND10_COMPONENT_FAILED
component b10-auth-4 (pid 5157) failed: process dumped core with exit
status 134 (killed by signal 6: SIGABRT)
2013-04-19 13:10:20.629 INFO [b10-init.init/5145] BIND10_COMPONENT_START
component b10-auth-4 is starting
2013-04-19 13:10:20.633 INFO [b10-init.init/5145] BIND10_STARTING_PROCESS
starting process b10-auth
2013-04-19 13:10:22.070 INFO [b10-auth.auth/14839] AUTH_SERVER_CREATED
server created
2013-04-19 13:10:22.070 INFO [b10-auth.auth/14839]
AUTH_DATASRC_CLIENTS_BUILDER_STARTED data source builder thread started
2013-04-19 13:10:22.173 INFO [b10-auth.auth/14839]
AUTH_DATASRC_CLIENTS_BUILDER_RECONFIGURE_STARTED data source
reconfiguration started
2013-04-19 13:10:22.174 INFO [b10-auth.auth/14839] AUTH_SERVER_STARTED
server started
2013-04-19 13:10:22.384 WARN [b10-auth.datasrc_memory/14839]
DATASRC_MEMORY_CHECK_WARNING BIND./CH: zone BIND/CH: NS has no address
records (A or AAAA)
2013-04-19 13:10:22.445 INFO [b10-auth.auth/14839]
AUTH_DATASRC_CLIENTS_BUILDER_RECONFIGURE_SUCCESS data source
reconfiguration completed successfully
}}}
Note this is using the tcp.diff patch.
Backtraces for the core file:
{{{
> thread apply all bt
Thread 2 (Thread 8032041c0 (LWP 101012)):
#0 0x000000080287503c in thr_kill () from /lib/libc.so.7
#1 0x00000008029111cb in abort () from /lib/libc.so.7
#2 0x0000000802462fa4 in __gnu_cxx::__verbose_terminate_handler ()
from /usr/lib/libstdc++.so.6
#3 0x00000008024674a3 in std::set_unexpected () from
/usr/lib/libstdc++.so.6
#4 0x00000008024674e3 in std::terminate () from /usr/lib/libstdc++.so.6
#5 0x000000080246744a in __cxa_throw () from /usr/lib/libstdc++.so.6
#6 0x0000000800f972c1 in boost::throw_exception<asio::system_error> (
e=@0x7fffffffd300) at throw_exception.hpp:61
#7 0x0000000800f97453 in asio::detail::do_throw_error
(err=@0x7fffffffd370)
at throw_error.ipp:32
#8 0x0000000800f974a8 in asio::detail::throw_error (err=@0x7fffffffd370)
at throw_error.hpp:34
#9 0x0000000801238e96 in asio::basic_socket<asio::ip::tcp,
asio::stream_socket_service<asio::ip::tcp> >::close (this=0x803337760) at
basic_socket.hpp:264
#10 0x00000008012239de in isc::asiodns::TCPServer::operator() (
this=0x7fffffffe008, ec=
{value_ = 0, category_ = asio::error::system_category}, length=81)
at tcp_server.cc:252
#11 0x0000000801236d13 in
asio::detail::write_op<asio::basic_stream_socket<asio::ip::tcp,
asio::stream_socket_service<asio::ip::tcp> >,
boost::array<asio::const_buffer, 2l>, asio::detail::transfer_all_t,
isc::asiodns::TCPServer>::operator() (this=0x7fffffffdfb0,
ec=@0x7fffffffe108, bytes_transferred=81, start=0)
at write.hpp:148
#12 0x0000000801236f02 in
asio::detail::binder2<asio::detail::write_op<asio::basic_stream_socket<asio::ip::tcp,
asio::stream_socket_service<asio::ip::tcp> >,
boost::array<asio::const_buffer, 2l>, asio::detail::transfer_all_t,
isc::asiodns::TCPServer>, asio::error_code, unsigned long>::operator()
(this=0x7fffffffdfb0)
at bind_handler.hpp:96
#13 0x0000000801236fac in
asio::asio_handler_invoke<asio::detail::binder2<asio::detail::write_op<asio::basic_stream_socket<asio::ip::tcp,
asio::stream_socket_service<asio::ip::tcp> >,
boost::array<asio::const_buffer, 2l>, asio::detail::transfer_all_t,
isc::asiodns::TCPServer>, asio::error_code, unsigned long> > (
function=@0x7fffffffdfb0) at handler_invoke_hook.hpp:63
#14 0x0000000801237000 in
asio_handler_invoke_helpers::invoke<asio::detail::binder2<asio::detail::write_op<asio::basic_stream_socket<asio::ip::tcp,
asio::stream_socket_service<asio::ip::tcp> >,
boost::array<asio::const_buffer, 2l>, asio::detail::transfer_all_t,
isc::asiodns::TCPServer>, asio::error_code, unsigned long>,
isc::asiodns::TCPServer> (function=@0x7fffffffe1b0,
context=@0x7fffffffe208)
at handler_invoke_helpers.hpp:39
#15 0x0000000801237071 in
asio::detail::asio_handler_invoke<asio::detail::binder2<asio::detail::write_op<asio::basic_stream_socket<asio::ip::tcp,
asio::stream_socket_service<asio::ip::tcp> >,
boost::array<asio::const_buffer, 2l>, asio::detail::transfer_all_t,
isc::asiodns::TCPServer>, asio::error_code, unsigned
long>,asio::basic_stream_socket<asio::ip::tcp,
asio::stream_socket_service<asio::ip::tcp> >,
boost::array<asio::const_buffer, 2l>, asio::detail::transfer_all_t,
isc::asiodns::TCPServer> (function=@0x7fffffffe1b0,
this_handler=0x7fffffffe1b0)
at write.hpp:289
#16 0x00000008012370a5 in
asio_handler_invoke_helpers::invoke<asio::detail::binder2<asio::detail::write_op<asio::basic_stream_socket<asio::ip::tcp,
asio::stream_socket_service<asio::ip::tcp> >,
boost::array<asio::const_buffer, 2l>, asio::detail::transfer_all_t,
isc::asiodns::TCPServer>, asio::error_code, unsigned long>,
asio::detail::write_op<asio::basic_stream_socket<asio::ip::tcp,
asio::stream_socket_service<asio::ip::tcp> >,
boost::array<asio::const_buffer, 2l>, asio::detail::transfer_all_t,
isc::asiodns::TCPServer> > (function=@0x7fffffffe1b0,
context=@0x7fffffffe1b0) at handler_invoke_helpers.hpp:39
#17 0x000000080123717f in
asio::detail::reactive_socket_send_op<asio::detail::consuming_buffers<asio::const_buffer,
boost::array<asio::const_buffer, 2l> >,
asio::detail::write_op<asio::basic_stream_socket<asio::ip::tcp,
asio::stream_socket_service<asio::ip::tcp> >,
boost::array<asio::const_buffer, 2l>, asio::detail::transfer_all_t,
isc::asiodns::TCPServer> >::do_complete (owner=0x8032271d0,
base=0x803308600) at reactive_socket_send_op.hpp:102
#18 0x0000000800f87495 in
asio::detail::task_io_service_operation::complete (
this=0x803308600, owner=@0x8032271d0) at
task_io_service_operation.hpp:34
#19 0x0000000800f96d67 in asio::detail::task_io_service::do_one (
this=0x8032271d0, lock=@0x7fffffffe480,
this_idle_thread=0x7fffffffe490)
at task_io_service.ipp:277
#20 0x00000008013bf56a in asio::detail::task_io_service::run (
this=0x8032271d0, ec=@0x7fffffffe4f0) at task_io_service.ipp:129
#21 0x00000008013bf816 in asio::io_service::run (this=0x80320d1e0)
at io_service.ipp:56
#22 0x00000008013bf845 in isc::asiolink::IOServiceImpl::run
(this=0x80320d1e0)
at io_service.cc:45
#23 0x00000008013be398 in isc::asiolink::IOService::run (this=0x803204380)
at io_service.cc:81
#24 0x00000000004463d4 in main (argc=1, argv=0x7fffffffea30) at
main.cc:250
Thread 1 (Thread 8032551c0 (LWP 101574)):
#0 0x000000080214f2bc in __error () from /lib/libthr.so.3
#1 0x000000080214d3b5 in pthread_cond_signal () from /lib/libthr.so.3
#2 0x0000000801973fa9 in isc::util::thread::CondVar::wait
(this=0x803204418,
mutex=@0x803204420) at sync.cc:239
#3 0x000000000043d53e in
isc::auth::datasrc_clientmgr_internal::DataSrcClientsBuilderBase<isc::util::thread::Mutex,
isc::util::thread::CondVar>::run (
this=0x803204440) at datasrc_clients_mgr.h:484
#4 0x0000000000433be2 in boost::_mfi::mf0<void,
isc::auth::datasrc_clientmgr_internal::DataSrcClientsBuilderBase<isc::util::thread::Mutex,
isc::util::thread::CondVar> >::operator() (this=0x80322cac0,
p=0x803204440)
at mem_fn_template.hpp:49
#5 0x0000000000433fa1 in
boost::_bi::list1<boost::_bi::value<isc::auth::datasrc_clientmgr_internal::DataSrcClientsBuilderBase<isc::util::thread::Mutex,
isc::util::thread::CondVar>*> >::operator()<boost::_mfi::mf0<void,
isc::auth::datasrc_clientmgr_internal::DataSrcClientsBuilderBase<isc::util::thread::Mutex,
isc::util::thread::CondVar> >, boost::_bi::list0> (this=0x80322cad0,
f=@0x80322cac0,
a=@0x7fffffbfeeff) at bind.hpp:253
#6 0x0000000000433fe6 in boost::_bi::bind_t<void, boost::_mfi::mf0<void,
isc::auth::datasrc_clientmgr_internal::DataSrcClientsBuilderBase<isc::util::thread::Mutex,
isc::util::thread::CondVar> >,
boost::_bi::list1<boost::_bi::value<isc::auth::datasrc_clientmgr_internal::DataSrcClientsBuilderBase<isc::util::thread::Mutex,
isc::util::thread::CondVar>*> > >::operator() (this=0x80322cac0)
at bind_template.hpp:20
#7 0x000000000043400d in
boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void,
boost::_mfi::mf0<void,
isc::auth::datasrc_clientmgr_internal::DataSrcClientsBuilderBase<isc::util::thread::Mutex,
isc::util::thread::CondVar> >,
boost::_bi::list1<boost::_bi::value<isc::auth::datasrc_clientmgr_internal::DataSrcClientsBuilderBase<isc::util::thread::Mutex,
isc::util::thread::CondVar>*> > >, void>::invoke
(function_obj_ptr=@0x80322cac0)
at function_template.hpp:153
#8 0x0000000800f98f91 in boost::function0<void>::operator()
(this=0x80322cab8)
at function_template.hpp:1013
#9 0x0000000801978342 in isc::util::thread::Thread::Impl::run (
impl_raw=0x80322cab0) at thread.cc:71
#10 0x0000000802145511 in pthread_getprio () from /lib/libthr.so.3
#11 0x0000000000000000 in ?? ()
Cannot access memory at address 0x7fffffbff000
#0 0x000000080287503c in thr_kill () from /lib/libc.so.7
}}}
--
Ticket URL: <http://bind10.isc.org/ticket/2903#comment:4>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development
More information about the bind10-tickets
mailing list