How to Trace "TCP Receive Error"

[Barry Finkel]

On 6-Jan-08, at 11:05 AM, Barry Finkel wrote:

>> I am seeing lots of messages like this one from BIND-9.4.1-P1:
>>
>>     [ID 873579 daemon.info] dispatch b090ef8:
>>       shutting down due to TCP receive error: 69.59.189.68#53:
>>       connection reset
>>
>> I tried a Solaris snoop trace of all traffic between the DNS server
>> (which has three IP addresses) to the IP address in the message:
>>
>>      snoop -v -s3000 -o /tmp/snoop.trace 69.59.189.68
>>
>> but I did not get any packets captured.  I ran the trace for one hour,
>> and after not capturing anything, I looked in /var/adm/messages.
>> There were about 300 such messages logged.  What snoop trace  
>> parameters
>> do I have to specify to trace this activity?  I am assuming (maybe
>> incorrectly) that snoop is tracing activity on all three IP addresses.
>> I have BIND query logging on, and I do not see this address in the
>> query.log file.  Thanks.


and Dave Knight <dave at knig.ht> replied:

>Snoop will listen to the first non-loopback interface it finds, I  
>would guess in this case it has picked the wrong one.
>
>You can list the available interfaces with:
>
>	netstat -i
>
>Then instruct snoop to listen on the correct one with:
>
>	-d <interface>

I do not understand your reply.  The DNS server has three IP addresses,
and ALL THREE are advertised and in use.  So, there is no "correct" one.

oberon% netstat -i
Name  Mtu  Net/Dest      Address        Ipkts  Ierrs Opkts  Oerrs Collis Queue
lo0   8232 loopback      localhost      465553 0     465553 0     0      0
bge0  1500 oberon.it.anl.gov oberon         5358043 0     1668993 0     0      0
bge1  1500 dns2.anl.gov  dns2.anl.gov   340299637 0     154842 0     0      0   
bge2  1500 dns2.anl.gov  dns2.anl.gov   286178523 0     689428381 0     0      0

oberon%

and I have no idea what interface is being used for these queries.
The DNS server is an internal server for our anl.gov clients.  It
is inaccessible for internet queries (but it will accept response
packets), so the queries that are triggering these messages must be
from one or more internal machines here.

On the DNS server I did an "rndc dumpdb", and these records appear in 
the database dump:

     ; glue
     support-intelligence.NET. 134497 NS     dns-eu1.powerdns.net.
			     134497  NS      dns-eu2.powerdns.net.
     ; authauthority
     a.support-intelligence.NET. 1775 \-AAAA ;-$NXRRSET
     ; glue
			     1891    A       69.59.189.68
     ; authauthority
     b.support-intelligence.NET. 1775 \-AAAA ;-$NXRRSET
     ; glue
			     1891    A       69.59.189.68
     ; glue
     dob.sibl.support-intelligence.NET. 1891 NS a.support-intelligence.net.
			     1891    NS      b.support-intelligence.net.
     ; glue

     ;
     ; Unassociated entries
     ;
     ;       69.59.189.68 [srtt 374780] [flags 00000000] [ttl 1773]

I assume that the comment lines come before the data line(s).
The queries seem to be associated somehow with the domain

     support-intelligence.net

A check of our BIND query log shows lots of queries from one of our
mail machines; here is one query.

     06-Jan-2008 17:38:01.101 queries: info:
       client 146.137.96.51#41548: query:
       achilles.ctd.anl.gov.dob.sibl.support-intelligence.net IN A +

I do not have access to that mail machine, so I am copying the
administrators of that machine, who might be able to tell me why these
queries are happening.
----------------------------------------------------------------------
Barry S. Finkel
Computing and Information Systems Division
Argonne National Laboratory          Phone:    +1 (630) 252-7277
9700 South Cass Avenue               Facsimile:+1 (630) 252-4601
Building 222, Room D209              Internet: BSFinkel at anl.gov
Argonne, IL   60439-4828             IBMMAIL:  I1004994