how do i understand inn_feed stats?

[Russ Allbery]

Pavel V Knyazev <pasha at surnet.ru> writes:

>> requeued are articles that innfeed tried to offer and failed to
>> successfully get a yes or no response on.  spooled are articles that
>> innfeed had to spool for some reason.  The next three break down
>> spooled into three categories:  articles spooled because the peer just
>> wasn't keeping up, articles spooled when innfeed needed to close the
>> connection for some reason, and articles spooled while innfeed couldn't
>> connect to the remote host.

> What if spooled is non-zero and those three fields are zero?
> I'm seeing it at the moment :-?

Articles were spooled for some other reason.  :)

>> unspooled are articles taken back out of the spool and fed to the site.
>> no queue are articles that went through immediately; if no queue is
>> high, the feed is extremely fast and near real-time.

> It's clear now. What is avr.cxns queue?

I'm not actually sure.

>> >    seconds: 51318     art. timeout: 600          ip name: [host-2]
>> >    offered: 14288    resp. timeout: 300             port: 119
>> >   accepted: 2632    want streaming: yes      active cxns: 1
>> >    refused: 11220     is streaming: yes    sleeping cxns: 0
>> >   rejected: 270         max checks: 64      initial cxns: 1
>> >    missing: 0          no-check on: 70.0%      idle cxns: 0
>> >   deferred: 0         no-check off: 50.0%       max cxns: 1/1
>> >   requeued: 279      no-check fltr: 50.0    queue length: 0.0/200
>> >    spooled: 2668    dynamic method: 0              empty: 100.0%
>> > [overflow]: 1073     dyn b'log low: 25.0%        >0%-25%: 0.0%
>> > [on_close]: 689     dyn b'log high: 50.0%        25%-50%: 0.0%
>> > [sleeping]: 906     dyn b'log stat: 37.5%        50%-75%: 0.0%
>> >  unspooled: 0       dyn b'log fltr: 25.0       75%-<100%: 0.0%
>> >   no queue: 6909    avr.cxns queue: 10.0             full: 0.0%
>> > accpt size: 0.993 GB drop-deferred: true    defer length: 0.0
>> > rejct size: 90.5 MB  min-queue-cxn: true
>> >                  spooling: DISABLED
>> >    offered:  0.24 art/s   accepted:  0.00 art/s, 0 B/s
>> >    refused:  0.24 art/s   rejected:  0.00 art/s, 0 B/s
>> >    missing 0 spooled 0
>> 
>> This host is completely keeping up at the moment,

> How did you find out that?

Look at the queue length statistics over on the side.  Note that the queue
is empty 100% of the time.  If the feed isn't keeping up, you'll see the
percentages of the other buckets climbing, until eventually it's all in
full.  That's the fastest way of telling whether a feed is keeping up.

>> There was some time in the past when it wasn't keeping up;

> And how did you find this?

The overflow spooling numbers.

>> it looked like it was either slow or down for a while.  You don't need
>> to add more streams, but I would just turn on spooling.

> I've got a lot of questions this time ;-) yet another goes: how do i
> know - don't *I* keep up with the feed (slow connection, high latency
> and so on) or doesn't the *PEER* keep with receiving it (slow hardware
> on its side, overload and so on).

That's pretty hard to tell.  Both show up as overflow spooling.  But
usually innfeed isn't the bottleneck, and you can look at the queue size
numbers.  If innfeed is queuing up a bunch of messages it's not the one
that's falling behind.

> One of my peers accepts small articles (32K) just fine with absolutely
> no spooling, but the same peer can't receive more than 1 Gbyte of big
> articles per day (using 1 stream) and shows me a lot of spooled
> articles, another peer in the same location and the same connectivity
> receives 10 Gbytes and more under the same conditions.

That's not uncommon; the peer that's not keeping up probably has slower
disk.

> You might have noticed it was a big binary feed (although i was not
> really sending a lot at that moment) - i'm concerned by the bandwidth it
> takes, as a small site we can't propagate thousands of gigabytes,
> receiving just about 20 per day. I always do backlogging for local
> postings and do not for transit traffic for obvious reasons (excl. peers
> that do backlogging for me).

Why would spooling use up more bandwidth?  The only thing that gets
retransmitted when it tries again is the message ID, and usually with
overflow spooling you never offered the messages in the first place.

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>

    Please send questions to the list rather than mailing me directly.
     <http://www.eyrie.org/~eagle/faqs/questions.html> explains why.