INN not keeping up with incoming news

[Russ Allbery]

Fabien Tassin <fta at sofaraway.org> writes:

> My ARTclean() takes around 5%. I don't know for NCproc() but I can add a
> timer for it (I'll probably do so). My current goal is to reduce HISxx()
> usage. Looking at your own stats, your numbers are 2 orders of magnitude
> lower than mines. Is your server already running your new code ?

This is 2.3.1:

INND timer:
Code region              Time    Pct    Invoked   Min(ms)    Avg(ms)    Max(ms)
article cancel   00:01:31.923   0.1%      21384     0.500      4.299     16.610
article control  00:01:26.012   0.1%      20352     0.308      4.226     19.947
article link     00:00:00.000   0.0%          0     0.000      0.000      0.000
article write    00:15:41.493   1.1%     291853     0.931      3.226      7.170
history grep     00:00:08.467   0.0%      15546     0.000      0.545      2.333
history lookup   00:16:41.039   1.2%   12002464     0.004      0.083      0.188
history sync     00:00:11.027   0.0%      99648     0.077      0.111      0.181
history write    00:15:40.940   1.1%     994805     0.165      0.946      2.256
idle             17:05:26.135  71.2%   27460949     1.574      2.240      2.968
overview write   00:00:00.000   0.0%          0     0.000      0.000      0.000
perl filter      00:47:37.648   3.3%     975298     1.958      2.930      5.679
site send        00:19:17.502   1.3%     555400     1.776      2.084      2.489

TOTAL: 24:00:01  19:03:42.186  79.4%          -         -          -          -

This is 2.4.0 as of about two months ago:

INND timer:
Code region                   Time    Pct    Invoked  Min(ms)  Avg(ms)  Max(ms)
article cancel        00:05:09.810   0.4%      22628    4.965   13.691   30.879
article cleanup       03:12:45.449  13.4%     856876    6.071   13.497   23.060
article control       00:04:49.482   0.3%      21595    4.939   13.405   31.185
article link          00:00:00.000   0.0%          0    0.000    0.000    0.000
article write         00:36:45.858   2.6%     293151    2.011    7.525   15.884
hisgrep               00:00:59.000   0.1%      14130    0.000    4.176   11.383
history lookup        01:41:29.956   7.0%    4038810    0.097    1.508    5.099
history sync          00:00:28.498   0.0%      86379    0.252    0.330    0.788
history write         01:25:48.693   6.0%     862158    1.125    5.972   16.669
idle                  09:43:35.807  40.5%    8927718    2.760    3.922    9.882
overview write        00:00:00.000   0.0%          0    0.000    0.000    0.000
perl filter           01:53:56.909   7.9%     853567    6.100    8.010   10.959
python filter         00:00:00.000   0.0%          0    0.000    0.000    0.000
site send             00:12:22.310   0.9%     565139    0.950    1.313    2.056

TOTAL: 24:00:03.221   18:58:11.772  79.0%          -        -        -        -

The latter server is writing history directly to a single normal disk.
The time isn't great, but it just doesn't seem as horrible as other people
are seeing.  Of course, I'm not taking a full feed, but still, you'd think
it would be proportional....

BTW, if you do write a pure transit server, I for one would be happy to
see about having it be part of the INN package....

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>