innfeed - "blocked sleeping 120" errors"

[Petr Novopashenniy]

Hi, Julien!

On Tue, 14 Apr 2009, Julien ?LIE wrote:

JL> Hi Petr,
JL> 
JL> > I am also concerned that:
JL> > 
JL> > PID USERNAME THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
JL> > 865 news       1  97    0   380M   313M select 1  22.3H  7.81% innd
JL> > 74712 news     1  97    0   259M   257M select 1 216:57  4.30% innfeed
JL> > 74713 news     1  96    0   491M   489M select 1 177:02  2.54% innfeed
JL> > 74714 news     1  96    0 17796K 15760K select 0  64:43  0.34% innfeed
JL> > 
JL> > innfeed-current uses a very large amount of memory.
JL> > 
JL> > I have conducted several tests in our test lab, but I could not get a
JL> > memory leak..
JL> 
JL> Maybe it could become from a thing I fixed on 28 March 2009:
JL> 
JL> Remove useless xstrdup() calls in vector_add().  Memory has been
JL> leaking in both innfeed (since commit 8176) and nnrpd (since
JL> a long time).
JL> 
JL> Index: C:/Users/Julien/Downloads/svn/INN/trunk/innfeed/host.c
JL> ===================================================================
JL> --- C:/Users/Julien/Downloads/svn/INN/trunk/innfeed/host.c (revision 8393)
JL> +++ C:/Users/Julien/Downloads/svn/INN/trunk/innfeed/host.c (revision 8394)
JL> @@ -3369,20 +3369,20 @@
JL>   if (host->ipAddrs) {
JL>     int  i;
JL>     char ip_addr[INET6_ADDRSTRLEN];
JL> -    char *family;
JL> +    const char *family;
JL> 
JL>     for(i = 0; host->ipAddrs[i] != NULL; i++) {
JL>       switch(host->ipAddrs[i]->sa_family) {
JL>         case AF_INET:
JL> -          family = xstrdup("IPv4");
JL> +          family = "IPv4";
JL>           break;
JL> #ifdef HAVE_INET6
JL>         case AF_INET6:
JL> -          family = xstrdup("IPv6");
JL> +          family = "IPv6";
JL>           break;
JL> #endif
JL>         default:
JL> -          family = xstrdup("????");
JL> +          family = "????";
JL>           break;
JL>       }
JL> 
JL> 
JL> Could you please try it?  Though it should probably not eat 300 MB if
JL> leaking!
JL> 
JL> Incidentally, here are the changes for innfeed since INN 2.4.0:
JL> 
JL>    http://inn.eyrie.org/trac/log/trunk/innfeed/?action=stop_on_copy&mode=stop_on_copy&rev=8394&stop_rev=6300
JL> 
JL> 
JL> I do not know what could have generated a problem here.  It is perhaps not
JL> something in this directory but in common INN libraries (?)
JL> 

Now I upgrade from inn-CURRENT-20090327 to inn-CURRENT-20090414, and move 
6 broken peers from one innfeed process to another (for broken peers 
only). And now I look over the situation.

Last night innfeed with pid 74713 crash with signal 10 (maybe out of 
memory), 74712 work fine but up to ~300 MBytes

PS: 
After upgrade, I made a mistake in newsfeeds file, not start new innfeed 
for broken peers (not add these lines):

innfeed-3!\
        :!*\
        :Tc,Wnm*:/usr/local/news/bin/innfeed -c innfeed-3.conf

And innd crash, this is log:

Apr 15 14:14:30 goblin innd: solace.miun.se funnel_bad                                             
Apr 15 14:14:30 goblin innd: surnet.ru funnel_bad                                                  
Apr 15 14:14:30 goblin innd: ya.com funnel_bad                                                     
Apr 15 14:14:30 goblin innd: news.bbh.net.cn funnel_bad                                            
Apr 15 14:14:30 goblin innd: warnet.at funnel_bad                                                  
Apr 15 14:14:30 goblin innd: garr.it funnel_bad                                                    
Apr 15 14:14:30 goblin innd: controlchan! closed                                                   
Apr 15 14:14:30 goblin innd: innfeed! closed                                                       
Apr 15 14:14:30 goblin innd: innfeed-2! closed  
<skip tail>

And this is gdb output:

#0  0x282d3674 in Perl_pop_scope ()
   from /usr/local/lib/perl5/5.8.8/mach/CORE/libperl.so
[New LWP 100303]
(gdb) bt
#0  0x282d3674 in Perl_pop_scope ()
   from /usr/local/lib/perl5/5.8.8/mach/CORE/libperl.so
#1  0x2825a1bb in perl_destruct ()
   from /usr/local/lib/perl5/5.8.8/mach/CORE/libperl.so
#2  0x08069aa9 in PerlClose () at perl.c:237
#3  0x08059227 in JustCleanup () at innd.c:280
#4  0x0805ee47 in SITEparsefile (StartSite=true) at newsfeeds.c:943
#5  0x08059b1e in main (ac=134813491, av=0x0) at innd.c:651
(gdb) 

This is not fatal error, but maybe this is not graceful shutdown?


JL> -- 
JL> Julien ?LIE
JL> 

--pety