cant store article: bogus Xref: header in INN 2.5 ?

[Matija Nalis]

On Wed, Aug 31, 2011 at 09:47:48PM +0200, Julien ?LIE wrote:
> Hi Matija,
> > Aug 30 05:52:54 news2 innd: SERVER bad_format newsfeed.CARNet.hr
> > Aug 30 05:52:54 news2 innd: SERVER cant store article: bogus Xref: header
> > 
> > Aug 31 03:14:37.637 - newsfeed.carnet.hr<fe278512-d9d1-4b46-976a-1cf252417112 at g30g2000vbu.googlegroups.com>  403 cant store article
> > 
> > sm `grephistory '<fe278512-d9d1-4b46-976a-1cf252417112 at g30g2000vbu.googlegroups.com>'` | grep Xref
> > Xref:  newsfeed.CARNet.hr sci.med.cardiology:211617 alt.support.diabetes:452777 misc.health.alternative:332298 alt.christnet.prayer:66586
> > 
> > Note the two spaces after "Xref:".
> 
> It could explain the problem.  I see in storage/tradspool/tradspool.c:
> 
> Yet, it does not explain why the error did not happen with an INN 2.4.6 xrefslave
> as the same code is present in the 2.4.6 version.

Don't know. I just doublechecked on third server (also xrefslave as
the upgraded one, but still running 2.4.6-snapshot-20090119) and it
shows no such errors in the logs (and it gets articles from the same 
"newsfeed" server as the upgraded one).

> > As some other random OK messages I
> > looked at give just one space after "Xref:", so that might be the cause.
> > Why it happens so sporadically I don't know...
> 
> I also do not know why two spaces are present.  I have had a look at the innd/art.c code
> and do not see where two spaces could be added.  I only see one.

newsfeed server is running cleanfeed and some custom cleanfeed.local
code, but all of it should be read-only (nothing should be modifying
article headers, it just rejects or accepts messages)

> It would be better to find out why two spaces are inserted, instead of modifying
> CrackXref to handle that weirdness.
> You are using tradspool in your xrefslave server, aren't you?  (Please tell me in case
> you are not, because CrackXref would otherwise be totally irrelevant for this bug.)

yes, all servers are runing tradspool.

> However, in the example you show, your "grep Xref" only returns *one* line,
> so it seems to be unrelated.
> Besides, the error is for "newsfeed.CARNet.hr", which is your own server, and
> not an external one.


Yes. It seems I have managed to reproduce the problem -- if I send to
newsfeed.carnet.hr article with Xref containing double spaces and
some unrelated host, like:

Message-ID: <slrnj5uvtq.3pot2.mnalis-news at newsfeed.carnet.hr>
Xref:  news.trigofacile.com misc.test:9

it seems the newsfeed running 2.4.6-snapshot-20090119 will update the
header with its own data, *but* leave the two spaces after Xref:

news at newsfeed:~$ sm `grephistory '<slrnj5uvtq.3pot2.mnalis-news at newsfeed.carnet.hr>'` | grep Xref
Xref:  newsfeed.CARNet.hr misc.test:2212835

and then such message will be propagated to news2 running 2.5.2-2~squeeze1, and it 
will promptly break.

> > I might have lived with that until I decide to upgrade newsfeed
> > (after this one shows rock stable), however today it got worse on
> > my upgraded machine:
> > 
> > Aug 31 12:00:53 news2 innd: SERVER bad_format newsfeed.CARNet.hr
> > Aug 31 12:00:53 news2 innd: SERVER throttle Interrupted system call writing SMstore file -- throttling
> > Aug 31 12:00:53 news2 innd: SERVER cant store article: bogus Xref: header
> > 
> > and now it's throttled and not accepting any new articles:
> 
> Doesn't the server work again after a manual unthrottling?  (ctlinnd go)

As this servers is currently pulled off production, I left it
throttled if there is anything interesting to see. However it seems
the nighly inn news.daily or something unthrottled it automatically
and it is again accepting articles.

> > any ideas?
> I am very puzzled by the reason of the extra space :-/

I think that mistery should be solved above...

As how to fix inn 2.5, I'm in favour of Russ idea of being more lax
and allowing multiple spaces (but maybe emiting only one when it is
rewritten).

Alternatively, reject such articles if we must (but if RFC allow it
as it seems to be the case, I'd rather accept them), but just reject
such articles instead of throttling the server.

-- 
Opinions above are GNU-copylefted.