XOVER failure

[Julien ÉLIE]

Hi Sam,

> I'm seeing something I do not understand. tradindexed overview.
>
> 200 commodore.email-scan.com InterNetNews NNRP server INN 2.5.0 ready (posting ok)
> XHDR Lines 584171-
> 221 Header information for Lines follows (from articles)
> 584171 1
> .
> XOVER 584171-584171
> 224 No articles in 584171-584171
> .
> XOVER 584171-584171
> 224 Overview information for 584171-584171 follows
> 584171 test message sam at email-scan.com Fri, 7 May 2010 03:25:42
> +0000 (UTC) <hs017m$opu$1 at commodore.email-scan.com>  558 1
> Xref: commodore.email-scan.com news.admin.net-abuse.email:584171
> .

It is indeed a bug I can also see in INN 2.5.2.
It is in fact a long-standing bug.

That's great to have spotted it!!


When you add a new article, the high water mark is increased.
Here to 584171.  But the cached index and the cached data files are not
immediately remapped to reflect the change.
When doing an overview search with (X)OVER, both the index and data files
are remapped, if need be.  The same for (X)HDR or XPAT *when* using the
overview.

Here, as "(from articles)" is replied by the server, the search is done
in the article itself and does not call an OVopensearch().  I see that
the code uses OVgetartinfo().  After having dug into it, I see that
OVgetartinfo() calls tdx_article_entry() which remaps the index file
but *not* the data file.  Therefore, the high water mark in the cached
data file is still mapped to 584170 after (X)HDR but tdx_article_entry()
updates data->high to be 584171...
Then (X)OVER is used.  It believes the cached data information is right.
But obviously does not find the last article.
Incidentally, you will notice in your logs something like:

May  7 22:47:44 news nnrpd[15094]: Invalid or inaccessible entry for article 424 in 
/home/news/spool/overview/t/t/trigofacile.test.IDX: offset 113737 length 564 datalength 113737

and the cached data file then is marked as to-be-remapped.
Which is done when doing the subsequent overview search with (X)OVER.


I hope my explanation is clear enough.

Here is a fix in tdx_article_entry() so that the remap is done
when the first search happens:


Index: storage/tradindexed/tdx-data.c
===================================================================
--- storage/tradindexed/tdx-data.c      (révision 9040)
+++ storage/tradindexed/tdx-data.c      (copie de travail)
@@ -425,6 +425,8 @@
         unmap_index(data);
         map_index(data);
         data->high = high;
+        /* Mark the data file to be remapped for the next opensearch. */
+        data->remapoutoforder = true;
     } else if (innconf->nfsreader && stale_index(data))
         unmap_index(data);
     if (data->index == NULL)



Thanks again for having reported the bug.
Have a nice week-end,

-- 
Julien ÉLIE

« -- Par Thor !
  -- Par Odin !
  -- Par exemple ! » (Astérix)