INN commit: trunk (doc/man/libstorage.3 storage/tradindexed/tdx-data.c)

[INN Commit]

    Date: Thursday, May 7, 2009 @ 01:30:03
  Author: jeff
Revision: 8451

Modify behavior of tradindexed (with OVSTATICSEARCH) to fix corrupted
XOVER responses caused by the mid-search attempt to unmap and remap
overview data in the hopes of making additional data accessible.

Clarify documentation on behavior of OVSTATICSEARCH.

Modified:
  trunk/doc/man/libstorage.3
  trunk/storage/tradindexed/tdx-data.c

--------------------------------+
 doc/man/libstorage.3           |    8 ++++++++
 storage/tradindexed/tdx-data.c |   17 +++++++----------
 2 files changed, 15 insertions(+), 10 deletions(-)

Modified: doc/man/libstorage.3
===================================================================
--- doc/man/libstorage.3	2009-05-06 18:09:26 UTC (rev 8450)
+++ doc/man/libstorage.3	2009-05-07 08:30:03 UTC (rev 8451)
@@ -327,6 +327,9 @@
 OVSPACE	probe overview space usage
 OVSTATALL	stat all articles when
 	OVexpiregroup is called
+OVSTATICSEARCH	if results of OVsearch are stored
+	in a static buffer and must be copied
+	before the next call to OVsearch
 .fi
 .in -0.5i
 .sp 1
@@ -352,6 +355,11 @@
 .I low
 and
 .IR high .
+The setting of OVSTATICSEARCH determines how search result data must be
+handled.  (Note that with some storage methods, each call to OVopensearch
+may cause internal storage to be remapped.  Therefore multiple
+simultaneous searches may require data to be copied in between OVsearch
+calls even if OVSTATICSEARCH is false.)
 .PP
 .I OVsearch
 retrieves information; article number, overview data, or arrival time.

Modified: storage/tradindexed/tdx-data.c
===================================================================
--- storage/tradindexed/tdx-data.c	2009-05-06 18:09:26 UTC (rev 8450)
+++ storage/tradindexed/tdx-data.c	2009-05-07 08:30:03 UTC (rev 8451)
@@ -462,6 +462,7 @@
         unmap_index(data);
         map_index(data);
         data->high = high;
+	unmap_data(data);
     }
     if (start > data->high)
         return NULL;
@@ -512,17 +513,13 @@
     if (search->current > search->limit || search->current > max)
         return false;
 
-    /* Make sure that the offset into the data file is sensible, and try
-       remapping the data file if the portion the offset is pointing to isn't
-       currently mapped.  Otherwise, warn about possible corruption and return
-       a miss. */
+    /* There is a small chance that remapping the data file could make this
+       offset accessible, but changing the memory location in the middle of
+       a search conflicts with our OVSTATICSEARCH.  And it seems not to
+       be an issue in limited testing, although write caching that leads to 
+       on-disk IDX and DAT being out of sync could trigger a problem here. */
     if (entry->offset + entry->length > search->data->datalen) {
-        unmap_data(search->data);
-        if (!map_data(search->data))
-            return false;
-    }
-    if (entry->offset + entry->length > search->data->datalen) {
-        warn("Invalid entry for article %lu in %s.IDX: offset %lu length %lu",
+        warn("Invalid or inaccessible entry for article %lu in %s.IDX: offset %lu length %lu",
              search->current + search->data->base, search->data->path,
              (unsigned long) entry->offset, (unsigned long) entry->length);
         return false;