Think I found a bunch of the makehistory problem

[Russ Allbery]

Jeffrey M Vinocur <jeff at litech.org> writes:

> I actually wanted to rebuild everything (newsreading has gotten very
> slow, and I think it's overview's fault) anyway, I'll try that on 1.3 GB
> of text-only gradspool and see how it goes).

You may want to try poking at overview with tdx-util too to see if you can
figure out why it might be so slow.  Just to see what all is in the
current overview files.

> By the way, I think the weird errors in syslog for this (IIRC, we were
> seeing "Can't pack group, no such file or directory") are indicative of
> a larger coding issue.  I haven't looked at the overview code, but in
> history/ we sometimes return NULL/false to indicate an error but don't
> set errno.  However, in the parent code, we log that error and thus get
> whatever garbage happenned to be in errno.

I think the error message we were seeing was this one, though, wasn't it?

    if (pwrite(fd, &gh->indexmem[ge->low - ge->base] , nbytes,
               sizeof(INDEXENTRY)*(ge->low - ge->base + delta)) != nbytes) {
        syslog(L_ERROR, "tradindexed: packgroup cant write to %s: %m", newidx);
        close(fd);
        OV3closegroup(gh, FALSE);
        GROUPlock(gloc, INN_LOCK_UNLOCK);
        return FALSE;
    }

Hm.  I wonder if write is returning a partial write to disk.  (No error
message, but not writing out all of the data.)  That would suck; that
would mean that we'd have to put loops around all those write calls.

*sigh*  The operating system really should satisfy the entirety of a
write() call to a local disk device in one pass.

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>

    Please send questions to the list rather than mailing me directly.
     <http://www.eyrie.org/~eagle/faqs/questions.html> explains why.