Nonblocking I/O and POLL_BUG

Russ Allbery rra at stanford.edu
Mon Oct 25 08:58:52 UTC 1999


I'm working on redoing lib/nonblocking.c so that we can get rid of some
more rat's mazes in configure.in, and in so doing I ran across the fact
that we've been using O_NDELAY rather than O_NONBLOCK as the flag.  From
the comment in my new rework:

/*
**  Only use fcntl if O_NONBLOCK is available; O_NDELAY is *not* the same
**  thing historically.  The semantics of O_NDELAY are that if the read
**  would block, it returns 0 instead.  This is indistinguishable from an
**  end of file condition.  POSIX added O_NONBLOCK, which requires read to
**  return -1 and set errno to EAGAIN, which is what we want.
**
**  FNDELAY (4.3BSD) does the correct thing, but has a different
**  incompatibility (affecting all users of a socket rather than just a
**  file descriptor and returning EWOULDBLOCK instead of EAGAIN) that we
**  don't care about (here).  Using it is *probably* safe, but BSD should
**  also have the ioctl, and at least on Solaris FNDELAY does the same thing
**  as O_NDELAY, not O_NONBLOCK.  So if we don't have O_NONBLOCK, fall back
**  to the ioctl instead.
**
**  Reference:  Sevens, Advanced Unix Programming, pg. 364.
*/

I'm pretty sure this means that we've been getting far more of these than
we should have been:

newsfeed:~/log> grep 'readclose' news.notice | wc -l
    532

because INN has been misinterpreting a blocked read as a remote close in
some circumstances.  While I was in this code, though, I also found I was
getting a lot of "cant read: Resource temporarily unavailable" messages in
my logs, which sounds like EAGAIN on read wasn't being handled correctly.
When I went looking for that, I saw that what I'd consider to be correct
handling of EAGAIN (pretend as if the connection had never selected true
for reading and return to the select loop) was protected by #ifdef
POLL_BUG.  Looks like that also used to be in config.data for Solaris and
dates back quite a while.

I'd like to propose turning that code on permanently.  The only difference
between the behavior with it on and the behavior with it off is that when
off it logs a syslog message when it gets EAGAIN and increments the bad
I/O count on that channel, which means that INN was closing it after 5
rounds of that.

Checking my logs, I don't see nearly enough of those error messages to
worry about INN spinning in a tight loop of EAGAINs, but that's just
Solaris.  Does anyone know if there are any problems with this on other
platforms?

This is the proposed patch:

--- chan.c      1999/10/10 09:39:17     1.39
+++ chan.c      1999/10/25 08:55:34
@@ -635,12 +635,10 @@
     maxbyte = (cp->State != CSgetcmd || bp->Left < BUFSIZ) ? bp->Left : BUFSIZ;
     i = read(cp->fd, &bp->Data[bp->Used], maxbyte-1);
     if (i < 0) {
-#ifdef POLL_BUG
     /* return of -2 indicates EAGAIN, for SUNOS5.4 poll() bug workaround */
         if (errno == EAGAIN) {
             return -2;
         }
-#endif
        oerrno = errno;
        p = CHANname(cp);
        errno = oerrno;

--- nc.c        1999/10/04 02:43:46     1.52
+++ nc.c        1999/10/25 08:56:09
@@ -1137,12 +1137,10 @@
     /* Read any data that's there; ignore errors (retry next time it's our
      * turn) and if we got nothing, then it's EOF so mark it closed. */
     if ((i = CHANreadtext(cp)) <= 0) {
-#ifdef POLL_BUG
         /* return of -2 indicates EAGAIN, for SUNOS5.4 poll() bug workaround */
         if (i == -2) {
             return;
         }
-#endif
        if (i == 0 || cp->BadReads++ >= innconf->badiocount) {
            if (NCcount > 0)
                NCcount--;

I've been running a server with this patch and with the new nonblocking.c
that uses O_NONBLOCK instead of O_NDELAY for about a half-hour now and
haven't seen any problems, but this code used to always be enabled for
Solaris.  I don't have a good way of testing if it messes up any other
platform.

-- 
Russ Allbery (rra at stanford.edu)         <URL:http://www.eyrie.org/~eagle/>


More information about the inn-workers mailing list