Proposed patch: cancel channel

[Olaf Titz]

> > hmmm can't we just open socket and send X cancel commands directly to
> > innd, by avoiding ctlinnd ? should I RTFS ?
>
> Hm... yeah, it looks like the innd side would support that.  Normally
> ctlinnd creates a Unix domain socket to receive responses and then tears
> it down afterwards, which is a lot of overhead, but it doesn't look you
> have to do that.

You could in theory create one Unix domain datagram socket and send
several commands over it, then expect the responses all on this socket
(just not with the inndcomm library, which always uses a new one).
However, because this is a datagram socket, you have to wait for
timeout, and you have to do this for _every_ control message you send
or responses will get lost (overflowing queue).

The other, more serious concern is with the way innd handles I/O
channels: you can only receive one command on the datagram command
socket per run through the main loop. I suspect this could become much
more of a limiting factor when there are many peers. With a stream
socket (AF_UNIX or TCP alike) you simply get what's in the buffer
during one run, which can be several commands. Note that the responses
are mostly handled by the TCP internal buffering and then are not
subject to the I/O loop limit.

I've had to deal with a severely congested innd before, it's not funny
(waiting for commands, as well as new connections, on the order of
_minutes_; I've even added code to prioritze the channels and recheck
the localconn, remconn, control channels after handling them, but that
wasn't enough.)

That said, I'm still not sure if the cancel channel protocol is
designed right, and this is the reason I didn't push it more. :-)
The current application does not need responses at all - for NoCeM,
and for the proposed cancel-distribution network, it doesn't matter if
a cancel was sucessful, proactive, lost, or failed. (Other than for
statistics.) Especially with regard to the I/O loop and buffering
concerns it would be nice if the client would just send cancels and
never look for a response. (My current client doesn't do that; it
could just throw the responses away instead of waiting for them.)

So there is the question: do we need the responses or would it be
better to just skip them? The code change in innd would be trivial of
course.

Olaf