Cancel distribution network ideas

[Olaf Titz]

The idea of a more efficient cancel distribution network has just been
brought up again, and here are a few thoughts of mine on how this
could be efficiently implemented.

Premise: build a network which distributes cancels between
Internet-connected news sites fast. (This could probably be extended
to other types of controls.) The cancels should be authenticated by an
issuer identification and a public-key signature. They should carry an
indication of the reason (e.g. autodetected spam, manual admin
intervention, etc.) When this gets widely adopted, the current
insecure and inefficient "Control: cancel" protocol would become
mostly (not completely) unnecessary.

It is probably unadvisable to put this function into INN proper. My
cancel channel patch would allow to receive remote cancels, but this
is only one part of the picture - distribution is the other one. I
think distribution is best taken care of by a specialized daemon,
which links to other daemons and sends the cancel commands to INN. I
call this "cancelnet" here. So we have this data flow:

                issuer --> cancelnet --> INN
  (e.g. spam detector)        | ^
                              | |
                              v |
                issuer --> cancelnet --> INN
                              |
                              v
                           cancelnet --> INN

The cancelnet daemons form a network of their own, as they exchange
messages bidirectionally with configured peers, where the peers are
not necessarily related to the news distribution peers. This network
operates in a flood-fill manner, on loop detection see below.

We should be able to use TCP or UDP for message transport, because
with UDP we can use multicast IP where available. The protocol below
allows both.

Messages should be marked with an issuer's identity and carry a
digital signature. One message should contain one or more cancels and
a reason given for the cancel. They should carry a timestamp and a hop
counter.

When the cancelnet daemon receives a message, it immediately sends it
along to all the configured peers, if the configured maximum age and
hop count isn't exceeded. It increments the hop count when sending.
It then checks the signature, and if correct and permitted, acts on
the cancels.

And here is the proposed message format:

When run over TCP, messages are simply sent in the stream (probably
with a maximum length of 64k, because of signature checking). When run
over UDP, each datagram carries one or more complete messages, subject
to a length limit (in the order of 8k?)

Each message is a message head followed by a sequence of elements.

The message head is one byte version magic, one byte hopcount, two
bytes length, four bytes timestamp.

Each element is a type byte, followed by a length byte, followed by
data.

The elements are, in recommended order of appearance:
Type "I": issuer's name. (One per message.)
Type "R": Reason for cancel. (One per message.)
Type "C": Message-ID of cancelled article. (Arbitrary number.)
Type "S": Signature over the whole preceding message, hop count=0;
          must be last element.

This is basically like NoCeM with all the formatting removed, and
without indication of the target newsgroups. (Should we keep the
target newsgroups? This would make the messages considerably longer,
perhaps use a compression algorithm like for DNS?)

I tested the performance of signature generation and verification on
an unloaded K6-3-400 with RSAEURO. With a 512 bit RSA key, it can sign
around 40 messages per second and verify around 440 messages per
second, largely independent of message length. This seems to indicate
that requiring signatures on all those cancel messages is feasible.

An implementation would need the following features:
Networking:
- Receive messages on a UDP port,
- Accept connections from configured peers on a TCP port,
- Send messages on a UDP port to configured peers (including multicast
  addresses?),
- Make TCP connections to configured peers,
- Send messages on existing TCP connections;
Message generation:
- Receive Message-IDs and reason codes from a local process or user,
- format them into messages, sign and send them out;
Signature verification:
- Keep a file of trusted public keys from issuers,
- Verify signatures on each incoming message,
- Decide on a configured policy which issuers to trust.

Message loop detection:
- Each message contains a hop counter which is incremented on the way.
  When it reaches a configured maximum, the message is discarded.
- Messages carry a timestamp, they are discarded when too old.
- It can be assumed that one issuer does not issue more than one
  message with the same timestamp (i.e. more than one per second).
  So the cancelnet program can keep a cache of recent (timestamp,
  issuer) pairs and discard dups. Perhaps the "I" element should
  carry a serial number too.

Obvious disadvantages:
- Someone has to write the code...

Comments?

Olaf