[bind10-dev] OpenReg's communications framework

[Michael Graff]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

All wire format data is encapsulated into "BIND 9 wire format (with
modifications.)"  This is a simple tag/type/value format where types can
be nested.

Here are the words from "wire-format.txt" in OpenReg:

**********

Wire format for messages.

The general format:

message = {
	VERSION
	HASH
}

VERSION = 0x536b616e;

For item types, the lower 4 bits describe the type of item, while the
upper 4 bits describe the length prefix used:

	ITEM_DATA = 0x01
	ITEM_HASH = 0x02
	ITEM_LIST = 0x03
	ITEM_NULL = 0x04

	LENGTH_32 = 0x00
	LENGTH_16 = 0x10
	LENGTH_8  = 0x20

An "item" is either a literal string of bytes, a NULL, a HASH, or a LIST.

NULL = ITEM_NULL

NULL values are encoded without a data length or any data.

HASH values are encoded with a type of ITEM_HASH, followed by a length
of the hash itself.  The one exception to this is the top level hash,
which has no item type or length encoded.

HASH = ITEM_HASH length *(tag-len-byte tag-name item-type item-len item)

LIST values are encoded with a type of ITEM_LIST, followed by a length
of the list.

LIST = ITEM_LIST length *(item-type item-len item)

For example, the following Perl structure would be encoded on the wire
like this, assuming all length bytes are 1 byte long:

$msg = {
  foo => "bar",
  baz => { abc => 123 },
  arr => [ "1", "a", [ "5", "t" ] ],
  test => "yes"
}

0x536b6e61			VERSION

				{
3 "foo" 0x21 3 "bar"		  foo => "bar",
3 "baz"	0x22 9			  baz => {
	3 "abc" 0x21 3 "123"	    abc => 123,
				  },
3 "arr" 0x23  12		  arr => [
	0x21 1 "1"		   "1",
	0x21 1 "a"		   "a",
	0x23 6			   [
		0x21 1 "5"	    "5",
		0x21 1 "t"	    "t"
				   ]
				  ],
4 "test" 0x21 3 "yes"		  test => "yes",
				}



*******

There is a "msgq" daemon that implements the above wire format called
"MsgBus."

Each client connects to a well-known TCP port on localhost.  Upon
connection, the client sends a request for a client identifier (aka a
"local name.")  The server then creates a unique local name for this
client and notifies the client of this name.

Each message has multiple header fields:  to, from, group, instance,
type, and subtype.  These are filled in by the client and are used both
for message routing as well as filtering on the receivers.  The data
field is also in the message header, but is opaque and is otherwise
untouched by the routing.

The "to" field is either a specific name or is the wildcard.  It is of
the format localname at hostname, where localname is unique at any given
point and should be unique over time.  I believe the format used by
OpenReg was a pointer location in hex followed by a counter.

The "from" field is the local name of the sender, and is enforced by the
message daemon to be such.  It must be included just to make it so the
message daemon need not copy data to add it.

For example, statistics may be periodically sent to the group/instance
of "stats/authority-servers" for authority daemons.  Receivers would
subscribe to "stats/*" to receive all statistic updates, or to
"stats/authority-servers" to only receive the ones specific to them.

The message type is used to indicate what this message is:  a group
subscription, a group unsubscribe, a PING, or a message.

The "subtype" is not actually part of the message "type" but means
"subscription type."  The values supported are:

  normal:  All messages on this group to any wildcard or to me.
  promisc:  All messages on this group.
  meonly:  Messages addresses specifically to me.

Subscription requests may be:

  */*/meonly <--- messages only to me no matter where.
  foo/* <--- all messages on channel "foo"
  foo/bar <--- all messages on channel "foo" with instance == "bar"

On the last two, the modifiers descried above were accepted.  On the
global wildard of */*/meonly, only "meonly" was supported.  However,
*/*/promisc was also supported to enable snooping.

For simplicity in the message daemon, subscribing to foo/* and foo/bar
would cause two messages to be received, one to the foo/* callback and
one to the foo/bar callback.  It was up to the application to filter these.

In messages of type "send" -- which is just a normal transmission, the
data portion of the message is opaque to the message daemon.

- --Michael

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpbpisACgkQ+NNi0s9NRJ3pkwCfZDYIHtoqdv+GLlbCLE71bSRt
3msAoJ/HG0AWqMymbMDyRQnALpjYGb5F
=fIfF
-----END PGP SIGNATURE-----