[bind10-dev] python script to convert doxygen to pydoc

[JINMEI Tatuya / 神明達哉]

I've pushed a preliminary personal branch to share the python script
for C++ doxygen->pydoc conversion I mentioned today's meeting.  It's
on branch trac933preview (I created a ticket for this work, #933).

The script is located in src/lib/util/python and named
doxygen2pydoc.py (generated by ./configure from .in).

This is a quick how-to-use-it in case someone is interested in playing
with it:
1. you'll first need to generate XML (not HTML) output files from the
   doxygen comments.  To do this:
   - go to (bind10/)doc
   - run doxygen with Doxyfile-xml as the configuration file:
     % doxygen Doxyfile-xml
2. identify the XML file for the C++ doc you want to convert.  XML
   files are generated under doc/html/xml/.  Just for an arbitrary
   example, the XML file for the isc::dns::EDNS class documentation is
   "classisc_1_1dns_1_1_e_d_n_s.xml".
3. run the convert script with the XML file as the command line
   argument.  The corresponding C++ string definitions will be dumped
   to stdout.  I've copied the output for the EDNS class below.
4. the script file tries to do some trivial method name conversion
   using heuristics, e.g., getUDPSize() -> get_udp_size(), but you'll
   normally need to edit the output further so that the documentation
   strings really match the python behavior.
5. copy-paste or include the resulting doc string definitions to/from
   the wrapper code.

The script is mostly complete feature-wise.  When we include #933 in a
sprint I'll plan to finish it with cleanups and documentation.

 ---
 JINMEI, Tatuya

namespace {
#ifdef COPY_THIS_TO_MAIN_CC
    { "get_version", reinterpret_cast<PyCFunction>(EDNS_getVersion), METH_NOARGS,
      EDNS_getVersion_doc },
    { "get_udp_size", reinterpret_cast<PyCFunction>(EDNS_getUDPSize), METH_NOARGS,
      EDNS_getUDPSize_doc },
    { "set_udp_size", reinterpret_cast<PyCFunction>(EDNS_setUDPSize), METH_VARARGS,
      EDNS_setUDPSize_doc },
    { "get_dnssec_awareness", reinterpret_cast<PyCFunction>(EDNS_getDNSSECAwareness), METH_NOARGS,
      EDNS_getDNSSECAwareness_doc },
    { "set_dnssec_awareness", reinterpret_cast<PyCFunction>(EDNS_setDNSSECAwareness), METH_VARARGS,
      EDNS_setDNSSECAwareness_doc },
    { "to_wire", reinterpret_cast<PyCFunction>(EDNS_toWire), METH_VARARGS,
      EDNS_toWire_doc },
    { "to_wire", reinterpret_cast<PyCFunction>(EDNS_toWire), METH_NOARGS,
      EDNS_toWire_doc },
    { "to_text", reinterpret_cast<PyCFunction>(EDNS_toText), METH_NOARGS,
      EDNS_toText_doc },
#endif // COPY_THIS_TO_MAIN_CC

const char* const EDNS_doc = "\
The EDNS class represents the EDNS OPT RR defined in RFC2671.\n\
\n\
This class encapsulates various optional features of EDNS such as the\n\
UDP payload size or the DNSSEC DO bit, and provides interfaces to\n\
manage these features. It is also responsible for conversion to and\n\
from wire-format OPT RR. One important exception is about the extended\n\
RCODE: The EDNS class is only responsible for extracting the 8-bit\n\
part of the 12-bit extended RCODE from the OPT RR's TTL field of an\n\
incoming message, and for setting the 8-bit part into the OPT RR TTL\n\
of an outgoing message. It's not supposed to know how to construct the\n\
complete RCODE, much less maintain the RCODE in it. It is the caller's\n\
responsibility (typically the Message class).\n\
\n\
When converting wire-format OPT RR into an EDNS object, it normalizes\n\
the information, i.e., unknown flags will be ignored on construction.\n\
\n\
This class is also supposed to support EDNS options such as NSID, but\n\
the initial implementation does not include it. This is a near term\n\
TODO item.\n\
\n\
Notes to developers\n\
\n\
The rest of the description is for developers who need to or want to\n\
understand the design of this API.\n\
\n\
Representing EDNS is tricky. An OPT RR is no different from other RRs\n\
in terms of the wire format syntax, and in that sense we could use the\n\
generic RRset class to represent an OPT RR (BIND 9 adopts this\n\
approach). But the resulting interface would be inconvenient for\n\
developers. For example, the developer would need to know that the UDP\n\
size is encoded in the RR Class field. It's better to provide a more\n\
abstract interface along with the special semantics of OPT RR.\n\
\n\
There is still a non trivial point: How to manage extended RCODEs. An\n\
OPT RR encodes the upper 8 bits of extended 12-bit RCODE. In general,\n\
it would be better to provide a unified interface to get access to\n\
RCODEs whether or not they are traditional 4 bit codes or extended\n\
ones that have non 0 upper bits. However, since an OPT RR may not\n\
appear in a message the RCODE cannot be maintained in the EDNS class.\n\
But it would not be desirable to maintain the extended RCODEs\n\
completely in the Message class, either, because we wanted to hide the\n\
mapping between EDNS semantics and its wire format representation\n\
within the EDNS class; if we moved the responsibility about RCODEs to\n\
the Message class, it would have to parse and render the upper 8 bits\n\
of the RCODEs, dealing with wire representation of OPT RR. This is\n\
suboptimal in the sense of encapsulation.\n\
\n\
As a compromise, our decision is to separate the knowledge about the\n\
relationship with RCODE from the knowledge about the wire format as\n\
noted in the beginning of this description.\n\
\n\
This decoupling is based on the observation that the extended RCODE is\n\
a very special case where EDNS only has partial information. If a\n\
future version of the EDNS protocol introduces further relationship\n\
between the message and the EDNS, we might reconsider the interface,\n\
probably with higher abstraction.\n\
\n\
EDNS(version=SUPPORTED_VERSION)\n\
\n\
    Constructor with the EDNS version.\n\
\n\
    An application would use this constructor to specify EDNS\n\
    parameters and/or options for outgoing DNS messages.\n\
\n\
    All other parameters than the version number will be initialized\n\
    to reasonable defaults. Specifically, the UDP payload size is set\n\
    to Message.DEFAULT_MAX_UDPSIZE, and DNSSEC is assumed to be not\n\
    supported. These parameters can be altered via setter methods of\n\
    this class. Note, however, that the version number cannot be\n\
    changed once constructed.\n\
\n\
    The version number parameter can be omitted, in which case the\n\
    highest supported version in this implementation will be assumed.\n\
    When specified, if it is larger than the highest supported\n\
    version, an exception of class isc.InvalidParameter will be\n\
    thrown.\n\
\n\
    This constructor throws no other exception.\n\
\n\
    Parameters:\n\
      version    The version number of the EDNS to be constructed.\n\
\n\
EDNS(name, rrclass, rrtype, ttl, rdata)\n\
\n\
    Constructor from resource record (RR) parameters.\n\
\n\
    This constructor is intended to be used to construct an EDNS\n\
    object from an OPT RR contained in an incoming DNS message.\n\
\n\
    Unlike many other constructors for this purpose, this constructor\n\
    does not take the bare wire-format data in the form of an\n\
    InputBuffer object. This is because parsing incoming EDNS is\n\
    highly context dependent and it's not feasible to handle it in a\n\
    completely polymorphic way. For example, a DNS message parser\n\
    would have to check an OPT RR appears at most once in the message,\n\
    and if it appears it should be in the additional section. So, the\n\
    parser needs to have an explicit check to see if an RR is of type\n\
    OPT, and then (if other conditions are met) construct a\n\
    corresponding EDNS object. At that point the parser would have\n\
    already converted the wire data into corresponding objects of\n\
    Name, RRClass, RRType, etc, and it makes more sense to pass them\n\
    directly to the constructor.\n\
\n\
    In practice, top level applications rarely need to use this\n\
    constructor directly. It should normally suffice to have a higher\n\
    level class such as Message do that job.\n\
\n\
    This constructor checks the passed parameters to see if they are\n\
    valid in terms of the EDNS protocol specification. name must be\n\
    the root name (\".\"); otherwise, an exception of class\n\
    DNSMessageFORMERR will be thrown. rrtype must specify the OPT RR\n\
    type; otherwise, an exception of class isc.InvalidParameter will\n\
    be thrown. The ENDS version number is extracted from rrttl. If it\n\
    is larger than the higher supported version, an exception of class\n\
    DNSMessageBADVERS will be thrown. Note that this is different from\n\
    the case of the same error in the other constructor. This is\n\
    intentional, so that the application can transparently convert the\n\
    exception to a response RCODE according to the protocol\n\
    specification.\n\
\n\
    This initial implementation does not support EDNS options at all,\n\
    and rdata is simply ignored. Future versions will support options,\n\
    and may throw exceptions while validating the given parameter.\n\
\n\
    Note: since no other type than OPT for rrtype is allowed, this\n\
    parameter could actually have been omitted. But it is\n\
    intentionally included as a parameter so that invalid usage of the\n\
    construction can be detected. As noted above the caller should\n\
    normally have the corresponding RRType object at the time of call\n\
    to this constructor, so the overhead of having the additional\n\
    parameter should be marginal.\n\
\n\
    Parameters:\n\
      name       The owner name of the OPT RR. This must be the root\n\
                 name.\n\
      rrclass    The RR class of the OPT RR.\n\
      rrtype     This must specify the OPT RR type.\n\
      ttl        The TTL of the OPT RR.\n\
      rdata      The RDATA of the OPT RR.\n\
\n\
Class constant data:\n\
SUPPORTED_VERSION =  0\n\
  The highest EDNS version this implementation supports.\n\
\n\
";

const char* const EDNS_getVersion_doc = "\
get_version() -> integer\n\
\n\
Returns the version of EDNS.\n\
\n\
This method never throws an exception.\n\
\n\
";

const char* const EDNS_getUDPSize_doc = "\
get_udp_size() -> integer\n\
\n\
Returns the maximum payload size of UDP messages for the sender of the\n\
message containing this EDNS.\n\
\n\
This method never throws an exception.\n\
\n\
";

const char* const EDNS_setUDPSize_doc = "\
set_udp_size(udp_size) -> void\n\
\n\
Specify the maximum payload size of UDP messages that use this EDNS.\n\
\n\
Unless explicitly specified, DEFAULT_MAX_UDPSIZE will be assumed for\n\
the maximum payload size, regardless of whether EDNS OPT RR is\n\
included or not. This means if an application wants to send a message\n\
with an EDNS OPT RR for specifying a larger UDP size, it must\n\
explicitly specify the value using this method.\n\
\n\
This method never throws an exception.\n\
\n\
Parameters:\n\
  udp_size   The maximum payload size of UDP messages for the sender\n\
             of the message containing this EDNS.\n\
\n\
";

const char* const EDNS_getDNSSECAwareness_doc = "\
get_dnssec_awareness() -> bool\n\
\n\
Returns whether the message sender is DNSSEC aware.\n\
\n\
This method never throws an exception.\n\
\n\
true if DNSSEC is supported; otherwise false.\n\
\n\
";

const char* const EDNS_setDNSSECAwareness_doc = "\
set_dnssec_awareness(is_aware) -> void\n\
\n\
Specifies whether the sender of the message containing this EDNS is\n\
DNSSEC aware.\n\
\n\
If the parameter is true, a subsequent call to to_wire() will set the\n\
DNSSEC DO bit on for the corresponding OPT RR.\n\
\n\
This method never throws an exception.\n\
\n\
Parameters:\n\
  is_aware   true if DNSSEC is supported; false otherwise.\n\
\n\
";

const char* const EDNS_toWire_doc = "\
to_wire(renderer, extended_rcode) -> unsigned int\n\
\n\
Render the EDNS in the wire format.\n\
\n\
This method renders the EDNS object as a form of DNS OPT RR via\n\
renderer, which encapsulates output buffer and other rendering\n\
contexts. Since the EDNS object does not maintain the extended RCODE\n\
information, a separate parameter extended_rcode must be passed to\n\
this method.\n\
\n\
If by adding the OPT RR the message size would exceed the limit\n\
maintained in renderer, this method skips rendering the RR and returns\n\
0; otherwise it returns 1, which is the number of RR rendered.\n\
\n\
In the current implementation the return value is either 0 or 1, but\n\
the return type is unsigned int to be consistent with RRset.to_wire().\n\
In any case the caller shouldn't assume these are only possible return\n\
values from this method.\n\
\n\
This method is mostly exception free, but it requires memory\n\
allocation and if it fails a corresponding standard exception will be\n\
thrown.\n\
\n\
In practice, top level applications rarely need to use this method\n\
directly. It should normally suffice to have a higher level class such\n\
as Message do that job.\n\
\n\
Note to developer: the current implementation constructs an RRset\n\
object for the OPT RR and calls its to_wire() method, which is\n\
inefficient. In future, we may want to optimize this method by caching\n\
the rendered image and having the application reuse the same EDNS\n\
object when possible.\n\
\n\
Parameters:\n\
  renderer   DNS message rendering context that encapsulates the\n\
             output buffer and name compression information.\n\
  extended_rcode Upper 8 bits of extended RCODE to be rendered as part\n\
             of the EDNS OPT RR.\n\
\n\
Return Value(s): 1 if the OPT RR fits in the message size limit;\n\
otherwise 0.\n\
";

const char* const EDNS_toWire_doc = "\
to_wire(buffer, extended_rcode) -> unsigned int\n\
\n\
Render the EDNS in the wire format.\n\
\n\
This method is same as to_wire(message_renderer&,uint8_t)const except\n\
it renders the OPT RR in an OutputBuffer and therefore does not care\n\
about message size limit. As a consequence it always returns 1.\n\
\n\
";

const char* const EDNS_toText_doc = "\
to_text() -> string\n\
\n\
Convert the EDNS to a string.\n\
\n\
The string will be terminated with a trailing newline character.\n\
\n\
When EDNS options are supported the output of this method will be\n\
extended.\n\
\n\
This method is mostly exception free, but it may require memory\n\
allocation and if it fails a corresponding standard exception will be\n\
thrown.\n\
\n\
A string representation of EDNS. See above for the format.\n\
\n\
";
} // unnamed namespace