BIND 10 #853: boost::mpi (or MPI in general?) as a possible replacement of MSGQ

[BIND 10 Development]

#853: boost::mpi (or MPI in general?) as a possible replacement of MSGQ
-------------------------------------+-------------------------------------
                   Reporter:  shane  |                 Owner:  shane
                       Type:  task   |                Status:  accepted
                   Priority:         |             Milestone:
  critical                           |  Sprint-20110802
                  Component:  msgq   |            Resolution:
                   Keywords:         |             Sensitive:  0
            Defect Severity:  N/A    |           Sub-Project:  Core
Feature Depending on Ticket:         |  Estimated Difficulty:  0.0
        Add Hours to Ticket:  0      |           Total Hours:  0
                  Internal?:  0      |
-------------------------------------+-------------------------------------

Comment (by shane):

 MPI is a quite mature technology, from the 1990 originally. It is in heavy
 use by academia and governments. There are many implementations, and
 standards work continues to this day (although in a MPI-specific standards
 body).

 Wikipedia has a longish article about it:

 http://en.wikipedia.org/wiki/Message_Passing_Interface

 To run it, we would need a daemon or set of daemons. There are many
 implementations so we would need to choose one.

 We would also need to choose a library to use in our applications. It
 might make sense to use the boost::mpi version, since we are already using
 Boost. However, we could not use the header-only mode for C++, so this is
 not a "slam dunk" option. Also, Boost has certain preferred daemon
 implementations:

   * [http://www.open-mpi.org/ Open MPI 1.0.x]
   * [http://www.lam-mpi.org/ LAM/MPI 7.x]
   * [http://www-unix.mcs.anl.gov/mpi/mpich/ MPICH 1.2.x]

 Taken from:
 http://www.boost.org/doc/libs/1_37_0/doc/html/mpi/getting_started.html#mpi.mpi_impl

 I'll go through the points we came up with, then have discussion at the
 end.

   * Nice: Cross-machine

     MPI is explicitly designed to work across a network. It is also
 designed to be high-performance, so often uses shared memory for local
 communication, and specialized network protocols for working with remote
 machines.

   * We have to be able to secure if it uses other computers.

     This seems trickier. Some implementations can use SSH as the
 underlying mechanism, although MPI is designed more around running jobs.
 Think "batch processing".

   * Compatible license (not GPL)

     BSD licensed versions of C++ and Python libraries exist. We may be
 able to use the boost::mpi for both C++ and Python, however we will not be
 able to use C++ in header-only mode. BSD licensed versions of the "actual"
 MPI install also exist.

     * Nice: send file descriptors

     This does ''not'' appear to be supported.

     * Nice: minimize dependencies

     Depending on which implementation we choose, this can be true. If we
 use boost::mpi then we have only added another use of Boost for the
 libraries. We'll need the MPI install too.

     * Nice: Client-side in Python and C++

     Yes.

     * Nice: Not have to declare types before sending

     Types can be built at runtime (according to the Wikipedia article).
 I'm not sure how easy this is in practice, especially in C++, but it is
 possible.

     * Nice: high-performance

     Used in supercomputers around the world.

     * Point-to-point and group communication (including no listeners)

     Both are nicely supported.

     * Separate instances (to run multiple BIND 10 in parallel)

     Yes.

     * Multiple OS support

     Yes.

     * Nice: unit-tests for the code itself

     Not sure... the boost::mpi presumably has the same testing as the rest
 of Boost. Some other libraries definitely do. The non-library part of MPI
 will have varying levels of testing, depending on the implementation.
 There are many!!!

     * Nice: mock test support

     I did not see any.

 I am pretty sure that MPI could replace msgq for us. We'd get increased
 performance and fewer bugs. However, MPI is not really a match
 ''conceptually'' for what we want to do. It is designed for batch
 processing, especially in the context of large computing efforts. We would
 need to spend an amount of time getting and testing our implementations
 together, and providing a way to install & configure these so the user
 does not have to learn the details.

 So... I think MPI is a possibility, but probably not a strong one.

-- 
Ticket URL: <http://bind10.isc.org/ticket/853#comment:5>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development