BIND 10 #2858: memory manager update processing

[BIND 10 Development]

#2858: memory manager update processing
-------------------------------------+-------------------------------------
            Reporter:  jinmei        |                        Owner:
                Type:  task          |                       Status:  new
            Priority:  medium        |                    Milestone:  New
           Component:  shmem         |  Tasks
  manager                            |                   Resolution:
            Keywords:                |                 CVSS Scoring:
           Sensitive:  0             |              Defect Severity:  N/A
         Sub-Project:  DNS           |  Feature Depending on Ticket:
Estimated Difficulty:  5             |  shared memory data source
         Total Hours:  0             |          Add Hours to Ticket:  0
                                     |                    Internal?:  0
-------------------------------------+-------------------------------------
Description changed by jinmei:

Old description:

> Subtask of #2830. Depend on #2857.
>
> In this task, implement the memmgr side of
> http://bind10.isc.org/wiki/SharedMemoryIPC#a5.2.UpdateaSegment
> and
> http://bind10.isc.org/wiki/SharedMemoryIPC#a5.3.SwitchSegments
>
> Deal with the case where a reader dies at any timing.

New description:

 (Revised, based on more details of resulting #2854, although the
 details may even change more after review, especially about how we
 handle full data source reconfiguration).

 Subtask of #2830. Depend on #2857 and #2931.

 In this (revised) task, we implement membership management for the
 `MemoryReaders` group and exchanges between the memmgr and readers
 (whether it's initial phase or subsequent updates doesn't matter).

 As described in http://bind10.isc.org/wiki/SharedMemoryIPC, we'll use
 "segment_info_update" and "segment_info_update_ack" messages
 (commands).  Commonly used parameters of these messages are:
 - generation-id (int): the generation ID of the corresponding
   configuration.  for now (until we support whole datasource
   reconfiguration), it's just a placeholder, would always be set to 0
   by the sender and ignored by receivers.
 - data-source-name (str): the data source name
 - data-source-class (str): textual representation of the RR class of
   the data source, e.g, 'IN'

 In addition, the "segment_info_update_ack" message has an additional
 parameter:
 - segment-params (dict): segment-type specific parameters.  only
   memmgr and `ZoneTableSegment` implementations know its meaning;
   readers will handle it as opaque data.

 The 'loadzone' command from other modules such as xfrin would take the
 following parameter:
 - generation-id (int): same as that for segment_info_update for now.
 - data-source-name (str): the data source that stores the updated zone
 - class (str): the RR class of the zone to be loaded (should be
   consistent with the data source)
 - origin (str): the origin name of the zone to be loaded

 Then update `Memmgr` class as follows:
 - update `_setup_module()` to register notification for the
 `MemoryReaders`
   group (this needs #2931), and then add pre-existing readers to all
   `SegmentInfo` objects (this should be done after add_remote_config).
 - complete the `__handle_builder_result()` (first introduced in #2857)
 method:
 {{{#!python
             elif len(sgmt_info.old_readers) > 0:
                 for reader in sgmt_info.old_readers:
                     # send segment_info_update command to the reader
                     # with param of sgmt_info.get_reset_param(READER)
 }}}
 - introduce `_mod_command_handler()` for the 'loadzone' command from
   other modules, and 'segment_info_update_ack' message (command)
   from readers.  the former case would look like:
 {{{#!python
     def _mod_command_handler(self, cmd, args):
         if cmd == 'loadzone':
             # extract gen_id, dsrc_name, rrclass, zone_name from 'args'
             for dsrc_info in self.__datasrc_info_list:
                 if dsrc_info.gen_id == gen_id:
                     break
             cmd = ('load', zone_name, dsrc_info, rrclass, dsrc_name)
             sgmt_info = dsrc_info.segment_info_map[(rrclass, dsrc_name)]
             sgmt_info.add_event(cmd)
             cmd = sgmt_info.start_update()
             if cmd is not None:
                 # send it to the builder
 }}}
   and the latter would look like:
 {{{#!python
         elif cmd == 'segment_info_update_ack':
             # identify the SegmentInfo, then move the reader to the
 'current'
             # set
             cmd = sgmt_info.sync_raeder(reader_id) # it will update state
             if cmd is not None:
                 # send it to the builder thread
 }}}
 - add `__handle_update_ack()` method.  this would be called from
   `_mod_command_handler()` to handle 'segment_info_update_ack':
 {{{#!python
     def __handle_update_ack(self):
         # identify the SegmentInfo, then move the reader to the 'current'
 set
         cmd = sgmt_info.sync_raeder(reader_id) # it will update state
         if cmd is not None:
             # send it to the builder
 }}}
 - add `__handle_reader_notify()` method: this is the callback for
   group membership notifications (see above):
 {{{#!python
     def __handle_reader_notify(self, params):
         if params[0] == 'subscribed':
             reader_id = params[1]['client']
             for sgmt_info in \
                     self._datasrc_info_list[0].segment_info_map.values():
                 sgmt_info.add_reader(reader_id)
                 param = sgmt_info.get_reset_param(READER)
                 if param is not None: # this can be None in init stage
                     # send segment_info_update command to the reader
                     # with 'param' (and data source name, etc)
         elif params[0] == 'unsubscribed':
             reader_id = params[1]['client']
             for sgmt_info in \
                     self._datasrc_info_list[0].segment_info_map.values():
                 cmd = sgmt_info.remove_reader(reader_id)
                 if cmd is not None: # this triggers SYNCHRONIZING=>COPYING
                     # send it to the builder
 }}}

--

-- 
Ticket URL: <http://bind10.isc.org/ticket/2858#comment:1>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development