[bind10-dev] milestones for shared memory support

[JINMEI Tatuya / 神明達哉]

Mukund asked me to make milestones toward the goal of shared memory
(mmap) based in-memory data source.  I've examined the current list of
tasks, considered relationship and dependency between them, identified
missing tasks, and then come up with a proposed plan.  This plan
expects we'll complete the basic feature in 4 sprints, and we can add
a few more attractive features in two more sprints (but as usual, it's
quite likely I'm too optimistic).

First, some notes:
- I believe we'll need to fully focus on this feature for the moment;
  i.e., we won't be able to afford time for any distraction such as
  resolver research.  that's a pity, but I think that's the reality we
  need to face especially for the coming several months (as our
  development resource is very limited and new member(s) will need
  some learning time).
- So I basically assumed we can complete about 50-ish estimation
  points each sprint, and we can allow around a half of the points for
  the shared memory tasks.
- I think we need to complete full datasrc reconfig as part of basic
  feature, as it's not so uncommon operation even though it's
  generally not expected to take place often.
- And, personally, I believe we need to make it something attractive,
  not just to make it okay level by not requiring memory copy for
  multi-core environment.  In that sense I also believe it's mostly
  mandatory to support faster startup with handling zone diffs.  note
  that we'll need to be able to handle zone diffs independently from
  the shared memory work, because it's necessary to avoid full
  reloading after ixfr-in or DDNS.

And, the actual plan follows.  I tried to make it so we can add some
user visible result in each sprint (but that's not always the case).
The number before the ticket ID is the current estimation point
(marked with '?' if not yet estimated).  These don't include tasks
already in the current sprint (but we'll probably need to work on some
of them in the next sprint, so it's already a bit too optimistic).

1st sprint: 1st preparation work.  this is mostly invisible to users,
but with #2851 we should be able to fix one known issue as a side
effect: being able to load zone from a file after a failure due to
load error (syntax error in zone file, etc).
? #2851 extensions to ConfigurableClientList::getCachedZoneWriter
3 #2852 Add an API for (re)setting a memory segment to ConfigurableClientList
5 #2853 Python wrapper of data source extensions
4 #2836 update in-memory zone loading so it can work with shmem segment
? #2916 asio extension: "local" derived class of asiolink::IOSocket
? #2922 msgq extension: subscriber info handling

2nd sprint: 2nd preparation work. no user visible result.
? #2906 define and implement (generic) ZoneTable class
? #2907 add ConfigurableClientList::getZoneTable class
? #2908 Python wrapper for ZoneTable and getZoneTable
? #2923 IPC user extension: getting subscriber info, way to unicast specific
  subscriber (identified by the session ID)
9 #2854 memory manager framework

3rd sprint: complete basic feature for b10-auth.  still not fully
workable without memmgr, but at this point we should be able to play
with the feature using some quick hack script mocking the mgr
behavior.
4 #2861 synchronization between auth main thread and datasrc builder
4 #2855 introduce a "zone builder" thread in memory manager
4 #2856 memory manager initialization
5 #2862 update b10-auth to recognize data source memory segments
4 #2863 handle "segment_info_update" message in b10-auth
? #2918 support config generation ID

4th sprint: complete the shared memory support with data source reconfig
5 #2857 memory manager initial exchanges
5 #2858 memory manager update processing
3/5 #2864 update xfrin/DDNS
? #2919 handle full data source reconfig in memmgr
? #2920 handle full data source reconfig with shmem support in b10-auth

5th sprint: make faster startup possible
? #2912 make in-memory zone data loader a class
? #2913 update CacheConfig and ZoneWriter using class version of ZoneDataLoader
? #2914 extend ZoneDataLoader so it can use a journal reader
? #2915 update CacheConfig to support diff mode
7 #2750 support DomainTree::delete()

6th sprint: complete diff support and cleanup
5 #2751 add "subtract RDATA" interface to RdataSet
6 #2752 define and implement ZoneDataUpdater::delete()
11 #2867 (placeholder for cleanup)

---
JINMEI, Tatuya