BIND 10 #2836: update in-memory zone loading so it can work with shmem segment

[BIND 10 Development]

#2836: update in-memory zone loading so it can work with shmem segment
-------------------------------------+-------------------------------------
            Reporter:  jinmei        |                        Owner:
                Type:  task          |  vorner
            Priority:  medium        |                       Status:
           Component:  data source   |  reviewing
            Keywords:                |                    Milestone:
           Sensitive:  0             |  Sprint-20130514
         Sub-Project:  DNS           |                   Resolution:
Estimated Difficulty:  4             |                 CVSS Scoring:
         Total Hours:  0             |              Defect Severity:  N/A
                                     |  Feature Depending on Ticket:
                                     |  shared memory data source
                                     |          Add Hours to Ticket:  0
                                     |                    Internal?:  0
-------------------------------------+-------------------------------------

Comment (by jinmei):

 As you mentioned a crash on the daily call, I've looked into the
 latest trac2836 branch (at 2f220f9).

 I'm not sure what exactly happened to you, but it caused a crash
 in my environment, too.  In my case, address reallocation (mapped on a
 different address) never happened until growing the segment finally
 fails, at which point the `MemorySegmentMapped` object breaks its
 integrity (already reset the underlying boost segment).  So the
 subsequent cleanup attempt triggered NULL pointer dereference.

 That's certainly bad, but I also think the
 `SegmentObjectHolderTest.grow` test is very dangerous.  Like in my
 case, there's no guarantee that relocation happens by repeating grow()
 a reasonable number of times, so it could end up creating hundreds of
 GBs of segment (in my case it reached ENOSPC at around 70GB).  I'm
 afraid there's no easy and portable/reliable way to cause this
 situation, but from some quick experiments I found that if I open
 another mapped segment at the same time reallocation happens after the
 first extension in several platforms.

 So, I'd suggest using this approach as a workaround, and for those it
 don't work (if any) just disable this particular test case.

 I've created a separate branch named trac2836-2 and pushed the
 suggested test.

 The crash problem will then become moot at least in the scope of this
 task.  But I also tried to avoid it in the same branch (although the
 solution may be controversial).  If you can review that part and think
 make sense, I can incorporate it in this task.  Otherwise we could
 defer that part to a separate task; it should be fixed, but not an
 immediate show stopper problem for our development.

 On a related note, we may want to be able to specify the max segment
 size for safety.  In our usage it doesn't make much sense to create a
 memory segment larger than the amount of available physical memory
 (an auth server frequently swapping in/out wouldn't be usable anyway),
 so it should be reasonable to specify the max around that value.  But
 that should certainly go to a separate ticket.

 And finally: while very small possibility, it's still possible that
 (revised) `SegmentObjectHolder` constructor throws an exception.  And
 if that happens it's quite likely that the passed `obj` will leak.
 So, to be very safe, we'll need to separate the constructor and
 actually setting the object:
 {{{#!cpp
     SegmentObjectHolder(util::MemorySegment& mem_sgmt, ARG_T arg) :
         mem_sgmt_(mem_sgmt), arg_(arg),
         holder_name_(getNextHolderName()), holding_(true)
     {
         // On construction, only prepare the space for the named address
         mem_sgmt_.setNamedAddress(holder_name_.c_str(), NULL);
     }
     void set(T* obj) {
         mem_sgmt_.setNamedAddress(holder_name_.c_str(), obj);
     }
 }}}

-- 
Ticket URL: <http://bind10.isc.org/ticket/2836#comment:15>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development