BIND 10 #2375: implement MasterLexer getNextToken() and ungetToken()

[BIND 10 Development]

#2375: implement MasterLexer getNextToken() and ungetToken()
-------------------------------------+-------------------------------------
            Reporter:  jinmei        |                        Owner:
                Type:  task          |  UnAssigned
            Priority:  medium        |                       Status:  new
           Component:  libdns++      |                    Milestone:  New
           Sensitive:  0             |  Tasks
         Sub-Project:  DNS           |                     Keywords:
Estimated Difficulty:  0             |              Defect Severity:  N/A
         Total Hours:  0             |  Feature Depending on Ticket:
                                     |  loadzone-ng
                                     |          Add Hours to Ticket:  0
                                     |                    Internal?:  0
-------------------------------------+-------------------------------------
 Subtask of #2368.  Depend on #2369, #2370, #2371, #2372, #2373, #2374.

 With the help of state classes, getToken() should become pretty simple
 like this:

 {{{#!cpp
     MasterToken getNextToken(Options options) {
         using namespace internal = master_lexer_internal;
         for (internal::State* state = State::getInstance(Start);
              !state->isCompleted();
              state = state->handle(*this, options /*maybe source*/))
         {}
         return (token_); // should have been set by state object
     }
 }}}

 The others shouldn't be difficult to implement from the description.

 (Of course, corner cases and tests can be tricky.)

-- 
Ticket URL: <http://bind10.isc.org/ticket/2375>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development