BIND 10 #2369: InputSource helper class for MasterLexer

[BIND 10 Development]

#2369: InputSource helper class for MasterLexer
-------------------------------------+-------------------------------------
                   Reporter:         |                 Owner:  vorner
  jinmei                             |                Status:  reviewing
                       Type:  task   |             Milestone:
                   Priority:         |  Sprint-20121106
  medium                             |            Resolution:
                  Component:         |             Sensitive:  0
  libdns++                           |           Sub-Project:  DNS
                   Keywords:         |  Estimated Difficulty:  4
            Defect Severity:  N/A    |           Total Hours:  0
Feature Depending on Ticket:         |
  loadzone-ng                        |
        Add Hours to Ticket:  0      |
                  Internal?:  0      |
-------------------------------------+-------------------------------------
Changes (by muks):

 * owner:  muks => vorner


Comment:

 Hi Michal

 Replying to [comment:7 vorner]:
 > '''General interface discussion''':
 >
 > From how the interface in the ticket looks like, it really means the
 internal
 > storage should probably grow indefinitely. Or, well, see below for
 ideas.
 >
 > Anyway, is there any use for the `ungetAll()`? If it always rewinds to
 the
 > start, maybe it is better to just create new `InputSource` in that case.

 I had mentioned on the weekly call too about creating a new `InputSource`
 every time. But the caller will still have to track line numbers in this
 case, make it seek, etc. `compact()` has been added that works around the
 buffer size issue, so it should be fine I suppose.

 > Also, in this ticket, I don't see the use of the `saveLine()`. Does it
 make any
 > sense to save the line and then rewind all the way to the beginning?
 Then the
 > line numbers will be wrong, won't they? Should the `saveLine()` and
 `compact()`
 > be a single method, so the `saveLine()` saves both the line and the
 position?

 Yes it would be a problem if we rewind to a position before where the
 saved line starts. IIRC we thought that as it's an internal class to
 `MasterLexer`, the lexer uses it knowing this behavior. In any case, it
 may be worth merging `saveLine()` and `compact()`, and perhaps call it
 something more meaningful like `mark()`.

 > '''Idea how to solve the growing''':
 >
 > The problem is with `ungetAll()`. As I mentioned above, I don't know if
 it is
 > really needed. But the `ungetChar()` can have the parameter ‒ the
 character to
 > return, then we could have buffer only for the returned characters.

 We don't use it this way (at least in my reading of it), i.e., as a pure
 push-back buffer where we only read back what we `ungetChar()`.

 > And, usually, we we'll be on top of files. Rewinding a file is possible
 ‒ we
 > just start reading it again. So we can have a special-case for the file
 > version, that does not buffer, but rewinds the underlying file.

 As long as the lexer `compact()`s when starting to scan new tokens, we
 don't need to worry about the buffer size I guess.

-- 
Ticket URL: <http://bind10.isc.org/ticket/2369#comment:14>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development