BIND 10 #1648: Abstract pool/lease store: design

[BIND 10 Development]

#1648: Abstract pool/lease store: design
-------------------------------------+-------------------------------------
                   Reporter:  tomek  |                 Owner:  UnAssigned
                       Type:  task   |                Status:  reviewing
                   Priority:         |             Milestone:  Sprint-
  medium                             |  DHCP-20120703
                  Component:  data   |            Resolution:
  source                             |             Sensitive:  0
                   Keywords:         |           Sub-Project:  DHCP
            Defect Severity:  N/A    |  Estimated Difficulty:  0
Feature Depending on Ticket:         |           Total Hours:  0
        Add Hours to Ticket:  0      |
                  Internal?:  0      |
-------------------------------------+-------------------------------------

Comment (by tomek):

 Replying to [comment:6 stephen]:
 > Commment that DhcpPoolLeaseRequirements contains the requirements for
 the pool/lease store, but DhcpPoolLeaseDesign appears to contain more
 requirements - for the API - and not the design.
 I'm not sure what you meant by that. Are you proposing some shuffling
 between those two? Feel free to do it.

 > '''LEASES4 and LEASES6'''[[BR]]
 > Everything in this apart from available-leases or used-leases seems to
 be related to a single lease.  These two items are summary information
 about all leases.  Also, we just need to be able to obtain that
 information: we don't necessarily need to store it.  Whether it would be
 useful to denormalise the database and maintain a counter, or whether it
 is better to query the database everytime we need the numbers depends on
 the access pattern.
 >
 > A state transition table describing the lease lifetime (or a reference
 to one) would be helpful.
 Added a separate section with a diagram. It is done in open source tool
 called Dia. It is available in many distros.
 >
 > I think the design should also enumerate the operations needed in this
 part of the API, e.g. add, renew etc.
 That is completely different thing. It will be done as part of the
 abstract API design (as a header file).

 > '''DDNS information in lease structures'''[[BR]]
 > I think there is a need to decouple the DDNS information from the
 leases. We have talked about the idea of a separate DDNS process: if this
 is done, the information it needs should be stored in one place (the DNS
 table), not split across that and the lease table.  When a lease expires,
 the code should signal the DDNS process that the name should be removed.
 >
 > So I would suggest that the lease information just contains a dns-id.
 This is returned when a DNS entry is created, and is passed to the DDNS
 code on a renewal or expiration.  Also, as the fqdn may be shared by many
 leases, in a normalised database it should be stored only once (probably
 in one of the DDNS structutres): whether it should be stored (possibly
 multiple times) in the lease structure would depend on access pattern.
 That is still outstanding.

 > '''SUBNET/SUBNET6'''[[BR]]
 > Should this structure store a first address/last address?  Isn't that an
 attribute of the pool?
 True. Using network address/prefix length is better.

 > Regarding parameters we need to make a decision as to how we view the
 inheritance aspects as used in ISC DHCP.  For example, if we add
 parameters to a subnet declaration - and have a pool point to it and
 override some parameters - then we have a form of inheritance.  Is this
 what we want?
 That is part of the configuration design, not the lease storage design.

 > Also related to inheritance is the question of whether the inheritance
 is static or dynamic (i.e. if we change the subnet parameters, does the
 change immediately propagate to leases that don't override them)?
 See above.

 > The design should enumerate the operations needed.
 I was hoping to do this as part of the abstract API implementation, but
 here it is. Added a list of operations for all entities (execept DDNS, see
 below).

 > As the pool structures store comments, the subnet structures should do
 so as well.
 Added comments to all structures.

 > '''POOL4'''[[BR]]
 > The type code for this pool is wrong.
 >
 > I don't think we should use "lease" nomenclature here. To keep V4/V6
 consistency, why not just used "first" and "last" for the first/last
 addresses/prefixes?
 Fixed.

 > '''DDNS'''[[BR]]
 > I agree with the idea that setup information should be stored in the
 configuration database, but as suggested above, information about the data
 added to the zone should be stored in the database.
 >
 > I'm particularly thinking of the case where there is some problem in
 adding or removing the DNS information.  If the lease management component
 of the software puts relevant DNS information in the database, the DDNS
 component can run more or less independently.  For example, on lease
 expiration the lease could could remove the lease and signal the DDNS code
 to remove the DNS entry.  If there is some problem, the DNS code could
 keep retrying independently of the rest of the DHCP code. (This deals with
 the "DDNS clean up failure" point in the "Extra Features" section.)
 My impression was that the current proposal exactly does that.


 > '''HOST'''[[BR]]
 > Address information could be within a lease table.  All we need is a
 pointer to the lease information that could be NULL if the host is only
 being used to store options.
 Modified HOST structure to have pointers to other structures.

 > '''Extra Features'''[[BR]]
 > I agree with the need for leases consistency.  However:
 >
 > Pool deletion: as the pool is stored in the configuration database, the
 proposal here suggests that this would be modified by the code, which
 seems to be against the current philosphy of BIND 10.  Storing it in the
 database would make this easier, but I agree it logically fits in the
 configuration database.
 >
 > Automatic pool deletion when all leases have expired would be better
 but, for the reasons above, I think somewhat awkward to implement.  I
 would suggest:
 > * Allowing the current ISC DHCP feature of "(don't) use (after <date>)"
 in the pool configuration statements.  This forces the user to plan when
 migrating between pools.
 > * Having the configuration verification code reject an incompatible
 configuration change if leases are active in pools.
 >
 > By the latter, I mean that if one lease is active, shrinking the pool to
 contain just that address would be allowed: deleting it, or shrinking it
 so that the address is excluded would not be allowed. (An interesting
 question is whether we should allow configuration changes if the address
 were moved to a different pool.  In the short term, I would suggest not -
 we could look at it again in the longer term.)
 We are risking premature optimization here. For the first implementation,
 we will have a pool with leases in it. If a user deletes a pool that
 contains active leases, those clients will be out of luck. That is the
 simplest and the fastest way to implement this. Feel free to add your
 comments or improvement ideas to Extra features section.

 > '''Questions'''[[BR]]
 > __Q4 (representation of data)__[[BR]]
 > I think binary form will be faster (although the actual decision should
 not be made here - this is the API design).
 >
 > In an SQL database, we could always provide views to convert the data to
 human readable form (even SQLite3 allows a range of functions in a SELECT
 statement).
 We will use binary form.

 > __Q6 (fixed values or ranges for times)__[[BR]]
 > If it is easy to accept ranges, I suggest we allow ranges.
 Done.

 > __Q8 (what is in the database and what is in the configuration
 store)__[[BR]]
 > I think that the answer is that small amounts of data that is rarely
 updated got ijnto the configuration store, the rest goes into the
 database.  In practice, this is likely to be subnets, pools, DDNS
 configuration information (such as server, secret key etc.).
 >
 > This is touched upon in the paragraph after the LEASES6 section.  It
 might be clearer if this paragraph were expanded and moved to the
 introduction or into a separate section on data stores.
 Done.

 > __Q9 (reserved leases)__[[BR]]
 > I think there is an argument for putting fixed leases in pools.
 No, not anymore. See my proposal 1 in mail "Kea PoolLeaseDesign:
 configuration proposals" from 2012-07-09.

 > After all, when allocating a lease from a pool, we still need to check
 the collection of existing leases allocated from the pool before assigning
 a new address.  Having the permanent lease be part of that collection
 would not add much overhead.  However, we would still need some global
 list to locate the lease when the client for whom the lease is reserved
 connects.
 Yes, but then we will need to check if released lease should really be
 released or put back to "fixed, but not used now" state. The same holds
 true for expiration. It is cleaner to keep fixed and dynamic allocations
 separated - easier, cleaner and better performance.

 > __Q12 (specifying DDNS Information)__[[BR]]
 > Q13 has suggested that we need some form of parameter inheritance, of
 the form:
 > {{{
 > global -> subnet -> pool
 > }}}
 > I suggest that the DDNS information be inheritable information, i.e.
 defined global unless overridden at subnet or pool level.
 Note that this is a configuration aspect, not a storage aspect.

 > __Q13 (inheritance hierarchy)__[[BR]]
 > The current ISC DHCP product allows classes and the specification of
 parameters based on class.  Is this something we want to natively include?
 If not, should we allow the creation of class-based collections of
 information that are unused by the native Kea code, but accessible to user
 hook code?
 No. We will deal with classes at a later date. Classes are likely to be
 implemented as hooks.

-- 
Ticket URL: <http://bind10.isc.org/ticket/1648#comment:8>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development