BIND 10 #1634: Abstract pool/lease store: initial requirements list

[BIND 10 Development]

#1634: Abstract pool/lease store: initial requirements list
-------------------------------------+-------------------------------------
                   Reporter:  tomek  |                 Owner:  tomek
                       Type:  task   |                Status:  reviewing
                   Priority:  major  |             Milestone:  Sprint-
                  Component:  dhcp   |  DHCP-20120220
                   Keywords:         |            Resolution:
            Defect Severity:  N/A    |             Sensitive:  0
Feature Depending on Ticket:         |           Sub-Project:  DHCP
        Add Hours to Ticket:  0      |  Estimated Difficulty:  0
                  Internal?:  0      |           Total Hours:  0
-------------------------------------+-------------------------------------

Comment (by tomek):

 Replying to [comment:5 stephen]:
 > '''General Points'''[[BR]]
 There are many suggestions that I agree with. I have applied them.

 > '''Scalability Requirements'''[[BR]]
 > ''MUST support option definitions''[[BR]]
 > I would put this as "MUST be able to store user-defined options".
 Copied this. This are actually 2 requirements: first is to be able to
 store user-defined options and the second is to remember them for each
 assigned lease if present.

 > ''MUST allow multiple r/w access''[[BR]]
 > Clearer to put "MUST be able to support concurrent read/write access
 from multiple processes".
 Added as suggested, but that is somewhat difficult, e.g. sqlite blocks if
 more than one concurrent write is performed.

 > ''MUST scale from small (mostly sqlite) to medium (e.g. Postgres, MySQL,
 Oracle) to large (e.g. cassandra, hadoop) databases''[[BR]]
 > I would put this as "MUST be able to work with different databases".  It
 is quite possible that an SQLite database would be too slow and not
 support a large DHCP installation.  But that is the fault of the database
 and not a problem with the system.
 Reworded it and explained that this is a requirement for API itself, not
 the backend. API must support both small and very large installations.
 Some back-ends will be more appropriate for specific deployments than
 others. Added clarification to that effect.

 > '''Lease Information'''[[BR]]
 > Focus on the information to be stored, not the format.
 Done as suggested. Kept requirements unified, but lease and pool
 information is split into separate v4/v6 sections.

 > The following comments are about individual items but, unless the format
 of the data is mandated by an RFC, just identify the data to be stored.
 > * client-id/client-id-length/hw-address: No need to worry about the
 length of the client-id field.  If this is stored in a string in a
 database, the database will handle storing the size information.
 > * pool-id: There are no pointers in SQL true, but storing in one table
 the ID of a row in another table is very common, so does not need a
 remark.  This would be better described as "Pool from which the lease was
 assigned".
 > * string fqdn: if the fqdn is shared by multiple leases, it may be
 better not to store this as a string - perhaps a pointer to a row in
 another table.  For this reason, just refer to it as fqdn.
 > * address fqdn_dns: I would by this as "Indication of DNS server where
 DNS update was performed".  It might not be an address - it could be a
 pointer into a separate table that holds both the address of the server
 along with the TSIG information needed to access it.
 Removed exact type information from the design.

 While we don't have to decide on specific types, I think we should not be
 vague anymore regarding actual DB layout. I think it is a right time to
 start concrete DB layout design (i.e. tables, fields and relations). Once
 this is done, we can start thinking about API definitions to
 handle/manipulate such data.

 > '''Pool Information'''[[BR]]
 > Not sure what start_lease/end_lease is - it seems to be separate things
 for different address types.
 It is first and last lease in a pool. For address pools, it is first and
 last address, e.g. when you use 192.168.1.2 - 192.168.1.100
 (start_lease=192.168.1.2, end_lease = 192.168.1.100).
 But I get the point. This naming convention is somewhat ambiguous. Renamed
 to first_lease, last_lease.

 I don't want to name it first_addr and last_addr, because I want the same
 table format for both addresses and prefixes.

 > Would we want to store comments with individual pool entries?  They
 might be useful when inspecting the database.
 Yes, that may be useful. User configs frequently have comments around
 pools.

 > Do we want to store additional flags relating to pools, such as:
 > * Don't use before ''time''
 > * Don't use after ''time''
 > * Don't use
 I was more thinking about using hooks for that. We could define hooks for
 pool selection and use more rich semantics there than just a time-based
 selection. That would require adding extra hook, but I suppose that is ok.

 > '''Comments on Questions'''[[BR]]
 > ''Do we want to keep separate list of fixed-leases?''[[BR]]
 > This is a design issue, but I would say no.  Treating them the same as
 temporary leases - perhaps not even marking them as fixed, just setting
 the lease end time to a thousand years in the future - would seem to be
 the easiest for implementation purposes.
 I have thought about this more and I think we need separate table for host
 reservation after all. Keeping fixed-leases in table would be only part of
 the solution. On its own, it wouldn't work. Lease would have stopped being
 fixed after first client's RELEASE. The problem is that we now allow host
 definition to be populated with a lost of different things and fixed-
 address or fixed-prefix are just two of them. We can decide that this
 specific host gets extra option, or a different option. I think users will
 be very unhappy if we take away that possibility. Therefore

 > The need for name reservation is a requirement and should be explicitly
 included in the requirements.
 Added separate section for host reservation requirement.

 >
 > ''Do we want IPv4 and IPv6 lease structures unified?''[[BR]]
 > This is a also design issue - the requirements should just state what
 needs to be stored and modified.
 The more I think about it, the more differences I see. I split them for
 now. If we decide to go with unified format, we will just merge those
 lists.


 > ''Do we want to have subnets and pools split?''[[BR]]
 > The current split does seem a bit confusing - I can specify a range of
 addresses in a subnet, with subnet-specific parameters, or in a pool, with
 pool-specific parameters.  If we can get away from subnets, I would drop
 them.
 After giving more thought, I think we need both subnet and pool. Subnet
 defines that certain subnet is available on-link. That is required for 2
 things:
 - in v6 decide which pool to use, when we have link address of the relay
 only (relay includes  a global address that can be used to identify a
 link, where client is attached). This, by definition is not a dynamically
 assigned address, so it will always be outside of any pool. Therefore we
 need subnets.
 - when handling CONFIRM, server is supposed to answer a question "is this
 address valid for this link?", not "do you have binding for this lease?".
 To properly answer that question, server must have knowledge about full
 subnet on link, not only its dynamically allocated subset.
 Given those two facts, we must use both subnet and pool.

 Also there are valid cases, when there is one subnet, but many pools.

 > The more general question though is whether we want some form of pool
 hierarchy with the ability to associate attributes with any level of the
 hierarchy and with the resulting set of attributes for an address
 determined from the attributes at its level and those above it.
 No. One of the reasons we are rethinking configuration is to get rid of
 complexity, not exchange one complexity to another.

 > ''Do we want to store information in human readable form''[[BR]]
 > I would go for binary data first and then move to human-readable data if
 we want.  The reason here comes because human-readable data can be bad for
 operations that we want to do, e.g. the string "10.25.0.0" sorts after
 "10.192.0.0" even though numerically it comes before it.  This could cause
 problems if we need to sort retrieved data.
 Agree.

 > Alternatively, we could store both - use the binary data on input, but
 add an operation to fill in the human readable fields on output.
 >
 > ''Network topology''[[BR]]
 > Go for the minimalist approach - just store the minimum needed, not the
 whole topology.
 >
 > ''DNS Updates''[[BR]]
 > I suggested starting a new section in the requirements for DNS.
 Agree. Created new section.

-- 
Ticket URL: <http://bind10.isc.org/ticket/1634#comment:9>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development