[kea-dev] update on this ....FRe: thoughts on 3601

[Thomas Markwalder]

On 10/30/15 8:42 AM, Stephen Morris wrote:
> On 30/10/15 10:57, Thomas Markwalder wrote:
>
>> Currently, it allows one to receive current schema data from
>> a file containing prior schema records. For example
>> if the current schema is 3.0 and Kea is given a 1.0 lease file:
>>
>>
>> 1.0 file --> VersionedCSVFile --> 3.0 data ---> MemfileLeaseMgr
>>
>> This will occur automatically whenever MemfileLeaseMgr loads the
>> lease file(s).  Currently, this happens inside the MemfileLeaseMgr
>> constructor.
>>
>> At this point, however, the physical lease file(s) are still 1.0 form.
>> The header row has the list of 1.0 columns, the data rows have 1.0 data.
>> Any new records written to the file will be in 3.0 form.  Operationally,
>> this is fine although not optimal. VersionedCSVFile can load files that
>> have mixed generation data.  In fact, such a file could contain 1.0, 2.0,
>> and 3.0 records and it will load correctly.  Whenever the file is loaded,
>> older schema records are upgraded, current schema records pass thru. Kea
>> always gets all records in current schema form.
>>
>> The twist is how do we physically convert the files?
>>
>> The most obvious answer is to let LFC take care of it.  LFC will convert
>> them automatically.  Old schema records go in, current scheme lease files
>> are written out.  Of course this only happens if LFC is enabled or they
>> run it manually.
> Is this really a problem?
>
> AIUI, the idea is that when Kea reads a line from the lease file, the
> number of columns determines what memfile schema was used to write that
> line. If the number of columns is fewer that that required by the
> current version of Kea, defaults are applied to obtain the missing
> values. If the number of columns is greater, the additional information
> is ignored. (Presumably this could occur if someone upgraded Kea, ran it
> for a bit, then reverted back to the previous version.)
You're correct, operationally it isn't a problem  Kea can work with
mixed-generation
files, across reloads/restarts indefinitely.   The files would look a
bit unusual upon
visual inspection or importation into a spreadsheet, but only to the
uninformed eye.

Minor point. currently VersionedCSVFile considers an input file with
more columns
than are defined as invalid.  Primarily because I didn't think a newer
schema might
remove columns.   One could address this by expanding the meta-data for each
column to include an upgrade flag such as  "delete" or "ignore".   I'll
look into it.

> If LFC is enabled, then the files will get converted to the new format
> when it first runs.  But Kea does not "need" the file to be converted
> and will work perfectly well if LFC is disabled.
100% correct.

>> 1. We log an error message, stating they should enable LFC or run it
>> manually.
>>
>> We make sure document by example how to do this for the purposes of
>> upgrading.
>> (We should probably do this anyway)
>>
>> 2. We trigger a one-shot run of LFC when Memfile lease manager starts up
>> IF we
>> detect an upgrade is needed AND LFC is disabled.
> I'd suggest just logging a warning message if we detect a schema upgrade
> and LFC is disabled.
I tend to agree.    On an tangential note a control channel command to
run LFC is probably a good thing to have.

> Of course, this assumes that any version of Kea will always be able to
> read files created with any previous version of Kea.
That is intent at least.   I suppose if we reach a point where the
schema or format of the file change radically we might need to employ
a different strategy.


> Stephen
> _______________________________________________
> kea-dev mailing list
> kea-dev at lists.isc.org
> https://lists.isc.org/mailman/listinfo/kea-dev