[kea-dev] update on this ....FRe: thoughts on 3601

Thomas Markwalder tmark at isc.org
Fri Oct 30 13:38:00 UTC 2015


On 10/30/15 9:13 AM, Tomek Mrugalski wrote:
> On 30/10/15 19:57, Thomas Markwalder wrote:
>> So there is a minor twist to the VersionedCSVFile approach.
>>
>> Currently, it allows one to receive current schema data from
>> a file containing prior schema records. For example
>> if the current schema is 3.0 and Kea is given a 1.0 lease file:
>>
>> 1.0 file --> VersionedCSVFile --> 3.0 data ---> MemfileLeaseMgr
>>
>> This will occur automatically whenever MemfileLeaseMgr loads the
>> lease file(s).  Currently, this happens inside the MemfileLeaseMgr
>> constructor.
>>
>> At this point, however, the physical lease file(s) are still 1.0 form.
>> The header row has the list of 1.0 columns, the data rows have 1.0 data.
>> Any new records written to the file will be in 3.0 form.  Operationally,
>> this is fine although not optimal. VersionedCSVFile can load files that
>> have mixed generation data.  In fact, such a file could contain 1.0, 2.0,
>> and 3.0 records and it will load correctly.  Whenever the file is loaded,
>> older schema records are upgraded, current schema records pass thru. Kea
>> always gets all records in current schema form.
>>
>> The twist is how do we physically convert the files?
> Do I understand it correctly that once you load 1.0 file into Kea
> version that expects 3.0 data, it will append 3.0 data to a file that
> has 1.0 header? 
Yes, that is what would happen if the server began appending to
the 1.0 file.

> If that is so, I think it's a very bad idea. It would
> make the lease file inconsistent and difficult to process by any
> software that happens to not be Kea. That was one of the reasons why we
> decided to use CSV in the first place: to be able to process the file by
> other software.
I didn't say it was necessarily a good thing, I said it would be operate.

> I think that once the version inconsistency is detected, the software
> should rewrite the whole CSV file to a new format. This will introduce
> an extended startup time after Kea upgrade, but that's perfectly ok.
> Software upgrades are infrequent (and planned) events.
>
>> The most obvious answer is to let LFC take care of it.  LFC will convert
>> them automatically.  Old schema records go in, current scheme lease files
>> are written out.  Of course this only happens if LFC is enabled or they
>> run it manually.
>>
>> Keeping in mind the VerionedCSVFile can expose the knowledge that the data
>> required updating, what should we do if we detect at start up that
>> LFC is disabled AND the files need upgrading?
>>
>> 1. We log an error message, stating they should enable LFC or run it
>> manually.
>>
>> We make sure document by example how to do this for the purposes of
>> upgrading.
>> (We should probably do this anyway)
>>
>> 2. We trigger a one-shot run of LFC when Memfile lease manager starts up
>> IF we detect an upgrade is needed AND LFC is disabled.
> I would go even one step further and refuse to start the server until
> the LFC upgrade run is completed. It's much better to have couple
> seconds downtime that is planned and avoidable if needed. Does 10
> seconds pause bothers you? Then don't upgrade at this time. The
> alternative, to have an inconsistent database, is much worse.
Making the server not process requests until LFC is complete on a one shot
basis may be harder than one might think.   Though I suppose there might be
other states when we would want servers to drop requests until some
condition
clears.  Would need to investigate this. 

Also keep in mind how LFC works.   Before the actual process is invoked
the server copies the current lease file to the copy file, so the 1.0
file goes to 1.0.copy.
Then it opens a new file, which will be 3.0 and hence all records
written to it would be
3.0.  In other words, just by doing the setup prior to LFC running, Kea
starts working
with a current schema lease file.

I think it might be close enough to have it start LFC with the smallest
possible
interval.   It might even be possible to kick it without the timer, I'd
have to look at that
more closely.

> On a related note, I think having Kea the ability to upgrade on load is
> fine, but we still should provide kea-admin upgrade capability, even if
> it only calls kea-lfc. This is purely from operational perspective.
> Without it, one could not upgrade the database without running the DHCP
> service. Update and providing service are two different operations and
> there may be valid scenarios, where you want to just do the upgrade, but
> not run the service.
We could make kea-admin call LFC for this.   In reality, we ought to be
able to do that
through kea-admin anyway.  I don't have an issue with that.  It still keeps
all of the conversion know-how in one place.   I would be opposed to
implementing a
secondary script type of conversion for this,  unless of we discard the
idea of automatic
conversion.


> Tomek
>
> _______________________________________________
> kea-dev mailing list
> kea-dev at lists.isc.org
> https://lists.isc.org/mailman/listinfo/kea-dev




More information about the kea-dev mailing list