[kea-dev] Lease File Cleanup in Kea - Design Document

Mon Jan 26 11:03:13 UTC 2015

All,

I have updated the Lease File Cleanup design with the comments from
Stephen and Tomek (see ticket #3685). You may want to have a look and
see if there is anything glaring. Minor comments like: spelling errors,
some naming conventions may also be useful but they have little chance
of being applied by the 0.9.1 release.

The major change (which was actually applied in the LFC requirements
document): http://kea.isc.org/wiki/LeaseFileCompressionRequirements is
the removal of the following requirement:

"b. The LFC should not remove the contents of the lease file which is
not identified as a lease information or is an invalid lease information."

Although, I initially thought it would be a useful capability, I now
tend to think that it is not worth the effort for the following reasons:
- We don't support comments in the lease files, and I don't think that
supporting them really makes any sense, because lease file is not meant
to be edited manually
- If the server crashes when it is writing a lease into the lease file,
and the lease information is incomplete, the server could rewrite the
incomplete lease to the new file. But, how this incomplete entry would
be fixed? Manually? By that time the client would probably return and
get its lease back.

However, the most important point is that implementation of this
capability would require tracking the position of the unparsable data in
the source lease file so as it will be rewritten in the same position
(e.g. between lease X and Y) in the output file. At the same time, the
updated LFC algorithm now loads all leases into memory, instead of
reading and writing them one-by-one which makes it troublesome to match
the position of the particular piece of unparsable information within
the lease data structure, because current data
structures do not support this capability.

Please read the design document: http://kea.isc.org/wiki/LFCDesign and
report issues on the mailing list.

Marcin

On 01/09/15 15:51, Marcin Siodelski wrote:
> Stephen,
> 
> Thanks for the comments. I will go through them some time next week.
> Note that Shawn had a reviewed the design and the relevant ticket was
> closed. Also, the implementation is already in progress. However, it is
> probably still easy to include some changes.
> 
> Marcin
> 
> On 01/09/15 15:33, Stephen Morris wrote:
>> My comments are mainly concerned with the recovery from crashes.
>>
>> Terminology
>> ---
>> I think that "Lease File Copy" is a misnomer, because the file is not a
>> copy of the current lease file: it is the lease file as it was before a
>> new one was started.
>>
>> I would also suggest that numberic suffixes are clearer to indicate
>> the ordering of lease files.  So:
>>
>> lease-file ... is the current lease file
>> lease-file.1 ... is the what the current lease file is renamed to
>> lease-file.2 ... is the result of the previous LFC run
>>
>> Reading the lease files in descending order of version (2, 1, <blank>)
>> reads the lease information in the correct order.
>>
>> We do need two more filenames, and I suggest:
>>
>> lease-file.output (currently named LFC Output File)
>> lease-file.completed (LFC Output File when LFC has completed, see below)
>>
>> I'm not hung up on names though, so choose whatever seems best.  In
>> the remaining comments I've used the names given in the design.
>>
>>
>> State Diagram
>> ---
>> The bit that concerned me was the comment about doubled information if
>> the LFC crashes between renaming the LFC Output File and deleting the
>> Lease File Copy. (Yes, it is unlikely, but I have a pessimistic view
>> of the Universe.)
>>
>> The described situation is due to the fact that after the LFC Output
>> File has been renamed to the Previous Lease File, the state is
>> ambiguous. Either:
>>
>> a) The last LFC completed successfully: the Previous Lease File holds
>> the result of the last LFC run and the Lease File Copy is the result
>> of the server renaming the current lease file since then.
>>
>> b) The last LFC completed successfully: the Previous Lease File holds
>> the result of the last LFC run and the Lease File Copy is one of the
>> input files to the last LFC run.
>>
>> To address this and remove the (admittedly small) risk of duplicate
>> information, once the cleanup process has completed, the LFC Output
>> File should be renamed before the file deletion starts. So the
>> sequence would be:
>>
>>    Rename LFC Output File to LFC Completed File
>>    Delete Previous Lease File
>>    Delete Lease File Copy
>>    Rename LFC Completed File to Previous Lease File
>>
>> Now the state is unambiguous when the LFC starts up:
>>
>>    If (LFC Completed File exists) {
>>       // Previous LFC processing completed apart from file
>>       // deletion.  Finish the deletion & exit.
>>       Delete Previous Lease File
>>       Delete Lease File Copy
>>       Rename LFC Completed File to Previous Lease File
>>       exit
>>
>>    } else if (LFC Output File exists) {
>>       // Previous LFC processing did not complete.  Restart
>>       // the processing from the beginning.
>>       Delete LFC Output File
>>       Do LFC Processing
>>
>>    } else {
>>       // Previous LFC processing completed successfully. Start
>>       // the next set of processing.
>>       Do LFC Processing
>>    }
>>
>>
>> Server Startup
>> ---
>> When the server starts up, it needs to read the lease files.  This
>> operation needs to take account of the possibility of a "LFC Completed
>> File".  If that is present, it should read that instead of the "Lease
>> File Copy" and "Previous Lease" files.
>>
>> A second point about server startup is that it should ensure that no
>> LFC process is running when it reads the lease files.  The scenario
>> exists whereby LFC is running when the server crashes; as the server
>> starts up again and reads the lease files, the LFC process completes
>> and starts deleting those files.
>>
>>
>> Stephen
>> _______________________________________________
>> kea-dev mailing list
>> kea-dev at lists.isc.org
>> https://lists.isc.org/mailman/listinfo/kea-dev
>>
> _______________________________________________
> kea-dev mailing list
> kea-dev at lists.isc.org
> https://lists.isc.org/mailman/listinfo/kea-dev
> 