[kea-dev] Lease File Cleanup in Kea - Design Document

[Tomek Mrugalski]

On 05.01.2015 16:31, Marcin Siodelski wrote:
> Following the thread regarding the requirements for Lease File Cleanup
> in Kea
> (https://lists.isc.org/pipermail/kea-dev/2014-December/000211.html), I
> have created a short design document for this feature:
> 
> http://kea.isc.org/wiki/LFCDesign
Apologies for replying after 4 days, but there was a holiday in between
and I'm a bit under the weather.

Anyway, I read that document and think it's in a good shape. I have two
comments, though.

1. The clean-up algorithm will work, but will be very inefficient. Let
me give you specific example. You have T1 set to 10 minutes and lifetime
to 1 hour. A client comes in, gets and address, then keeps renewing
every 10 minutes. After an hour, LFC process is triggered. How many
entries for his lease you will have after LFC finishes. I think the
answer is 5. You can tweak the parameters and get this algorithm as
inefficient as you want, just lower the T1 to lifetime ratio. I think we
should do something different: kea-lfc should instantiate
MemfileLeaseMgr, tell it to read previous lease file and lease file copy
(just as planned now). The code there is already able to handle
sequential entries for the same lease. Once it's loaded, the code has to
write down the whole content.

The huge advantage of this approach is that for X leases you'll have
exactly X entries in the output file. Another advantage is that exactly
the same code is used to clean up the lease file and by the actual DHCP
server, so it's much easier to maintain and the chances for and
discrepancies are minimal. Finally, that approach would the
implementation easier, because you don't need any extra columns, like
the released. (That column can be useful for address affinity, but
that's a separate matter and is a different feature, that has nothing to
do with lease cleanup). So you could remove #3674 completely and make
implementation of #3665 and #3666 simpler.

The only real disavantage I see of this approach is that there won't be
a way to retain unparseable entries. I think that's not a big issue,
though. First, the unparseable entry will be ignored by the server (or
worse, will cause the server to refuse to load the lease file). Second,
I'm not aware of any good use cases for this. Two were given: one is a
partially written lease when server experienced power outage and another
was a sysadmin that added comments to the lease file. Both are weak.
Partially written lease is unusable and often is followed by a random
garbage. This cannot be recovered. The second one is really bizarre -
lease file is an autogenerated file intended for automatic processing.
Someone shutting down the server, injecting comments there and then
restarting the server should not be surprised for them to be removed.

Having said that, there's an easy workaround if you think that there's a
value in keeping the unparsable elements. When the MemfileLeaseMgr code
loads lease files and discovers something unparsable, it should log is
(so sysadmin becomes aware of this) and copy the whole file aside
(lease-file-unparsed-YYYY-MM-DD-HH-MM), so the sysadmin could intervene
(and possibly manually recover the lease).

2. My second comment is trivial. I think we should not use 'lfc' in the
kea-lfc name. Nobody would know off hand or even have a clue what that
means. I would propose kea-cleanup, kea-db-maintenance,
kea-memfile-maint or something similar. It's probably useful to signify
that the cleanup is specific to memfile backend and there's no
equivalent for MySQL or Postgres.

I understand that my comments were sent a bit late, so I won't insist on
implementing them now. In any case, if you think it's too late to
consider comment #1 now, this is something that will have to be
implemented at some later stage. I can imagine that there may be other
clean up strategies that are better suited for specific deployments.
Hence it seems reasonable to assume that sooner or later other cleanup
algorithms will become available.

Tomek