[kea-dev] Lease File Cleanup in Kea - Design Document

Fri Jan 9 16:25:56 UTC 2015

Tomek,

As we discussed this over the jabber, I will propose the change to the
design to bulk load leases from the input file and remove the
duplicates. I agree there are networks where that would perform much
better. I was thinking about the example of the cable network with a
lease lifetime being a week or so. Switching on and off the client would
cause a lot of new entries in the lease file that will not be removed
for a long period of time. On the other hand, if this is your ISPs
network you'd probably have a router connected to it which would be
powered-on for the whole time and the renewals would occur every half of
the lease duration or so. But, that's obviously depends on other things
like T1/T2 and it hard to make a good assumption for all networks.

So, overall I agree that it is better to do what you propose and relax
on the requirement that the unparsable parts of the lease file are not
rewritten. We can probably add support for this over time, if we feel it
is important.

Regarding the second request. I think it is a judgement call and there
will always be people who don't like a specific name for the
application. It is just a file name after all, so I don't want to spend
to much time on pushing one way or another. Note that this application
is not meant to be executed directly but will rather be executed by the
server process. Also, the purpose of this application is to perform
"Lease File Cleanup" and it is already abbreviated "LFC" in the existing
documentation so I think the name precisely describes the purpose.
Things like database-maintanance do not defend themselves in my opinion
because they are ambiguous. It is not the database, but a file. It is
not maintanance but cleanup, unless you precisely describe what
maintanance means in this context. If we wanted to call it like this we
should have started from naming the whole feature like this in the
design, requirements and other discussions. So, I object to changes to
the name and I prefer keeping names as short as possible.

Marcin

On 01/09/15 16:56, Tomek Mrugalski wrote:
> On 05.01.2015 16:31, Marcin Siodelski wrote:
> > Following the thread regarding the requirements for Lease File Cleanup
> > in Kea
> > (https://lists.isc.org/pipermail/kea-dev/2014-December/000211.html), I
> > have created a short design document for this feature:
> >
> > http://kea.isc.org/wiki/LFCDesign
> Apologies for replying after 4 days, but there was a holiday in between
> and I'm a bit under the weather.
>
> Anyway, I read that document and think it's in a good shape. I have two
> comments, though.
>
> 1. The clean-up algorithm will work, but will be very inefficient. Let
> me give you specific example. You have T1 set to 10 minutes and lifetime
> to 1 hour. A client comes in, gets and address, then keeps renewing
> every 10 minutes. After an hour, LFC process is triggered. How many
> entries for his lease you will have after LFC finishes. I think the
> answer is 5. You can tweak the parameters and get this algorithm as
> inefficient as you want, just lower the T1 to lifetime ratio. I think we
> should do something different: kea-lfc should instantiate
> MemfileLeaseMgr, tell it to read previous lease file and lease file copy
> (just as planned now). The code there is already able to handle
> sequential entries for the same lease. Once it's loaded, the code has to
> write down the whole content.
>
> The huge advantage of this approach is that for X leases you'll have
> exactly X entries in the output file. Another advantage is that exactly
> the same code is used to clean up the lease file and by the actual DHCP
> server, so it's much easier to maintain and the chances for and
> discrepancies are minimal. Finally, that approach would the
> implementation easier, because you don't need any extra columns, like
> the released. (That column can be useful for address affinity, but
> that's a separate matter and is a different feature, that has nothing to
> do with lease cleanup). So you could remove #3674 completely and make
> implementation of #3665 and #3666 simpler.
>
> The only real disavantage I see of this approach is that there won't be
> a way to retain unparseable entries. I think that's not a big issue,
> though. First, the unparseable entry will be ignored by the server (or
> worse, will cause the server to refuse to load the lease file). Second,
> I'm not aware of any good use cases for this. Two were given: one is a
> partially written lease when server experienced power outage and another
> was a sysadmin that added comments to the lease file. Both are weak.
> Partially written lease is unusable and often is followed by a random
> garbage. This cannot be recovered. The second one is really bizarre -
> lease file is an autogenerated file intended for automatic processing.
> Someone shutting down the server, injecting comments there and then
> restarting the server should not be surprised for them to be removed.
>
> Having said that, there's an easy workaround if you think that there's a
> value in keeping the unparsable elements. When the MemfileLeaseMgr code
> loads lease files and discovers something unparsable, it should log is
> (so sysadmin becomes aware of this) and copy the whole file aside
> (lease-file-unparsed-YYYY-MM-DD-HH-MM), so the sysadmin could intervene
> (and possibly manually recover the lease).
>
> 2. My second comment is trivial. I think we should not use 'lfc' in the
> kea-lfc name. Nobody would know off hand or even have a clue what that
> means. I would propose kea-cleanup, kea-db-maintenance,
> kea-memfile-maint or something similar. It's probably useful to signify
> that the cleanup is specific to memfile backend and there's no
> equivalent for MySQL or Postgres.
>
> I understand that my comments were sent a bit late, so I won't insist on
> implementing them now. In any case, if you think it's too late to
> consider comment #1 now, this is something that will have to be
> implemented at some later stage. I can imagine that there may be other
> clean up strategies that are better suited for specific deployments.
> Hence it seems reasonable to assume that sooner or later other cleanup
> algorithms will become available.
>
> Tomek
>
> _______________________________________________
> kea-dev mailing list
> kea-dev at lists.isc.org
> https://lists.isc.org/mailman/listinfo/kea-dev
>