On lease expiration
tomasz at isc.org
Tue Jul 8 20:36:36 UTC 2014
This is another mail preparing us for tasks that are planned past 0.9
Currently Kea does not take any specific actions when a lease expires.
However, when picking new leases for a new client, it can reuse a lease
if it is expired. This approach is not ideal, it was just the best we
could do given the schedules we had. We know that this is not ideal,
hence we plan to implement it properly after 0.9 release.
Upon lease expiration, we need to initiate one or more actions. Those
actions are (the list is not exhaustive and may grow over time):
- logging the expiration (that sometimes is a requirement from the law
- triggering DDNS removal (required every time DDNS is used)
- calling hooks (new hook point will be defined for this)
- notifying failover partner (we don't have failover support yet, but
we will get it one day)
- notifying active leasequery requestors (we don't have active LQ
support, but we will get it one day)
- remove the lease from the DB (that may be optional, as the lease
could be fixed or maybe sysadmin prefers to keep expired leases, so
the returning client will get the same address if it's still
In all cases we need to implement house-keeping routine that will pick
expired leases, perform all actions on it and possibly remove it. That
routine will look the same, regardless of how we choose to trigger it.
How this could work? There are couple possible approaches:
a) Implement house-keeping process that runs in the background. It will
be periodically sweeping the database and will pick expired leases. The
biggest advantage of this approach is its relative simplicity.
Implementation is not related to any existing code, so the likelihood of
any side effects are minimal. It seems easy to implement, but in fact it
is quite complex, as it will need to implement its own configuration
parser. Once we decide to go multi-process, it would be reasonable to
have a single house-keeping process. These are all benefits I can think
of. Unfortunately, there are numerous disadvantages. First, it would be
inherently incompatible with memfile, our default backend. This is a big
problem for me as I think that memfile will also be the most popular
backend due to its installation simplicity. Second, stand alone process
would call all external actions in a separate process, so users writing
hook libraries will need to develop multiprocess capabilities, even if
we have only one Kea server. Third, I have no idea how to implement
failover with stand alone house keeper. It will have to notify somehow
the Kea server that a lease expired and its partner must be notified (or
other failover actions must take place). So we're talking about
inter-process notification protocol here on top of failover. Finally,
this is a new process that needs to be maintained (new makefiles, new
man page, new binaries to install, new documentation etc.). The bottom
line for me is that if we decide to do stand-alone process, we're
getting short term benefits, but we'll pay for it with much larger
development and maintenance costs in the long term.
b) implement house-keeping as part of the existing process. There would
be a configuration parameter that would tell the server to trigger the
routine every X seconds. The server would call select() for at most X
seconds, run house-keeping and then call select() again. This approach
seems very flexible. If you really need a notification the very second
when the lease expires, set X to 1. If you're experiencing an event
where extra performance is needed, set it to some large value, so
expiration will be checked upon the next day. Of course, there are
tradeoffs. The longer value set means that, although the procedure will
not happen that often, each house-keeping will take longer as it will
have more leases to process. The benefit of this approach is that
logging, hooks, ddns happen in the same process, so no multi-process
hassles for us and for hook lib developers. Also, failover does not
become more complicated. It is easy to implement as the whole design of
existing code is ready for this (calculating timeout + calling select()
with that timeout).
c) that's really b) + extra capability of being able to disable
house-keeping, at least during normal operation. House-keeping would be
called during startup and shutdown. So if you really need the absolute
max performance and intermediate pauses needed for house-keeping are a
problem for you, use this. You will get the lease expiration
notification after some time ("Least A expired Y minutes ago"), but
that's ok if max performance is your goal.
d) That's b) + an optimization. When assigning or updating a lease, we
can keep the timestamp to the shortest expiration event and dynamically
set X to that value. So if there are no leases, we'll select() for
MAX_UINT. If there are 1000s of leases, we'll select() until the
shortest (or oldest) one expires. This is how expiration is implemented
in Dibbler. Seems to be working fine.
Solutions b) and c) have a small disadvantage when we decide to enable
Kea to run multiple instances. That will mean that if you have X
processes, the house-keeping will be run X times. In principle that's
not optimal, but it isn't that bad. Although there will be multiple
checks run, but on average each routine will have to only process X
times less leases. Also, this can be easily solved by having a
designated process (e.g. only the first instance runs house-keeping,
additional instances have house-keeping disabled).
e) Over time, we will also need to implement a new command: lease-expire
or db-cleanup. It would add extra capability for admins who tweaked
something in the DB and want Kea to process the changes. Also, some
admins would possibly want to use their external scripts to do lease
expirations, e.g. during low traffic at 3am.
Actual house-keeping routine implementation is lease database specific.
For SQL-based backends, we'll probably implement a query that returns
expired lease. For memfile, we'll either add an index or will have to
traverse the leases. It would be good if we could do that using
bisection. I haven't looked at what's available in multi-index containers.
More information about the kea-dev