[kea-dev] Pool usage monitoring in Kea ?

Marcin Siodelski marcin at isc.org
Mon Nov 24 17:26:57 UTC 2014


The ticket http://kea.isc.org/ticket/3457 (currently in the 0.9.1.
release milestone) is about enabling access for the hook libraries to
the lease manager. This would allow for making queries to the lease
database (of any type) and perhaps collecting some statistical
information. Unfortunately, this is a future work. Plus, depending on
the type of the statistics, the lease information returned by the lease
manager's API may require some additional processing to get the desired
counters. But, this is usually unavoidable and doesn't cause much burden
as it operates on in-memory objects.

Marcin

On 11/21/14 10:36, Chaigneau, Nicolas wrote:
> Alright, so I've a solution for the missing part: how to retrieve the leases currently in use, within each defined subnet, upon server startup.
> 
> In the hook library "load" function, grab the lease file and kea config file.
> Parse them, populate appropriate memory objects, and voila!
> 
> But it's not terribly efficient: the lease file is parsed twice, first by Kea, then by the hook library.
> 
> There should be a better way to do this...
> 
> 
> 
> Regards,
> Nicolas.
> 
> 
> -----Message d'origine-----
> De : kea-dev-bounces at lists.isc.org [mailto:kea-dev-bounces at lists.isc.org] De la part de Chaigneau, Nicolas
> Envoyé : mardi 18 novembre 2014 17:32
> À : Tomek Mrugalski; kea-dev at lists.isc.org
> Objet : Re: [kea-dev] Pool usage monitoring in Kea ?
> 
> Hello,
> 
> 
> Thanks for your input!
> 
> 
> I've looked into Kea hooks API, and built my very first (and very rudimentary) hook library to get a grasp of what can be done.
> 
> So far with lease4_select, lease4_release (and lease4_expire in the future) I'm able to iterate on the pools to get the total number of leases associated to a subnet, keep track of the number of used leases, and report the subnet pools usage.
> 
> One thing I don't know is how to get the number of leases loaded from the persistent lease file upon server startup.
> Could it be possible to add a hook for the event "lease loaded from persistent file" ? or provide another way to achieve this ?
> 
> Another question: would it be complicated to add the possibility to get the number of allocated leases from the hooks arguments ?
> (mustn't the server know how many leases are free/occupied when trying to allocate a new one ? that's probably a stupid question, but I don't know how subnets/pools and leases objects are linked...)
> 
> 
> Your proposal of allowing to query the server for statistics is interesting.
> 
> What I need is to be able to:
> 1) get the list of configured subnets (at least their id and text representation, eg: "10.156.0.0/15". Possibly also the associated relay addr).
> And, for each subnet: the total number of addresses in the pools, and the number of currently leased addresses.
> 2) get the number of each type of DHCP packet (Discover, Ack...) received and sent by the server during a given time interval (which would have to be configurable).
> 
> Most of this can be achieved through the hooks API:
> 2) can easily be done with hooks "pkt4_receive" and "pkt4_send".
> 1) can partially be done with hooks "lease4_select", "lease4_release" and "lease4_expire".
> 
> As you said it's a bit "hacky" though, and if there's a better way to do it, I'll take it :)
> 
> 
> 
> 
> Have a nice vacation !
> 
> Regards,
> Nicolas.
> 
> 
>> On 14/11/14 03:43, Chaigneau, Nicolas wrote:
>>> I'm looking into the possibility to monitor the pools usage in Kea, 
>>> more specifically for a "memfile" back-end. For dhcpd, we've been 
>>> using an external tool: dhcpd-pools
>>> (http://dhcpd-pools.sourceforge.net/) which works by parsing dhcpd 
>>> config and lease files. This tool is used at regular time intervals 
>>> to produce reports in a log file.
>>>
>>> This could be ported to support Kea, but I wonder if there could be 
>>> a better way ? For one thing, dhcpd-pools has to parse the lease 
>>> file, and this is quite slow for very large lease files.
>> Possibly, yes. One thing to remember is that the lease file will contain expired leases. They should not be counted as used as they are available for the server assignment.
>>
>>> Would it be possible to use Kea hook API to provide this 
>>> functionnality ? I've seen the "lease4_select" hook which seems 
>>> interesting. Do you think the "subnet4collection" argument could be 
>>> used to check how many leases are currently allocated for each pool, 
>>> and report the information ?
>> You could implement your own statistics module using hooks. You'd create a table of integers called subnets_counter and a size of subnet4collection returned in subnet4_select for example. You'd need to manage that statistic from two points. First, lease4_select callout would increase a given statistic when new lease is assigned (just pay attention to fake_allocation flag - the callout is called also during Discover processing and the lease is not assigned then) or existing lease is released (see lease4_release callout).
>>
>> That would give you very efficient statistics gathering mechanism. It would be accurate as long as no leases are expired. Unfortunately, we don't have the proper expiration mechanism yet. Once it is implemented, we'll likely add extra callouts: lease4_expire and lease6_expire.
>>
>> However, this approach is a bit hacky. The proper solution for statistics gathering will be more involved. In bind10 days, Kea had a control socket that used to receive commands from bind10 framework. The commands and the responses from the server were JSON structures. When we removed the bind10 framework, we lost the communication channel. Kea now has the capability to process the commands, just there's no way to receive those commands or send responses. This is something we plan to do in 0.9.1 release. It won't be super-fancy - probably just a unix socket for now.
>>
>> Currently Kea supports 3 commands: shutdown, libreload, config-reload.
>> As a temporary workaround, we use signals to trigger those commands.
>>
>> It seems reasonable to extend that commands list with an additional command, maybe 'stats-get'. If you are interested, please take a look at ControlledDhcpv4Srv::processCommand in src/bin/dhcp4/ctrl_dhcp4_srv.cc.
>>
>> As for the actual stats gathering, the statistics code should work 
>> uniformly for all backends. We would probably need to extend the 
>> generic API to LeaseMgr (see src/lib/dhcpsrv/lease_mgr.h) with 
>> something like
>> getStats() that would return an array with number of leases in each subnet (or rather an array of structures, with each structure having least count information. That way we could extra stats in the future without changing the API).
>>
>> Once LeaseMgr API is extended, we'd need to implement for specific backends, e.g. Memfile_LeaseMgr (see src/lib/dhcpsrv/memfile_lease_mgr.h).
>>
>> This is how I would do statistics. Please note that this is a napkin style design. Just something Marcin and I discussed very briefly, without giving it more thought. One thing to remember is that the number of statistics will keep growing over time. For now, you only need lease number, but after that, additional metrics will be requested, some of which can't be extracted from the lease manager. For example - a number of total leases, number of reserved hosts, number of reserved hosts that have active leases, number of malformed packets received, number of packets that caused subnet selection failures etc.
>>
>> I suppose that's all I can offer at this time. Statistics is something that we definitely will want to implement. What would be very useful is if you could think about what type of statistics you'd want. We won't be able to implement them all immediately, but knowing what will be needed eventually, will be helpful with the design decisions.
>>
>> Ok, this is my last mail to kea-dev for the next 2,5 weeks. IETF meeting is wrapping up today and I'm going on vacation.
>>
>> Cheers,
>> Tomek
>>
> This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.
> 
> _______________________________________________
> kea-dev mailing list
> kea-dev at lists.isc.org
> https://lists.isc.org/mailman/listinfo/kea-dev
> 


More information about the kea-dev mailing list