[Kea-users] lease reading error

Wlodek Wencel wlodek at isc.org
Fri Oct 28 11:42:20 UTC 2016


We investigated that issue. Leases that causes kea to start were those
which were declined before.

Declined lease is saved in csv file in this format:
172.31.30.240,,,86400,1471832013,1,0,0,,1
without mac address or client id and such leases kea treats as invalid
on startup. In your case 66241 lines from your csv file is enough to
prevent kea from starting up.

I opened the ticket for this issue https://kea.isc.org/ticket/5058

For now as quick fix for you csv file you can just remove declined
leases from file using sed
sed -i.backup '/,,,/d' /var/lib/kea/kea-leases4.csv

Thanks for reporting this issue.
Włodek Wencel
ISC QA

On 10/26/2016 06:16 PM, Wlodek Wencel wrote:
> Thank you very much I will try to reproduce this issue later today. But
> after quick look into your csv file I picked up one randomly choosen mac
> address and:
> 
> cat kea-leases4.csv | grep d8:50:e6:84:3d:80 | wc -l
> 2422
> 
> you have 2422 entries for single mac address so I encourage you to use
> LFC. Leases File Cleanup (LFC) will keep your csv file clean and you
> wont drown in millions of old leases. Extend your config file with
> "lfc-interval"
> 
> e.g.
>   },
>   "lease-database": {
>     "type": "memfile",
>     "lfc-interval": 2000
>   },
> 
> All the information about .csv file you will find here:
> http://kea.isc.org/docs/kea-guide.html#idp51897760
> 
> Włodek Wencel
> ISC QA team
> 
> On 10/26/2016 05:47 PM, Munroe Sollog wrote:
>> here is the full log of kea attempting to start I also gzipp'ed and uploaded my leases file to:
>> https://ufile.io/f2b9
>>
>>
>> Oct 26 11:40:01 rogi systemd[1]: Starting ISC KEA IPv4 DHCP daemon...
>> Oct 26 11:40:01 rogi systemd[1]: Started ISC KEA IPv4 DHCP daemon.
>> Oct 26 11:40:01 rogi kea-dhcp4[20635]: 2016-10-26 11:40:01.720 INFO  [kea-dhcp4.dhcp4/20635]
>> DHCP4_STARTING Kea DHCPv4 server version 1.0.0 starting
>> Oct 26 11:40:01 rogi kea-dhcp4[20635]: 2016-10-26 11:40:01.722 INFO  [kea-dhcp4.dhcpsrv/20635]
>> DHCPSRV_CFGMGR_ADD_IFACE listening on interface eth0
>> Oct 26 11:40:01 rogi kea-dhcp4[20635]: 2016-10-26 11:40:01.722 INFO  [kea-dhcp4.dhcpsrv/20635]
>> DHCPSRV_CFGMGR_SOCKET_TYPE_DEFAULT "dhcp-socket-type" not specified , using default socket type raw
>> Oct 26 11:40:01 rogi kea-dhcp4[20635]: 2016-10-26 11:40:01.723 INFO  [kea-dhcp4.dhcp4/20635]
>> DHCP4_CONFIG_NEW_SUBNET a new subnet has been added to configuration: 172.31.0.0/18 with params:
>> valid-lifetime=4000
>> Oct 26 11:40:01 rogi kea-dhcp4[20635]: 2016-10-26 11:40:01.724 INFO  [kea-dhcp4.commands/20635]
>> COMMAND_SOCKET_UNIX_OPEN Command socket opened: UNIX, fd=6, path=/var/kea/statistics-socket
>> Oct 26 11:40:01 rogi kea-dhcp4[20635]: 2016-10-26 11:40:01.724 INFO  [kea-dhcp4.dhcpsrv/20635]
>> DHCPSRV_MEMFILE_DB opening memory file lease database: type=memfile universe=4
>> Oct 26 11:40:01 rogi kea-dhcp4[20635]: 2016-10-26 11:40:01.724 INFO  [kea-dhcp4.dhcpsrv/20635]
>> DHCPSRV_MEMFILE_LEASE_FILE_LOAD loading leases from file /var/lib/kea/kea-leases4.csv
>> Oct 26 11:40:09 rogi kea-dhcp4[20635]: 2016-10-26 11:40:09.311 ERROR [kea-dhcp4.dhcp4/20635]
>> DHCP4_PARSER_FAIL failed to create or run parser for configuration element lease-database: exceeded
>> maximum number of failures 100 to read a lease from the lease file /var/lib/kea/kea-leases4.csv
>> Oct 26 11:40:09 rogi kea-dhcp4[20635]: 2016-10-26 11:40:09.311 ERROR [kea-dhcp4.dhcp4/20635]
>> DHCP4_CONFIG_LOAD_FAIL configuration error using file: /etc/kea/kea-dhcp4.conf, reason: exceeded
>> maximum number of failures 100 to read a lease from the lease file /var/lib/kea/kea-leases4.csv
>> Oct 26 11:40:09 rogi kea-dhcp4[20635]: 2016-10-26 11:40:09.311 ERROR [kea-dhcp4.dhcp4/20635]
>> DHCP4_INIT_FAIL failed to initialize Kea server: configuration error using file
>> '/etc/kea/kea-dhcp4.conf': exceeded maximum number of failures 100 to read a lease from the lease
>> file /var/lib/kea/kea-leases4.csv
>> Oct 26 11:40:09 rogi kea-dhcp4[20635]: 2016-10-26 11:40:09.311 INFO  [kea-dhcp4.commands/20635]
>> COMMAND_SOCKET_UNIX_CLOSE Command socket closed: UNIX, fd=6, path=/var/kea/statistics-socket
>> Oct 26 11:40:09 rogi systemd[1]: kea-dhcp4-server.service: main process exited, code=exited,
>> status=1/FAILURE
>> Oct 26 11:40:09 rogi systemd[1]: Unit kea-dhcp4-server.service entered failed state.
>>
>>
>> On 10/25/2016 05:37 PM, Wlodek Wencel wrote:
>>> text files like kea_leases.csv reach really good compression ratio. It
>>> shouldn't reach more than 5Mb, but it is still to much for mailing list
>>> so if you could upload it somewhere.
>>>
>>> Or change logging severity to DEBUG with debug level 99 and see if there
>>> are additional logs.
>>>
>>> Regards,
>>> Włodek Wencel
>>>
>>> On 10/25/2016 11:13 PM, Munroe Sollog wrote:
>>>> below is my config.  My leases file is ~350MB so can't easily attach that to an email.
>>>>
>>>>
>>>> {
>>>> "Dhcp4":
>>>> {
>>>>     "control-socket": {
>>>> 	"socket-type": "unix",
>>>> 	"socket-name": "/var/kea/statistics-socket"
>>>> },
>>>>   "interfaces-config": {
>>>>     "interfaces": ["eth0" ]
>>>>   },
>>>>   "lease-database": {
>>>>     "type": "memfile"
>>>>   },
>>>>   "expired-leases-processing": {
>>>>     "reclaim-timer-wait-time": 10,
>>>>     "flush-reclaimed-timer-wait-time": 25,
>>>>     "hold-reclaimed-time": 3600,
>>>>     "max-reclaim-leases": 100,
>>>>     "max-reclaim-time": 250,
>>>>     "unwarned-reclaim-cycles": 5
>>>>   },
>>>>   "valid-lifetime": 4000,
>>>>     "subnet4": [
>>>> 	{
>>>> 	    "subnet": "172.31.0.0/18",
>>>> 	    "pools": [ { "pool": "172.31.1.1 - 172.31.31.254"} ],
>>>> 	    "option-data": [
>>>>                 {
>>>> 		    "name": "domain-name-servers",
>>>> 		    "data": "8.8.8.8, 8.8.4.4"
>>>> 		},
>>>> 		{
>>>> 		    "name": "routers",
>>>> 		    "data": "172.31.0.1"
>>>> 		},
>>>> 	    ]
>>>> 	}
>>>> 	
>>>>     ]
>>>> },
>>>>
>>>>     "Logging":
>>>>     {
>>>> 	"loggers": [
>>>> 	    {
>>>> 		"name": "kea-dhcp4",
>>>> 		"output_options": [
>>>> 		    {
>>>> 			"output": "/var/log/kea-dhcp4.log"
>>>> 		    }
>>>> 		],
>>>> 		"severity": "INFO",
>>>> 		"debuglevel": 0
>>>> 	    },
>>>> 	]
>>>>     }
>>>> }
>>>>
>>>>
>>>> On 10/25/2016 05:10 PM, Wlodek Wencel wrote:
>>>>> Hello,
>>>>> thanks for reporting that issue, I never came across this kind of
>>>>> problem. Is there a possibility that you could send us leases file and
>>>>> kea config file?
>>>>>
>>>>> Regards,
>>>>> Włodek Wencel
>>>>> QA team
>>>>>
>>>>> On 10/25/2016 11:05 PM, Todd Simmons (todsimmo) wrote:
>>>>>> Understood, in my case it was caused by a power outage. I have 14 sites running Kea 1.0.0 and that's the only time it's happened.  
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>>> On Oct 25, 2016, at 4:01 PM, Munroe Sollog <mus3 at lehigh.edu> wrote:
>>>>>>>
>>>>>>> I agree that the file is 'damaged' in some way, and of course if I delete all of the leases I expect
>>>>>>> it to start fine, but I'd like to figure out the root cause before I just blow the file away.
>>>>>>>
>>>>>>> - Munroe
>>>>>>>
>>>>>>>> On 10/25/2016 04:59 PM, Todd Simmons (todsimmo) wrote:
>>>>>>>> I had the same issue after the vm server was shutdown improperly after a power outage. I opened the file in VI and deleted all the leases (after saving a copy of course) and then it worked just fine. 
>>>>>>>>
>>>>>>>> It's possible the file is damaged and that's causing the problem. 
>>>>>>>>
>>>>>>>> Sent from my iPhone
>>>>>>>>
>>>>>>>>> On Oct 25, 2016, at 3:44 PM, Munroe Sollog <mus3 at lehigh.edu> wrote:
>>>>>>>>>
>>>>>>>>> running 1.0.0 code.  I stopped the service a while and then when I went to start it I got the
>>>>>>>>> following error:
>>>>>>>>>
>>>>>>>>> kea-dhcp4[14193]: 2016-10-25 16:35:28.129 ERROR [kea-dhcp4.dhcp4/14193] DHCP4_INIT_FAIL failed to
>>>>>>>>> initialize Kea server: configuration error using file '/etc/kea/kea-dhcp4.conf': exceeded maximum
>>>>>>>>> number of failures 100 to read a lease from the lease file /var/lib/kea/kea-leases4.csv
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> # kea-dhcp4 -V
>>>>>>>>> 1.0.0
>>>>>>>>> tarball
>>>>>>>>> linked with:
>>>>>>>>> log4plus 1.0.4
>>>>>>>>> OpenSSL 1.0.1t  3 May 2016
>>>>>>>>> database:
>>>>>>>>> MySQL backend 3.0, library 5.5.52
>>>>>>>>> PostgreSQL backend 2.0, library 90409
>>>>>>>>> Memfile backend 2.0
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> A quick google search yeilded the comments from the header file that causes the error output, but I
>>>>>>>>> don't understand why it can't read its lease file all of a sudden.  I don't know if it is pertinent,
>>>>>>>>> but there *a lot* of leases in the lease file:
>>>>>>>>>
>>>>>>>>> # wc -l /var/lib/kea/kea-leases4.csv
>>>>>>>>> 4485457 /var/lib/kea/kea-leases4.csv
>>>>>>>>>
>>>>>>>>> I obviously don't have 4.4million users though.
>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> Munroe Sollog
>>>>>>>>> LTS - Senior Network Engineer
>>>>>>>>> x85002
>>>>>>>>> _______________________________________________
>>>>>>>>> Kea-users mailing list
>>>>>>>>> Kea-users at lists.isc.org
>>>>>>>>> https://lists.isc.org/mailman/listinfo/kea-users
>>>>>>>> _______________________________________________
>>>>>>>> Kea-users mailing list
>>>>>>>> Kea-users at lists.isc.org
>>>>>>>> https://lists.isc.org/mailman/listinfo/kea-users
>>>>>>>>
>>>>>>>
>>>>>>> -- 
>>>>>>> Munroe Sollog
>>>>>>> LTS - Senior Network Engineer
>>>>>>> x85002
>>>>>> _______________________________________________
>>>>>> Kea-users mailing list
>>>>>> Kea-users at lists.isc.org
>>>>>> https://lists.isc.org/mailman/listinfo/kea-users
>>>>>>
>>>>> _______________________________________________
>>>>> Kea-users mailing list
>>>>> Kea-users at lists.isc.org
>>>>> https://lists.isc.org/mailman/listinfo/kea-users
>>>>>
>>>>
>>
> _______________________________________________
> Kea-users mailing list
> Kea-users at lists.isc.org
> https://lists.isc.org/mailman/listinfo/kea-users
> 



More information about the Kea-users mailing list