[Kea-users] Fwd: Memory leak on dhcp6

Marcin Siodelski marcin at isc.org
Mon May 6 15:56:05 UTC 2019


Hi Nitzan,

We're trying to reproduce this issue using our test tools. If the leak
is related to the traffic volume processed by the server, it would be
useful for us to know the average number of packets/second that your
primary server is receiving.

You say that the memory consumption grows to 28% within an hour but I
don't know how many packets the server has received and processed to
reach that level. If you can provide the average DHCP traffic rate we
can craft the test that best mirrors your situation and confirm whether
or not we see the same thing.

Thanks in advance,

Marcin Siodelski
DHCP Software Engineer,
ISC

On 05/05/2019 16:29, Nitzan Tzelniker wrote:
> Hi,
> 
> It seems that the issue is related to the HA
> After I commented the HA part the memory consumption is stayed low 
> I am adding the relevant configuration if you have any idea
> 
> Thanks
> 
> Nitzan
> 
> Standby server ha configuration part: 
> 
>     "hooks-libraries": [
>         {
>             "library": "/usr/local/lib/hooks/libdhcp_lease_cmds.so",
>             "parameters": { }
>         },
>         {
>             "library": "/usr/local/lib/hooks/libdhcp_ha.so",
>             "parameters": {
>                 "high-availability": [ {
>                     "this-server-name": "KEA_DHCPv6_2",
>                     "mode": "hot-standby",
>                     "heartbeat-delay": 10000,
>                     "max-response-delay": 20000,
>                     "max-ack-delay": 10000,
>                     "max-unacked-clients": 0,
>                     "peers": [
>                         {
>                             "name": "KEA_DHCPv6_1",
>                             "role": "primary",
>                             "auto-failover": true
>                         },
>                         {
>                             "name": "KEA_DHCPv6_2",
>                             "role": "standby",
>                             "auto-failover": true
>                         }
>                     ]
>                 } ]
>             }
>         }
>     ]
> }
> 
> kea-ctrl-agent.conf  from both servers (Only the ip address is different ) 
> 
> {
> 
> "Control-agent": {
>     "http-host": "1.1.1.X",
>     "http-port": 8080,
> 
>     "control-sockets": {
>         "dhcp4": {
>             "socket-type": "unix",
>             "socket-name": "/tmp/kea-dhcp4-ctrl.sock"
>         },
>         "dhcp6": {
>             "socket-type": "unix",
>             "socket-name": "/tmp/kea-dhcp6-ctrl.sock"
>         }
>     },
> 
>     "hooks-libraries": [
>     ]
> },
> 
> "Logging":
> {
>   "loggers": [
>     {
>         "name": "kea-ctrl-agent",
>         "output_options": [
>             {
>                 "output": "/usr/local/var/log/kea-ctrl-agent.log"
>             }
>         ],
>         "severity": "INFO",
> 
>         "debuglevel": 0
>     }
>   ]
> }
> }
> ---------- Forwarded message ---------
> From: *Nitzan Tzelniker* <nitzan.tzelniker at gmail.com
> <mailto:nitzan.tzelniker at gmail.com>>
> Date: Fri, May 3, 2019 at 10:13 PM
> Subject: Memory leak on dhcp6
> To: <kea-users at lists.isc.org <mailto:kea-users at lists.isc.org>>
> 
> 
> Hi,
> 
> I am running two dhcpv6 server with high-availability
> We have about 2500 regular leases and 2500 PD leases
> The setup is working but it look like kea-dhcp6 leak memory 
> When it start (with all of the leases in the file ) it take less then 1%
> of the memory but after two hours it take 28% and if I will give it 8
> hours it will crash 
> 
> I don't see issue on the standby server only on the primary server
> 
> Anybody saw this behavior  
> Any idea how to debug it
> 
>  kea-dhcp6 -V
> 1.5.0
> tarball
> linked with:
> log4cplus 1.1.3
> OpenSSL 1.0.2k-fips  26 Jan 2017
> database:
> Memfile backend 2.1
> 
> Running on CentOS 7.6.1810 VM kernel 3.10.0-957.1.3.el7.x86 with 2 *
> vCPU and 2GB RAM 
> 
> bellow is the dhcp6 config file omitting 102 subsets 
> 
> {
> "Dhcp6": {
>     "interfaces-config": {
>         "interfaces": [ "ens192/1111:aef:aa77:83::100" ]
>     },
>     "mac-sources": [ "docsis-cmts", "client-link-addr-option" ],
> 
>     "control-socket": {
>         "socket-type": "unix",
>         "socket-name": "/tmp/kea-dhcp6-ctrl.sock"
>     },
> 
>     "lease-database": {
>         "type": "memfile",
>         "lfc-interval": 3600
>     },
>     "expired-leases-processing": {
>         "reclaim-timer-wait-time": 10,
>         "flush-reclaimed-timer-wait-time": 25,
>         "hold-reclaimed-time": 3600,
>         "max-reclaim-leases": 100,
>         "max-reclaim-time": 250,
>         "unwarned-reclaim-cycles": 5
>     },
> 
>     "renew-timer": 1000,
>     "rebind-timer": 2000,
>     "preferred-lifetime": 3000,
>     "valid-lifetime": 4000,
>     "option-data": [
>         {
>             "name": "dns-servers",
>             "data": "1111:aef:aa77:1::22, 1111:aef:aa77:2::22"
>         }
>     ],
>     "subnet6": [
>         {
>             "subnet": "1111:aec:0:10::/64",
>             "pools": [ { "pool":
> "1111:aec:0:10::10-1111:aec:0:10:ffff::" } ],
>             "pd-pools": [
>                 {
>                     "prefix": "1111:aec:10::",
>                     "prefix-len": 44,
>                     "delegated-len": 56
>                 }
>             ]
>         },
> //....
> //  101 more subnets 
> //....
>  "hooks-libraries": [
>         {
>             "library": "/usr/local/lib/hooks/libdhcp_ha.so",
>             "parameters": {
>                 "high-availability": [ {
>                     "this-server-name": "KEA_DHCPv6_1",
>                     "mode": "hot-standby",
>                     "heartbeat-delay": 10000,
>                     "max-response-delay": 20000,
>                     "max-ack-delay": 10000,
>                     "max-unacked-clients": 0,
>                     "peers": [
>                         {
>                             "name": "KEA_DHCPv6_1",
>                             "role": "primary",
>                             "auto-failover": true
>                         },
>                         {
>                             "name": "KEA_DHCPv6_2",
>                             "role": "standby",
>                             "auto-failover": true
>                         }
>                     ]
>                 } ]
>             }
>         }
>     ]
> },
> 
> "Logging":
> {
>   "loggers": [
>     {
>         "name": "kea-dhcp6",
>         "output_options": [
>             {
>                 "output": "/usr/local/var/log/kea-dhcp6.log"
>             }
>         ],
>         "severity": "INFO",
> 
>         "debuglevel": 0
>     }
>   ]
> }
> }
> Thanks
> 
> Nitzan
> 
> _______________________________________________
> Kea-users mailing list
> Kea-users at lists.isc.org
> https://lists.isc.org/mailman/listinfo/kea-users
> 




More information about the Kea-users mailing list