[Kea-users] High memory utilization for Kea server / Stork agent
mxhajduczenia at gmail.com
mxhajduczenia at gmail.com
Tue May 14 14:21:08 UTC 2024
This is what I see on node1 over the last 24 hours – yesterday around 10:45 am, I was doing configuration changes on dhcpv6 side, so the process would be restarted a few times, it would run, ramping up memory utilization is seems, and then got restarted again and has been running since. No changes were made to dhcpv4 process or stork agent
I am happy to share the config file(s) with you to see whether there is any issue with them and any potential ways of improving memory utilization.
Regards
Marek
From: mxhajduczenia at gmail.com <mxhajduczenia at gmail.com>
Sent: Tuesday, May 14, 2024 6:47 AM
To: 'Darren Ankney' <darren.ankney at gmail.com>
Cc: 'Kea user's list' <kea-users at lists.isc.org>; stork-users at lists.isc.org
Subject: RE: [Kea-users] High memory utilization for Kea server / Stork agent
Also, adding stork-focused email distro, though I suspect the membership is largely overlapping 😊
Regards
Marek
From: mxhajduczenia at gmail.com <mailto:mxhajduczenia at gmail.com> <mxhajduczenia at gmail.com <mailto:mxhajduczenia at gmail.com> >
Sent: Tuesday, May 14, 2024 5:55 AM
To: 'Darren Ankney' <darren.ankney at gmail.com <mailto:darren.ankney at gmail.com> >; 'Kea user's list' <kea-users at lists.isc.org <mailto:kea-users at lists.isc.org> >
Subject: RE: [Kea-users] High memory utilization for Kea server / Stork agent
Good morning, Darren
These were taken from node2, which is a backup node. And no, I have not done anything to the processes, not even restarted them over the course of a last week or so. I looked through the logs and I do not see any explanation for the behavior.
Node1 is where I constantly change the config (it is the lab after all) and the process is restarted over and over again (“service xxx restart” from shell) but even there I would not expect a process hanging for any reason
Marek
From: Darren Ankney <darren.ankney at gmail.com <mailto:darren.ankney at gmail.com> >
Sent: Tuesday, May 14, 2024 5:28 AM
To: mxhajduczenia at gmail.com <mailto:mxhajduczenia at gmail.com> ; Kea user's list <kea-users at lists.isc.org <mailto:kea-users at lists.isc.org> >
Subject: Re: [Kea-users] High memory utilization for Kea server / Stork agent
Hi Marek,
It appears that kea-dhcp4. kea-dhcp6, and stork-agent were started multiple times. I do not know how you are starting the daemons but perhaps something changed with the startup process (systemd?) that caused the thing that is starting the daemons to not realize they are running, thusly starting them multiple times?
Thank you,
Darren Ankney
On Mon, May 13, 2024 at 2:16 PM Marek Hajduczenia <mxhajduczenia at gmail.com <mailto:mxhajduczenia at gmail.com> > wrote:
Looking at the historic utilization over the week
you can clearly see a rather dramatic increase in memory utilization starting 5/10, with no specific changes to what the node is doing.
Marek
From: Marek Hajduczenia <mxhajduczenia at gmail.com <mailto:mxhajduczenia at gmail.com> >
Sent: Monday, May 13, 2024 12:12 PM
To: 'Kea user's list' <kea-users at lists.isc.org <mailto:kea-users at lists.isc.org> >
Subject: High memory utilization for Kea server / Stork agent
Dear colleagues,
I have been observing very high memory utilization for Kea VMs. In ~4 days, the memory utilization has grown from ~2 GB to close to 8GB for no good reason.
I seem to be running multiple copies of processes for some reason, with stork-agent likely being the primary culprit, and Kea DHCP servers being likely the second culprit. I have the very same situation on both lab nodes. I have to restart the server / stork agent periodically to eliminate the memory bloat.
634 stork-age 20 0 1866M 26240 15824 S 0.0 0.3 7:35.77 /usr/bin/stork-agent
709 stork-age 20 0 1866M 26240 15824 S 0.0 0.3 0:18.34 /usr/bin/stork-agent
710 stork-age 20 0 1866M 26240 15824 S 0.0 0.3 0:34.27 /usr/bin/stork-agent
711 stork-age 20 0 1866M 26240 15824 S 0.0 0.3 0:00.00 /usr/bin/stork-agent
713 stork-age 20 0 1866M 26240 15824 S 0.0 0.3 0:00.01 /usr/bin/stork-agent
727 stork-age 20 0 1866M 26240 15824 S 0.0 0.3 0:00.00 /usr/bin/stork-agent
740 stork-age 20 0 1866M 26240 15824 S 0.0 0.3 1:29.63 /usr/bin/stork-agent
894 stork-age 20 0 1866M 26240 15824 S 0.0 0.3 1:24.96 /usr/bin/stork-agent
1921 stork-age 20 0 1866M 26240 15824 S 0.0 0.3 1:29.49 /usr/bin/stork-agent
15986 stork-age 20 0 1866M 26240 15824 S 0.0 0.3 0:53.26 /usr/bin/stork-agent
1340 _kea 20 0 507M 23864 16784 S 0.0 0.3 4:54.19 /usr/sbin/kea-dhcp6 -c /etc/kea/kea-dhcp6.conf
1341 _kea 20 0 507M 23864 16784 S 0.0 0.3 0:00.00 /usr/sbin/kea-dhcp6 -c /etc/kea/kea-dhcp6.conf
1342 _kea 20 0 507M 23864 16784 S 0.0 0.3 0:00.00 /usr/sbin/kea-dhcp6 -c /etc/kea/kea-dhcp6.conf
1343 _kea 20 0 507M 23864 16784 S 0.0 0.3 0:00.00 /usr/sbin/kea-dhcp6 -c /etc/kea/kea-dhcp6.conf
1344 _kea 20 0 507M 23864 16784 S 0.0 0.3 0:00.00 /usr/sbin/kea-dhcp6 -c /etc/kea/kea-dhcp6.conf
1352 _kea 20 0 507M 23864 16784 S 0.0 0.3 0:05.43 /usr/sbin/kea-dhcp6 -c /etc/kea/kea-dhcp6.conf
1353 _kea 20 0 507M 23864 16784 S 0.0 0.3 0:05.49 /usr/sbin/kea-dhcp6 -c /etc/kea/kea-dhcp6.conf
1354 _kea 20 0 507M 23864 16784 S 0.0 0.3 0:08.22 /usr/sbin/kea-dhcp6 -c /etc/kea/kea-dhcp6.conf
1355 _kea 20 0 507M 23864 16784 S 0.0 0.3 0:08.25 /usr/sbin/kea-dhcp6 -c /etc/kea/kea-dhcp6.conf
20544 _kea 20 0 507M 23864 16784 S 0.0 0.3 0:00.00 /usr/sbin/kea-dhcp6 -c /etc/kea/kea-dhcp6.conf
20545 _kea 20 0 507M 23864 16784 S 0.0 0.3 0:00.00 /usr/sbin/kea-dhcp6 -c /etc/kea/kea-dhcp6.conf
679 root 20 0 107M 20736 12596 S 0.0 0.3 0:00.04 /usr/bin/python3 /usr/share/unattended-upgrades/unattended-upgrade-shutdown --wait-for-signal
732 root 20 0 107M 20736 12596 S 0.0 0.3 0:00.00 /usr/bin/python3 /usr/share/unattended-upgrades/unattended-upgrade-shutdown --wait-for-signal
1889 _kea 20 0 507M 19292 14552 S 0.0 0.2 3:05.41 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
1890 _kea 20 0 507M 19292 14552 S 0.0 0.2 0:00.00 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
1891 _kea 20 0 507M 19292 14552 S 0.0 0.2 0:00.00 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
1892 _kea 20 0 507M 19292 14552 S 0.0 0.2 0:00.00 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
1893 _kea 20 0 507M 19292 14552 S 0.0 0.2 0:00.00 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
1897 _kea 20 0 507M 19292 14552 S 0.0 0.2 0:05.47 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
1898 _kea 20 0 507M 19292 14552 S 0.0 0.2 0:05.15 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
1899 _kea 20 0 507M 19292 14552 S 0.0 0.2 0:10.02 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
1900 _kea 20 0 507M 19292 14552 S 0.0 0.2 0:09.95 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
20586 _kea 20 0 507M 19292 14552 S 0.0 0.2 0:00.09 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
20587 _kea 20 0 507M 19292 14552 S 0.0 0.2 0:00.08 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
Regards
Marek
--
ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.
To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users.
Kea-users mailing list
Kea-users at lists.isc.org <mailto:Kea-users at lists.isc.org>
https://lists.isc.org/mailman/listinfo/kea-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.isc.org/pipermail/kea-users/attachments/20240514/aa764b8f/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 26467 bytes
Desc: not available
URL: <https://lists.isc.org/pipermail/kea-users/attachments/20240514/aa764b8f/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.png
Type: image/png
Size: 11974 bytes
Desc: not available
URL: <https://lists.isc.org/pipermail/kea-users/attachments/20240514/aa764b8f/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.png
Type: image/png
Size: 7776 bytes
Desc: not available
URL: <https://lists.isc.org/pipermail/kea-users/attachments/20240514/aa764b8f/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image007.png
Type: image/png
Size: 12762 bytes
Desc: not available
URL: <https://lists.isc.org/pipermail/kea-users/attachments/20240514/aa764b8f/attachment-0007.png>
More information about the Kea-users
mailing list