<div dir="ltr">Yes Thanks Darren, found the issue after changing the file name it is working fine and do you have any idea like how can we get the failover status <br>In ISC DHCP failover we can see them in the lease file (communication interrupted, recover, partner-down....etc) with the failover name being specified but I don't see any such info in the kea-leas file <div><br></div><div>Thanks</div><div>Kraishak</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 6, 2023 at 4:23 PM Darren Ankney <<a href="mailto:darren.ankney@gmail.com">darren.ankney@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Looks like your socket locations don't match between the kea config<br>
and the ctrl agent configuration. Those are filenames, so they are<br>
going to need to match exactly. Kea is in the driver's seat there as<br>
it actually creates the file. the ctrl agent "talks" to kea through<br>
such file.<br>
<br>
example, your primary kea config says: /tmp/kea4-ctrl-socket1 and<br>
your ctrl agent config on the primary says: /tmp/control_socket<br>
<br>
On Mon, Mar 6, 2023 at 4:38 AM Kraishak Mahtha <<a href="mailto:kraishak.edu@gmail.com" target="_blank">kraishak.edu@gmail.com</a>> wrote:<br>
><br>
> Hi All,<br>
><br>
> I am trying to configure the kea-HA setup in my local environment but I am facing an issue the heartbeat is getting failed and the logs are as follows:<br>
> Logs on Failover<br>
> =============<br>
> 2023-03-06 09:04:43.761 DEBUG [kea-dhcp4.http/28550.140575742654656] HTTP_CLIENT_REQUEST_SEND sending HTTP request POST / HTTP/1.1 to <a href="http://192.168.0.168:8000/" rel="noreferrer" target="_blank">http://192.168.0.168:8000/</a><br>
> 2023-03-06 09:04:43.761 DEBUG [kea-dhcp4.http/28550.140575742654656] HTTP_CLIENT_REQUEST_SEND_DETAILS detailed information about request sent to <a href="http://192.168.0.168:8000/" rel="noreferrer" target="_blank">http://192.168.0.168:8000/</a>:<br>
> POST / HTTP/1.1<br>
> Host: 192.168.0.168<br>
> Content-Length: 53<br>
> Content-Type: application/json<br>
><br>
> { "command": "ha-heartbeat", "service": [ "dhcp4" ] }<br>
> 2023-03-06 09:04:43.762 DEBUG [kea-dhcp4.http/28550.140575742654656] HTTP_SERVER_RESPONSE_RECEIVED received HTTP response from <a href="http://192.168.0.168:8000/" rel="noreferrer" target="_blank">http://192.168.0.168:8000/</a><br>
> 2023-03-06 09:04:43.762 DEBUG [kea-dhcp4.http/28550.140575742654656] HTTP_SERVER_RESPONSE_RECEIVED_DETAILS detailed information about well-formed response received from <a href="http://192.168.0.168:8000/" rel="noreferrer" target="_blank">http://192.168.0.168:8000/</a>:<br>
> HTTP/1.1 200 OK<br>
> Content-Length: 133<br>
> Content-Type: application/json<br>
> Date: Mon, 06 Mar 2023 09:04:43 GMT<br>
><br>
> [ { "result": 1, "text": "unable to forward command to the dhcp4 service: Connection refused. The server is likely to be offline" } ]<br>
> 2023-03-06 09:04:43.762 WARN [kea-dhcp4.ha-hooks/28550.140575742654656] HA_HEARTBEAT_FAILED heartbeat to kea-dhcp-168 (<a href="http://192.168.0.168:8000/" rel="noreferrer" target="_blank">http://192.168.0.168:8000/</a>) failed: unable to forward command to the dhcp4 service: Connection refused. The server is likely to be offline, error code 1<br>
> 2023-03-06 09:04:43.762 WARN [kea-dhcp4.ha-hooks/28550.140575742654656] HA_COMMUNICATION_INTERRUPTED communication with kea-dhcp-168 is interrupted<br>
><br>
><br>
> Logs on Primary<br>
> ==============<br>
> 2023-03-06 09:01:50.143 DEBUG [kea-dhcp4.dhcpsrv/13620.140081380149440] DHCPSRV_TIMERMGR_START_TIMER starting timer: reclaim-expired-leases<br>
> 2023-03-06 09:01:52.145 DEBUG [kea-dhcp4.http/13620.140081380149440] HTTP_CLIENT_REQUEST_SEND sending HTTP request POST / HTTP/1.1 to <a href="http://192.168.0.169:8000/" rel="noreferrer" target="_blank">http://192.168.0.169:8000/</a><br>
> 2023-03-06 09:01:52.145 DEBUG [kea-dhcp4.http/13620.140081380149440] HTTP_CLIENT_REQUEST_SEND_DETAILS detailed information about request sent to <a href="http://192.168.0.169:8000/" rel="noreferrer" target="_blank">http://192.168.0.169:8000/</a>:<br>
> POST / HTTP/1.1<br>
> Host: 192.168.0.169<br>
> Content-Length: 53<br>
> Content-Type: application/json<br>
><br>
> { "command": "ha-heartbeat", "service": [ "dhcp4" ] }<br>
> 2023-03-06 09:01:52.146 DEBUG [kea-dhcp4.http/13620.140081380149440] HTTP_SERVER_RESPONSE_RECEIVED received HTTP response from <a href="http://192.168.0.169:8000/" rel="noreferrer" target="_blank">http://192.168.0.169:8000/</a><br>
> 2023-03-06 09:01:52.146 DEBUG [kea-dhcp4.http/13620.140081380149440] HTTP_SERVER_RESPONSE_RECEIVED_DETAILS detailed information about well-formed response received from <a href="http://192.168.0.169:8000/" rel="noreferrer" target="_blank">http://192.168.0.169:8000/</a>:<br>
> HTTP/1.1 200 OK<br>
> Content-Length: 133<br>
> Content-Type: application/json<br>
> Date: Mon, 06 Mar 2023 09:01:52 GMT<br>
><br>
> [ { "result": 1, "text": "unable to forward command to the dhcp4 service: Connection refused. The server is likely to be offline" } ]<br>
> 2023-03-06 09:01:52.146 WARN [kea-dhcp4.ha-hooks/13620.140081380149440] HA_HEARTBEAT_FAILED heartbeat to appliance-169 (<a href="http://192.168.0.169:8000/" rel="noreferrer" target="_blank">http://192.168.0.169:8000/</a>) failed: unable to forward command to the dhcp4 service: Connection refused. The server is likely to be offline, error code 1<br>
> 2023-03-06 09:01:53.558 INFO [kea-ctrl-agent.commands/13581.140159667833024] COMMAND_RECEIVED Received command 'ha-heartbeat'<br>
> 2023-03-06 09:01:53.558 INFO [kea-ctrl-agent.ctrl-agent/13581.140159667833024] CTRL_AGENT_COMMAND_RECEIVED command ha-heartbeat received from remote address 192.168.0.169<br>
><br>
><br>
> =========<br>
> Primary config<br>
> ==============<br>
> ##DHCPv4 192.168.0.168<br>
> {<br>
> "Dhcp4": {<br>
> "valid-lifetime": 300,<br>
> "renew-timer": 150,<br>
> "rebind-timer": 200,<br>
> "server-tag": "192.168.0.168",<br>
> "control-socket": {<br>
> "socket-name": "/tmp/kea4-ctrl-socket1",<br>
> "socket-type": "unix"<br>
> },<br>
><br>
> "lease-database": {<br>
> "type": "memfile",<br>
> "persist": true,<br>
> "name": "/var/lib/kea/dhcp4.leases"<br>
> },<br>
><br>
> "interfaces-config": {<br>
> "interfaces": ["eth0"]<br>
> },<br>
><br>
> "hooks-libraries": [<br>
> {<br>
> "library": "/usr/local/lib/kea/hooks/libdhcp_lease_cmds.so",<br>
> "parameters": { }<br>
> },<br>
> {<br>
> "library": "/usr/local/lib/kea/hooks/libdhcp_bootp.so",<br>
> "parameters": { }<br>
> },<br>
> {<br>
> "library": "/usr/local/lib/kea/hooks/libdhcp_ha.so",<br>
> "parameters": {<br>
> "high-availability": [{<br>
> "this-server-name": "kea-dhcp-168",<br>
> "mode": "hot-standby",<br>
> "heartbeat-delay": 10000,<br>
> "max-response-delay": 60000,<br>
> "max-ack-delay": 5000,<br>
> "max-unacked-clients": 5,<br>
> "peers": [{<br>
> "name": "kea-dhcp-168",<br>
> "url": "<a href="http://192.168.0.168:8000/" rel="noreferrer" target="_blank">http://192.168.0.168:8000/</a>",<br>
> "role": "primary",<br>
> "auto-failover": true<br>
> }, {<br>
> "name": "appliance-169",<br>
> "url": "<a href="http://192.168.0.169:8000/" rel="noreferrer" target="_blank">http://192.168.0.169:8000/</a>",<br>
> "role": "standby",<br>
> "auto-failover": true<br>
> }]<br>
> }]<br>
> }<br>
> }],<br>
> "subnet4": [{<br>
> "subnet": "<a href="http://4.0.0.0/24" rel="noreferrer" target="_blank">4.0.0.0/24</a>",<br>
> "option-data": [<br>
> {<br>
> "data": "2.2.2.2, 8.8.4.4",<br>
> "name": "domain-name-servers"<br>
> },<br>
> {<br>
> "data": "4.0.0.1",<br>
> "name": "routers"<br>
> }<br>
> ],<br>
> "pools": [<br>
> {<br>
> "pool": "4.0.0.25 - 4.0.0.250"<br>
> }<br>
> ]<br>
><br>
> }],<br>
> "loggers": [<br>
> {<br>
> "name": "kea-dhcp4",<br>
> "debuglevel": 99,<br>
> "output_options": [<br>
> {<br>
> "output": "/usr/local/var/log/kea-dhcp4.log"<br>
> }<br>
> ],<br>
> "severity": "DEBUG"<br>
> }<br>
> ]<br>
> }<br>
> }<br>
> kea-ctrl-agent.conf on Primary<br>
> =========================<br>
><br>
> {<br>
> "Control-agent": {<br>
> "http-host": "192.168.0.168",<br>
> "http-port": 8000,<br>
> "control-sockets": {<br>
> "dhcp4": {<br>
> "socket-type": "unix",<br>
> "socket-name": "/tmp/control_socket"<br>
> }<br>
> },<br>
> "cert-required": false<br>
> }<br>
> }<br>
> Netstat output<br>
> =============<br>
> [root@kea-dhcp-168 ~]# netstat -tulpna | grep 8000<br>
> tcp 0 0 <a href="http://192.168.0.168:8000" rel="noreferrer" target="_blank">192.168.0.168:8000</a> 0.0.0.0:* LISTEN 13581/kea-ctrl-agen<br>
> tcp 0 0 <a href="http://192.168.0.168:57308" rel="noreferrer" target="_blank">192.168.0.168:57308</a> <a href="http://192.168.0.169:8000" rel="noreferrer" target="_blank">192.168.0.169:8000</a> ESTABLISHED 13620/kea-dhcp4<br>
> tcp 0 0 <a href="http://192.168.0.168:8000" rel="noreferrer" target="_blank">192.168.0.168:8000</a> <a href="http://192.168.0.169:58432" rel="noreferrer" target="_blank">192.168.0.169:58432</a> ESTABLISHED 13581/kea-ctrl-agen<br>
> [root@kea-dhcp-168 ~]#<br>
><br>
> [root@kea-dhcp-168 ~]# netstat -tulpna | grep 67<br>
> udp 0 0 <a href="http://192.168.0.168:67" rel="noreferrer" target="_blank">192.168.0.168:67</a> 0.0.0.0:* 13620/kea-dhcp4<br>
> [root@kea-dhcp-168 ~]#<br>
><br>
><br>
> Failover DHCPv4 config<br>
> ===================<br>
> #DHCPv4 192.168.0.169<br>
> {<br>
> "Dhcp4": {<br>
> "valid-lifetime": 300,<br>
> "renew-timer": 150,<br>
> "rebind-timer": 200,<br>
> "server-tag": "192.168.0.169",<br>
> "control-socket": {<br>
> "socket-name": "/opt/testUAT/keaFolder/kea4-ctrl-socket",<br>
> "socket-type": "unix"<br>
> },<br>
><br>
> "lease-database": {<br>
> "type": "memfile",<br>
> "persist": true,<br>
> "name": "/opt/testUAT/var/lib/kea/dhcp4.leases"<br>
> },<br>
><br>
> "interfaces-config": {<br>
> "interfaces": ["eth0"]<br>
> },<br>
><br>
> "hooks-libraries": [<br>
> {<br>
> "library": "/opt/testUAT/lib/kea/hooks/libdhcp_lease_cmds.so",<br>
> "parameters": { }<br>
> },<br>
> {<br>
> "library": "/opt/testUAT/lib/kea/hooks/libdhcp_bootp.so",<br>
> "parameters": { }<br>
> },<br>
> {<br>
> "library": "/opt/testUAT/lib/kea/hooks/libdhcp_ha.so",<br>
> "parameters": {<br>
> "high-availability": [{<br>
> "this-server-name": "appliance-169",<br>
> "mode": "hot-standby",<br>
> "heartbeat-delay": 10000,<br>
> "max-response-delay": 60000,<br>
> "max-ack-delay": 5000,<br>
> "max-unacked-clients": 5,<br>
> "peers": [{<br>
> "name": "kea-dhcp-168",<br>
> "url": "<a href="http://192.168.0.168:8000/" rel="noreferrer" target="_blank">http://192.168.0.168:8000/</a>",<br>
> "role": "primary",<br>
> "auto-failover": true<br>
> }, {<br>
> "name": "appliance-169",<br>
> "url": "<a href="http://192.168.0.169:8000/" rel="noreferrer" target="_blank">http://192.168.0.169:8000/</a>",<br>
> "role": "standby",<br>
> "auto-failover": true<br>
> }]<br>
> }]<br>
> }<br>
> }],<br>
> "subnet4": [{<br>
> "subnet": "<a href="http://4.0.0.0/24" rel="noreferrer" target="_blank">4.0.0.0/24</a>",<br>
> "option-data": [<br>
> {<br>
> "data": "2.2.2.2, 8.8.4.4",<br>
> "name": "domain-name-servers"<br>
> },<br>
> {<br>
> "data": "4.0.0.1",<br>
> "name": "routers"<br>
> }<br>
> ],<br>
> "pools": [<br>
> {<br>
> "pool": "4.0.0.25 - 4.0.0.250"<br>
> }<br>
> ]<br>
><br>
> }],<br>
> "loggers": [<br>
> {<br>
> "name": "kea-dhcp4",<br>
> "debuglevel": 99,<br>
> "output_options": [<br>
> {<br>
> "output": "/opt/testUAT/var/log/kea-dhcp4.log"<br>
> }<br>
> ],<br>
> "severity": "DEBUG"<br>
> }<br>
> ]<br>
> }<br>
> }<br>
><br>
> kea-ctrl-agent.conf on Failover<br>
> =======================<br>
> {<br>
> "Control-agent": {<br>
> "http-host": "192.168.0.169",<br>
> "http-port": 8000,<br>
> "control-sockets": {<br>
> "dhcp4": {<br>
> "socket-type": "unix",<br>
> "socket-name": "/tmp/control_socket"<br>
> }<br>
> },<br>
> "cert-required": false<br>
> }<br>
> }<br>
> Netstat output<br>
> =============<br>
><br>
> [root@appliance-169 ~]# netstat -tulpna | grep 8000<br>
> tcp 0 0 <a href="http://192.168.0.169:8000" rel="noreferrer" target="_blank">192.168.0.169:8000</a> 0.0.0.0:* LISTEN 28520/kea-ctrl-agen<br>
> tcp 0 0 <a href="http://192.168.0.169:8000" rel="noreferrer" target="_blank">192.168.0.169:8000</a> <a href="http://192.168.0.168:57308" rel="noreferrer" target="_blank">192.168.0.168:57308</a> ESTABLISHED 28520/kea-ctrl-agen<br>
> tcp 0 0 <a href="http://192.168.0.169:58432" rel="noreferrer" target="_blank">192.168.0.169:58432</a> <a href="http://192.168.0.168:8000" rel="noreferrer" target="_blank">192.168.0.168:8000</a> ESTABLISHED 28550/kea-dhcp4<br>
> [root@appliance-169 ~]#<br>
><br>
> [root@kea-dhcp-168 ~]# netstat -tulpna | grep 67<br>
> udp 0 0 <a href="http://192.168.0.169:67" rel="noreferrer" target="_blank">192.168.0.169:67</a> 0.0.0.0:* 28550/kea-dhcp4<br>
> [root@kea-dhcp-168 ~]#<br>
><br>
> I am not sure why it is unable to forward commands to dhcp4 service, Can someone guide or help me if you face the same issue or are aware of the issue?<br>
><br>
> Thanks in advance<br>
> Kraishak<br>
><br>
> --<br>
> ISC funds the development of this software with paid support subscriptions. Contact us at <a href="https://www.isc.org/contact/" rel="noreferrer" target="_blank">https://www.isc.org/contact/</a> for more information.<br>
><br>
> To unsubscribe visit <a href="https://lists.isc.org/mailman/listinfo/kea-users" rel="noreferrer" target="_blank">https://lists.isc.org/mailman/listinfo/kea-users</a>.<br>
><br>
> Kea-users mailing list<br>
> <a href="mailto:Kea-users@lists.isc.org" target="_blank">Kea-users@lists.isc.org</a><br>
> <a href="https://lists.isc.org/mailman/listinfo/kea-users" rel="noreferrer" target="_blank">https://lists.isc.org/mailman/listinfo/kea-users</a><br>
-- <br>
ISC funds the development of this software with paid support subscriptions. Contact us at <a href="https://www.isc.org/contact/" rel="noreferrer" target="_blank">https://www.isc.org/contact/</a> for more information.<br>
<br>
To unsubscribe visit <a href="https://lists.isc.org/mailman/listinfo/kea-users" rel="noreferrer" target="_blank">https://lists.isc.org/mailman/listinfo/kea-users</a>.<br>
<br>
Kea-users mailing list<br>
<a href="mailto:Kea-users@lists.isc.org" target="_blank">Kea-users@lists.isc.org</a><br>
<a href="https://lists.isc.org/mailman/listinfo/kea-users" rel="noreferrer" target="_blank">https://lists.isc.org/mailman/listinfo/kea-users</a><br>
</blockquote></div>