<div dir="ltr"><font color="#888888"><div> Hi All,<br><br>I am trying to configure the kea-HA setup in my local environment but I am facing an issue the heartbeat is getting failed and the logs are as follows:<br>Logs on Failover<br>=============<br>2023-03-06 09:04:43.761 DEBUG [kea-dhcp4.http/28550.140575742654656] HTTP_CLIENT_REQUEST_SEND sending HTTP request POST / HTTP/1.1 to <a href="http://192.168.0.168:8000/">http://192.168.0.168:8000/</a><br>2023-03-06 09:04:43.761 DEBUG [kea-dhcp4.http/28550.140575742654656] HTTP_CLIENT_REQUEST_SEND_DETAILS detailed information about request sent to <a href="http://192.168.0.168:8000/">http://192.168.0.168:8000/</a>:<br>POST / HTTP/1.1<br>Host: 192.168.0.168<br>Content-Length: 53<br>Content-Type: application/json<br><br>{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }<br>2023-03-06 09:04:43.762 DEBUG [kea-dhcp4.http/28550.140575742654656] HTTP_SERVER_RESPONSE_RECEIVED received HTTP response from <a href="http://192.168.0.168:8000/">http://192.168.0.168:8000/</a><br>2023-03-06 09:04:43.762 DEBUG [kea-dhcp4.http/28550.140575742654656] HTTP_SERVER_RESPONSE_RECEIVED_DETAILS detailed information about well-formed response received from <a href="http://192.168.0.168:8000/">http://192.168.0.168:8000/</a>:<br>HTTP/1.1 200 OK<br>Content-Length: 133<br>Content-Type: application/json<br>Date: Mon, 06 Mar 2023 09:04:43 GMT<br><br>[ { "result": 1, "text": "unable to forward command to the dhcp4 service: Connection refused. The server is likely to be offline" } ]<br>2023-03-06 09:04:43.762 WARN  [kea-dhcp4.ha-hooks/28550.140575742654656] HA_HEARTBEAT_FAILED heartbeat to kea-dhcp-168 (<a href="http://192.168.0.168:8000/">http://192.168.0.168:8000/</a>) failed: unable to forward command to the dhcp4 service: Connection refused. The server is likely to be offline, error code 1<br>2023-03-06 09:04:43.762 WARN  [kea-dhcp4.ha-hooks/28550.140575742654656] HA_COMMUNICATION_INTERRUPTED communication with kea-dhcp-168 is interrupted<br><br><br>Logs on Primary<br>==============<br>2023-03-06 09:01:50.143 DEBUG [kea-dhcp4.dhcpsrv/13620.140081380149440] DHCPSRV_TIMERMGR_START_TIMER starting timer: reclaim-expired-leases<br>2023-03-06 09:01:52.145 DEBUG [kea-dhcp4.http/13620.140081380149440] HTTP_CLIENT_REQUEST_SEND sending HTTP request POST / HTTP/1.1 to <a href="http://192.168.0.169:8000/">http://192.168.0.169:8000/</a><br>2023-03-06 09:01:52.145 DEBUG [kea-dhcp4.http/13620.140081380149440] HTTP_CLIENT_REQUEST_SEND_DETAILS detailed information about request sent to <a href="http://192.168.0.169:8000/">http://192.168.0.169:8000/</a>:<br>POST / HTTP/1.1<br>Host: 192.168.0.169<br>Content-Length: 53<br>Content-Type: application/json<br><br>{ "command": "ha-heartbeat", "service": [ "dhcp4" ] }<br>2023-03-06 09:01:52.146 DEBUG [kea-dhcp4.http/13620.140081380149440] HTTP_SERVER_RESPONSE_RECEIVED received HTTP response from <a href="http://192.168.0.169:8000/">http://192.168.0.169:8000/</a><br>2023-03-06 09:01:52.146 DEBUG [kea-dhcp4.http/13620.140081380149440] HTTP_SERVER_RESPONSE_RECEIVED_DETAILS detailed information about well-formed response received from <a href="http://192.168.0.169:8000/">http://192.168.0.169:8000/</a>:<br>HTTP/1.1 200 OK<br>Content-Length: 133<br>Content-Type: application/json<br>Date: Mon, 06 Mar 2023 09:01:52 GMT<br><br>[ { "result": 1, "text": "unable to forward command to the dhcp4 service: Connection refused. The server is likely to be offline" } ]<br>2023-03-06 09:01:52.146 WARN  [kea-dhcp4.ha-hooks/13620.140081380149440] HA_HEARTBEAT_FAILED heartbeat to appliance-169 (<a href="http://192.168.0.169:8000/">http://192.168.0.169:8000/</a>) failed: unable to forward command to the dhcp4 service: Connection refused. The server is likely to be offline, error code 1<br>2023-03-06 09:01:53.558 INFO  [kea-ctrl-agent.commands/13581.140159667833024] COMMAND_RECEIVED Received command 'ha-heartbeat'<br>2023-03-06 09:01:53.558 INFO  [kea-ctrl-agent.ctrl-agent/13581.140159667833024] CTRL_AGENT_COMMAND_RECEIVED command ha-heartbeat received from remote address 192.168.0.169<br><br><br>=========<br>Primary config<br>==============<br>##DHCPv4 192.168.0.168<br>{<br>"Dhcp4": {<br>    "valid-lifetime": 300,<br>    "renew-timer": 150,<br>    "rebind-timer": 200,<br>    "server-tag": "192.168.0.168",<br>      "control-socket": {<br>            "socket-name": "/tmp/kea4-ctrl-socket1",<br>            "socket-type": "unix"<br>        },<br><br>    "lease-database": {<br>        "type": "memfile",<br>        "persist": true,<br>        "name": "/var/lib/kea/dhcp4.leases"<br>    },<br><br>   "interfaces-config": {<br>        "interfaces": ["eth0"]<br>  },<br><br>"hooks-libraries": [<br>            {<br>                "library": "/usr/local/lib/kea/hooks/libdhcp_lease_cmds.so",<br>                "parameters": { }<br>            },<br>            {<br>                "library": "/usr/local/lib/kea/hooks/libdhcp_bootp.so",<br>                "parameters": { }<br>            },<br>                        {<br>                                "library": "/usr/local/lib/kea/hooks/libdhcp_ha.so",<br>                                "parameters": {<br>                                        "high-availability": [{<br>                                                "this-server-name": "kea-dhcp-168",<br>                                                "mode": "hot-standby",<br>                                                "heartbeat-delay": 10000,<br>                                                "max-response-delay": 60000,<br>                                                "max-ack-delay": 5000,<br>                                                "max-unacked-clients": 5,<br>                                                "peers": [{<br>                                                        "name": "kea-dhcp-168",<br>                                                        "url": "<a href="http://192.168.0.168:8000/">http://192.168.0.168:8000/</a>",<br>                                                        "role": "primary",<br>                                                        "auto-failover": true<br>                                                }, {<br>                                                        "name": "appliance-169",<br>                                                        "url": "<a href="http://192.168.0.169:8000/">http://192.168.0.169:8000/</a>",<br>                                                        "role": "standby",<br>                                                        "auto-failover": true<br>                                                }]<br>                                        }]<br>                                }<br>                }],<br>            "subnet4": [{<br>        "subnet": "<a href="http://4.0.0.0/24">4.0.0.0/24</a>",<br>                 "option-data": [<br>                    {<br>                        "data": "2.2.2.2, 8.8.4.4",<br>                        "name": "domain-name-servers"<br>                    },<br>                    {<br>                        "data": "4.0.0.1",<br>                        "name": "routers"<br>                    }<br>                ],<br>        "pools": [<br>            {<br>                "pool": "4.0.0.25 - 4.0.0.250"<br>            }<br>        ]<br><br>                }],<br>         "loggers": [<br>        {<br>            "name": "kea-dhcp4",<br>            "debuglevel": 99,<br>            "output_options": [<br>                {<br>                    "output": "/usr/local/var/log/kea-dhcp4.log"<br>                }<br>            ],<br>            "severity": "DEBUG"<br>        }<br>    ]<br>}<br>}<br>kea-ctrl-agent.conf on Primary<br>=========================<br><br>{<br>    "Control-agent": {<br>        "http-host": "192.168.0.168",<br>        "http-port": 8000,<br>        "control-sockets": {<br>            "dhcp4": {<br>                "socket-type": "unix",<br>                "socket-name": "/tmp/control_socket"<br>            }<br>        },<br>        "cert-required": false<br>    }<br>}<br>Netstat output<br>=============<br>[root@kea-dhcp-168 ~]# netstat -tulpna | grep 8000<br>tcp        0      0 <a href="http://192.168.0.168:8000">192.168.0.168:8000</a>      0.0.0.0:*               LISTEN      13581/kea-ctrl-agen<br>tcp        0      0 <a href="http://192.168.0.168:57308">192.168.0.168:57308</a>     <a href="http://192.168.0.169:8000">192.168.0.169:8000</a>      ESTABLISHED 13620/kea-dhcp4<br>tcp        0      0 <a href="http://192.168.0.168:8000">192.168.0.168:8000</a>      <a href="http://192.168.0.169:58432">192.168.0.169:58432</a>     ESTABLISHED 13581/kea-ctrl-agen<br>[root@kea-dhcp-168 ~]#<br><br>[root@kea-dhcp-168 ~]# netstat -tulpna | grep 67<br>udp        0      0 <a href="http://192.168.0.168:67">192.168.0.168:67</a>        0.0.0.0:*                           13620/kea-dhcp4<br>[root@kea-dhcp-168 ~]#<br><br><br>Failover DHCPv4 config<br>===================<br>#DHCPv4 192.168.0.169<br>{<br>"Dhcp4": {<br>    "valid-lifetime": 300,<br>    "renew-timer": 150,<br>    "rebind-timer": 200,<br>    "server-tag": "192.168.0.169",<br>      "control-socket": {<br>            "socket-name": "/opt/testUAT/keaFolder/kea4-ctrl-socket",<br>            "socket-type": "unix"<br>        },<br><br>    "lease-database": {<br>        "type": "memfile",<br>        "persist": true,<br>        "name": "/opt/testUAT/var/lib/kea/dhcp4.leases"<br>    },<br><br>  "interfaces-config": {<br>        "interfaces": ["eth0"]<br>  },<br><br>"hooks-libraries": [<br>            {<br>                "library": "/opt/testUAT/lib/kea/hooks/libdhcp_lease_cmds.so",<br>                "parameters": { }<br>            },<br>            {<br>                "library": "/opt/testUAT/lib/kea/hooks/libdhcp_bootp.so",<br>                "parameters": { }<br>            },<br>                        {<br>                                "library": "/opt/testUAT/lib/kea/hooks/libdhcp_ha.so",<br>                                "parameters": {<br>                                        "high-availability": [{<br>                                                "this-server-name": "appliance-169",<br>                                                "mode": "hot-standby",<br>                                                "heartbeat-delay": 10000,<br>                                                "max-response-delay": 60000,<br>                                                "max-ack-delay": 5000,<br>                                                "max-unacked-clients": 5,<br>                                                "peers": [{<br>                                                        "name": "kea-dhcp-168",<br>                                                        "url": "<a href="http://192.168.0.168:8000/">http://192.168.0.168:8000/</a>",<br>                                                        "role": "primary",<br>                                                        "auto-failover": true<br>                                                }, {<br>                                                        "name": "appliance-169",<br>                                                        "url": "<a href="http://192.168.0.169:8000/">http://192.168.0.169:8000/</a>",<br>                                                        "role": "standby",<br>                                                        "auto-failover": true<br>                                                }]<br>                                        }]<br>                                }<br>                }],<br>            "subnet4": [{<br>        "subnet": "<a href="http://4.0.0.0/24">4.0.0.0/24</a>",<br>                 "option-data": [<br>                    {<br>                        "data": "2.2.2.2, 8.8.4.4",<br>                        "name": "domain-name-servers"<br>                    },<br>                    {<br>                        "data": "4.0.0.1",<br>                        "name": "routers"<br>                    }<br>                ],<br>        "pools": [<br>            {<br>                "pool": "4.0.0.25 - 4.0.0.250"<br>            }<br>        ]<br><br>                }],<br>         "loggers": [<br>        {<br>            "name": "kea-dhcp4",<br>            "debuglevel": 99,<br>            "output_options": [<br>                {<br>                    "output": "/opt/testUAT/var/log/kea-dhcp4.log"<br>                }<br>            ],<br>            "severity": "DEBUG"<br>        }<br>    ]<br>}<br>}<br><br>kea-ctrl-agent.conf on Failover<br>=======================<br>{<br>    "Control-agent": {<br>        "http-host": "192.168.0.169",<br>        "http-port": 8000,<br>        "control-sockets": {<br>            "dhcp4": {<br>                "socket-type": "unix",<br>                "socket-name": "/tmp/control_socket"<br>            }<br>        },<br>        "cert-required": false<br>    }<br>}<br>Netstat output<br>=============<br><br>[root@appliance-169 ~]#  netstat -tulpna | grep 8000<br>tcp        0      0 <a href="http://192.168.0.169:8000">192.168.0.169:8000</a>      0.0.0.0:*               LISTEN      28520/kea-ctrl-agen<br>tcp        0      0 <a href="http://192.168.0.169:8000">192.168.0.169:8000</a>      <a href="http://192.168.0.168:57308">192.168.0.168:57308</a>     ESTABLISHED 28520/kea-ctrl-agen<br>tcp        0      0 <a href="http://192.168.0.169:58432">192.168.0.169:58432</a>     <a href="http://192.168.0.168:8000">192.168.0.168:8000</a>      ESTABLISHED 28550/kea-dhcp4<br>[root@appliance-169 ~]#<br><br>[root@kea-dhcp-168 ~]# netstat -tulpna | grep 67<br>udp        0      0 <a href="http://192.168.0.169:67">192.168.0.169:67</a>        0.0.0.0:*                           28550/kea-dhcp4<br>[root@kea-dhcp-168 ~]#<br><br>I am not sure why it is unable to forward commands to dhcp4 service, Can someone guide or help me if you face the same issue or are aware of the issue?<br><br>Thanks in advance <br>Kraishak <br><br></div></font></div>