[kea-dev] kea 1.8.2 (with premium hooks) problem with gre tunnel interfaces

Stefan Berger Stefan.Berger at wvnet.eu
Tue Apr 6 12:58:25 UTC 2021


Hi all,

i have already asked the user-list but i havn't received an answer.
I thought i have solved the problem, but it still exist - maybe somebody can give me a hint.

I'm using kea 1.6.3 with premium hooks in an anycast setup (ipv4 only) .
Each anycast node has two GRE tunnels which are terminated on different broadband aggregation routers (BAR)
These BARs are cisco ASR1004 which are forwarding (via udp-helper) the dhcp requests from the clients to our kea cluster

The kea daemon itself is configured to listen on an specific loopback ip which is
advertised by BGP (bird) to the BARs and the next-hop ip is set to GRE tunnel IP (MY_INNER_IPADDR)
This is working pretty fine with version 1.6.3
All requests are received through GRE tunnel and the response is also send via GRE

During the upgrade  to 1.8 we realized that kea isn't able to send the response through GRE.
We also tried to configure  "outbound-interface": "use-routing" but it didn't work.

The reported error was 

[kea-dhcp4.packets/13945.140485403977856] DHCP4_PACKET_SEND_FAIL [hwtype=1 f4:cf:e2:98:75:e1], 
cid=[00:63:69:73:63:6f:2d:66:34:63:66:2e:65:32:39:38:2e:37:35:65:31:2d:47:69:30:2f:31:2e:39:30:30], 
tid=0x14ea: failed to send DHCPv4 packet: Interface tun1/8 does not have any suitable IPv4 sockets open

17:23:15 lseek(5, 0, SEEK_CUR)          = 9296497 <0.000008>
17:23:15 lseek(5, 0, SEEK_END)          = 9296497 <0.000008>
17:23:15 write(5, "2021-03-25 17:23:15.918 WARN  [kea-dhcp4.dhcpsrv/18423.139928109389952] DHCPSRV_OPEN_SOCKET_FAIL failed to open socket: failed to open socket on interface tun1, reason: Failed to bind socket 25 to 172.16.200.254/port=67\n", 220) = 220 <0.000015>
17:23:15 lseek(5, 0, SEEK_CUR)          = 9296717 <0.000008>
17:23:15 fcntl(4, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0 <0.000009>
17:23:15 fcntl(8, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0 <0.000009>
17:23:15 fcntl(8, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0 <0.000010>
17:23:15 fcntl(4, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0 <0.000009>

System:
--------------------------------------------------------
Oracle EL 7 3.10.0-1160.21.1.el7.x86_64
Networkmanager is disabled

net.ipv4.fib_multipath_use_neigh=1
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.lo.rp_filter = 0
net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.default.forwarding = 1
net.ipv4.ip_forward = 1
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.default.arp_notify = 1
net.ipv4.conf.default.arp_ignore=1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.all.arp_notify = 1
net.ipv4.conf.all.arp_ignore=1
net.ipv4.icmp_errors_use_inbound_ifaddr=1


GRE Tunnel Interface
----------------------------
#GRE Tunnel Config
#cat ifcfg-tun1
DEVICE=tun1
BOOTPROTO=none
ONBOOT=yes
TYPE=GRE
PEER_OUTER_IPADDR=172.16.213.253
PEER_INNER_IPADDR=172.16.200.254/24
MY_OUTER_IPADDR=10.214.200.4
MY_INNER_IPADDR=172.16.200.4/24

STRACE on kea-dchp4
------------------------------
172.16.160.248 is the anycast loopback ip on which kea-dhcp4 is listening

"interfaces-config": {
        "interfaces": [ "lo/172.16.160.248" ],
        "dhcp-socket-type": "udp",
          "outbound-interface": "use-routing"
        // "outbound-interface": "same-as-inbound"
       // same as inbound is default
    },

In DHCP4_PACKET_SEND everything looks ok - 172.16.160.248 is the loopback IP on which the request
was received and 192.168.168.200.254 is the Interface on the BAR (broadband aggregation router) 
with the cisco ip-helper.

DHCP4_RESPONSE_DATA shows the right DHCPOFFER

Then the DHCP4_PACKET_SEND_FAIL Message - tun1 is the interface on which the packet was received.
I think the right way should be sending the packet through the loopback and the linux stack should do the rest.

16:46:02 write(5, "2021-03-26 16:46:02.116 DEBUG [kea-dhcp4.options/13945.140485403977856] DHCP4_PACKET_PACK [hwtype=1 f4:cf:e2:98:75:e1], cid=[00:63:69:73:63:6f:2d:66:34:63:66:2e:65:32:39:38:2e:37:35:65:31:2d:47:69:30:2f:31:2e:39:30:30], tid=0x14ea: preparing on-wire format of the packet to be sent\n", 282) = 282 <0.000011>
16:46:02 lseek(5, 0, SEEK_CUR)          = 9600239 <0.000009>
16:46:02 fcntl(4, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0 <0.000010>
16:46:02 fcntl(26, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0 <0.000010>
16:46:02 fcntl(25, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0 <0.000011>
16:46:02 fcntl(4, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0 <0.000010>
16:46:02 lseek(5, 0, SEEK_END)          = 9600239 <0.000010>
16:46:02 lseek(5, 0, SEEK_CUR)          = 9600239 <0.000009>
16:46:02 lseek(5, 0, SEEK_END)          = 9600239 <0.000010>
16:46:02 write(5, "2021-03-26 16:46:02.117 DEBUG [kea-dhcp4.packets/13945.140485403977856] DHCP4_PACKET_SEND [hwtype=1 f4:cf:e2:98:75:e1], cid=[00:63:69:73:63:6f:2d:66:34:63:66:2e:65:32:39:38:2e:37:35:65:31:2d:47:69:30:2f:31:2e:39:30:30], tid=0x14ea: trying to send packet DHCPOFFER (type 2) from 172.16.160.248:67 to 192.168.200.254:67 on interface lo\n", 334) = 334 <0.000016>
16:46:02 lseek(5, 0, SEEK_CUR)          = 9600573 <0.000010>
16:46:02 fcntl(4, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0 <0.000010>
16:46:02 fcntl(25, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0 <0.000010>
16:46:02 fcntl(25, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0 <0.000011>
16:46:02 fcntl(4, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0 <0.000010>
16:46:02 lseek(5, 0, SEEK_END)          = 9600573 <0.000010>
16:46:02 lseek(5, 0, SEEK_CUR)          = 9600573 <0.000010>
16:46:02 lseek(5, 0, SEEK_END)          = 9600573 <0.000010>
16:46:02 write(5, "2021-03-26 16:46:02.117 DEBUG [kea-dhcp4.packets/13945.140485403977856] DHCP4_RESPONSE_DATA [hwtype=1 f4:cf:e2:98:75:e1], cid=[00:63:69:73:63:6f:2d:66:34:63:66:2e:65:32:39:38:2e:37:35:65:31:2d:47:69:30:2f:31:2e:39:30:30], tid=0x14ea: responding with packet DHCPOFFER (type 2), packet details: local_address=172.16.160.248:67, remote_address=192.168.200.254:67, msg_type=DHCPOFFER (2), transid=0x14ea,\noptions:\n  type=001, len=004: 4294967040 (uint32)\n  type=003, len=004: 192.168.200.254\n  type=006, len=008: 8.8.8.8 9.9.9.9\n  type=012, len=014: \"config-station\" (string)\n  type=051, len=004: 28800 (uint32)\n  type=053, len=001: 2 (uint8)\n  type=054, len=004: 172.16.160.248\n  type=061, len=031: 00:63:69:73:63:6f:2d:66:34:63:66:2e:65:32:39:38:2e:37:35:65:31:2d:47:69:30:2f:31:2e:39:30:30\n", 802) = 802 <0.000018>
16:46:02 lseek(5, 0, SEEK_CUR)          = 9601375 <0.000010>
16:46:02 fcntl(4, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0 <0.000010>
16:46:02 fcntl(25, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0 <0.000010>
16:46:02 fcntl(25, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0 <0.000010>
16:46:02 fcntl(4, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0 <0.000009>
16:46:02 lseek(5, 0, SEEK_END)          = 9601375 <0.000008>
16:46:02 lseek(5, 0, SEEK_CUR)          = 9601375 <0.000009>
16:46:02 lseek(5, 0, SEEK_END)          = 9601375 <0.000009>
16:46:02 write(5, "2021-03-26 16:46:02.118 ERROR [kea-dhcp4.packets/13945.140485403977856] DHCP4_PACKET_SEND_FAIL [hwtype=1 f4:cf:e2:98:75:e1], cid=[00:63:69:73:63:6f:2d:66:34:63:66:2e:65:32:39:38:2e:37:35:65:31:2d:47:69:30:2f:31:2e:39:30:30], tid=0x14ea: failed to send DHCPv4 packet: Interface tun1/8 does not have any suitable IPv4 sockets open.\n", 330) = 330 <0.000016>
16:46:02 lseek(5, 0, SEEK_CUR)          = 9601705 <0.000009>
16:46:02 fcntl(4, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0 <0.000009>
16:46:02 fcntl(25, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0 <0.000008>




More information about the kea-dev mailing list