dhcp gives right IP, but ddns adds wrong one to DNS

[Frank Price]

After staring at this for days, I throw myself at the feet of the list and
beg for enlightenment.

ISC DHCP 4.2.5 on Centos 7, + BIND 9.9.4-RedHat-9.9.4-18.el7_1.1 (Extended
Support Version)

relevant bits of dhcpd.conf:
  ddns-update-style interim;
  ignore client-updates;
  update-static-leases on;
  update-conflict-detection false;

I have a server which gets IP from dhcp.  It gets a lease, the IP is set
properly, it asks for dns updates, and the dns server does ... something
... which ends up in it having an A record that is completely wrong (the
name is right but the IP is wrong).

Even if I freeze my zone and correct everything, after the TTL for the RR,
something is setting the A RR back to the incorrect IP.  I am really really
losing it on this one.

Log snippets:
a)  from the dhcp server.  To me this looks pretty good.  10.xxx.145.177 is
the addr of the server.

Jul 21 14:05:10 dns2 dhcpd: DHCPREQUEST for 10.xxx.145.177 from
f0:1f:af:ce:d9:34 via 10.xxx.144.3
Jul 21 14:05:10 dns2 dhcpd: DHCPACK on 10.xxx.145.177 to f0:1f:af:ce:d9:34
(supreme) via 10.xxx.144.3
Jul 21 14:05:10 dns2 dhcpd: DHCPREQUEST for 10.xxx.145.177 from
f0:1f:af:ce:d9:34 (supreme) via 10.xxx.144.2
Jul 21 14:05:10 dns2 dhcpd: Added new forward map from supreme.lpdev.foo.com
to 10.xxx.145.177
Jul 21 14:05:10 dns2 dhcpd: Added reverse map from
177.145.xxx.10.in-addr.arpa. to supreme.lpdev.foo.com
Jul 21 14:05:10 dns2 dhcpd: bind update on 10.xxx.145.177 got ack from
partner: xid mismatch.

b) from the dns server's updates logging.  Also looks ok to me:
21-Jul-2015 14:05:10.479 update: info: client 10.xxx.21.46#56082/key
lpdev.key: updating zone 'lpdev.foo.com/IN': update unsuccessful:
supreme.lpdev.foo.com: 'name not in use' prerequisite not satisfied
(YXDOMAIN)
21-Jul-2015 14:05:10.496 update: info: client 10.xxx.21.46#56082/key
lpdev.key: updating zone 'lpdev.foo.com/IN': update unsuccessful:
supreme.lpdev.foo.com: 'name not in use' prerequisite not satisfied
(YXDOMAIN)
21-Jul-2015 14:05:10.508 update-security: info: client
10.xxx.21.46#56082/key lpdev.key: signer "lpdev.key" approved
21-Jul-2015 14:05:10.508 update: info: client 10.xxx.21.46#56082/key
lpdev.key: updating zone 'lpdev.foo.com/IN': deleting rrset at '
supreme.lpdev.foo.com' TXT
21-Jul-2015 14:05:10.508 update: info: client 10.xxx.21.46#56082/key
lpdev.key: updating zone 'lpdev.foo.com/IN': adding an RR at '
supreme.lpdev.foo.com' TXT
21-Jul-2015 14:05:10.508 update: info: client 10.xxx.21.46#56082/key
lpdev.key: updating zone 'lpdev.foo.com/IN': deleting rrset at '
supreme.lpdev.foo.com' A
21-Jul-2015 14:05:10.508 update: info: client 10.xxx.21.46#56082/key
lpdev.key: updating zone 'lpdev.foo.com/IN': adding an RR at '
supreme.lpdev.foo.com' A
21-Jul-2015 14:05:10.532 update-security: info: client
10.xxx.21.46#56082/key lpdev.key: signer "lpdev.key" approved
21-Jul-2015 14:05:10.532 update: info: client 10.xxx.21.46#56082/key
lpdev.key: updating zone '145.xxx.10.in-addr.arpa/IN': deleting rrset at
'177.145.xxx.10.in-addr.arpa' PTR
21-Jul-2015 14:05:10.532 update: info: client 10.xxx.21.46#56082/key
lpdev.key: updating zone '145.xxx.10.in-addr.arpa/IN': adding an RR at
'177.145.xxx.10.in-addr.arpa' PTR

c)  here's the lease.  Things still match up.
lease 10.xxx.145.177 {
  starts 2 2015/07/21 18:05:10;
  ends 3 2015/07/22 18:05:10;
  tstp 4 2015/07/23 06:05:10;
  tsfp 4 2015/07/23 06:05:10;
  atsfp 4 2015/07/23 06:05:10;
  cltt 2 2015/07/21 18:05:10;
  binding state active;
  next binding state expired;
  hardware ethernet f0:1f:af:ce:d9:34;
  uid "\001\360\037\257\316\3314";
  set ddns-rev-name = "177.145.xxx.10.in-addr.arpa.";
  set ddns-txt = "313e29237150b143ecae66d87b9f881997";
  set ddns-fwd-name = "supreme.lpdev.foo.com";
  client-hostname "supreme";
}

Now the kicker:  dig -t any supreme.lpdev.foo.com.  The TXT record is the
same as the lease, but the IP is 157.yyy.6.171 -- not 10.xxx.145.177.

; <<>> DiG 9.9.4-RedHat-9.9.4-18.el7_1.1 <<>> -t any supreme.lpdev.foo.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27655
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 4, ADDITIONAL: 7

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;supreme.lpdev.foo.com. IN   ANY

;; ANSWER SECTION:
supreme.lpdev.foo.com. 900 IN A      157.yyy.6.171
supreme.lpdev.foo.com. 900 IN TXT    "319f076f60d81714bcd70f25de57df86b1"

Any clues will be very much appreciated!

-Frank
--
Frank Price
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.isc.org/pipermail/dhcp-users/attachments/20150721/c5fcfcff/attachment.html>