<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p>It looks like the original issue was caused by an auto
      configuration I had made which changed permissions on
      /etc/kea/     .</p>
    <p><br>
    </p>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 7/26/25 09:36, fbcadmin wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:20250726133623.3E55423073@Mail.fantinibakery.com">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <p>I did these steps to fix. <br>
      </p>
      <p><br>
      </p>
      <p><span style="font-family:monospace"><span
            style="color:#000000;background-color:#ffffff;">chgrp _kea
            /etc/kea</span><br>
        </span></p>
      <p><span style="font-family:monospace">chmod 750 /etc/kea</span></p>
      <p><span style="font-family:monospace"><br>
        </span></p>
      <p>on most systems for some unknown reason had this at most
        systems</p>
      <p><font face="monospace">aptitude  search isc-kea-ctrl-agent<br>
        </font></p>
      <p><font face="monospace">idA
          isc-kea-ctrl-agent                                             
          - ISC Kea DHCP server REST API
          service                             </font>    </p>
      <p><br>
      </p>
      <p>did this to fix:</p>
      <p><span style="font-family:monospace"><span
            style="color:#000000;background-color:#ffffff;">apt-get -y
            install isc-kea-ctrl-agent</span><br>
        </span><span style="font-family:monospace"><span
            style="color:#000000;background-color:#ffffff;">systemctl
             enable  isc-kea-ctrl-agent.service</span></span></p>
      <p><span style="font-family:monospace"><span
            style="color:#000000;background-color:#ffffff;">ystemctl
             start   isc-kea-ctrl-agent.service</span></span></p>
      <p><span style="font-family:monospace"><span
            style="color:#000000;background-color:#ffffff;"><br>
          </span></span></p>
      <p><span style="font-family:monospace"><span
            style="color:#000000;background-color:#ffffff;">then
            rebooted<br>
          </span></span></p>
      <p><br>
      </p>
      <p><br>
      </p>
      <div class="moz-cite-prefix">On 7/24/25 10:29, Corby Stephens
        wrote:<br>
      </div>
      <blockquote type="cite"
        cite="mid:20250724143006.B63B6200A5@Mail.fantinibakery.com">
        <meta http-equiv="Content-Type"
          content="text/html; charset=UTF-8">
        <style type="text/css" style="display:none;">P {margin-top:0;margin-bottom:0;}</style>
        <div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"
          class="elementToProof"> Is kea-ctrl-agent still running?</div>
        <div class="elementToProof" id="Signature">
          <div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"
            class="elementToProof"> <br>
          </div>
          <p
style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0in;"
            class="elementToProof"> <span
style="font-family: Calibri, sans-serif; font-size: 11pt; color: rgb(36, 36, 36);"><br>
            </span></p>
          <p
style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0in;"
            class="elementToProof"> <span
style="font-family: "Calibri Light", sans-serif; font-size: 12pt; color: rgb(31, 73, 125);"><b>Corby
                Stephens, Fiber Network Administrator</b> | </span><span
style="font-family: "Calibri Light", sans-serif; font-size: 12pt; color: rgb(10, 28, 65);"><b>ACCESS</b> Anacortes
              Fiber Internet</span><span
style="font-family: "Calibri Light", sans-serif; font-size: 12pt; color: rgb(31, 73, 125);"> |
              P.O. Box 547 | 904 Sixth Street | Anacortes, WA  98221</span></p>
          <p
style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0in;"
            class="elementToProof"> <span
style="font-family: "Calibri Light", sans-serif; font-size: 12pt; color: rgb(31, 73, 125);">Office: 
              360.588.8362 | Cell:  360.873.3770 | </span><span
style="font-family: Aptos, sans-serif; font-size: 12pt; color: rgb(31, 73, 125);"><a
                style="color: rgb(31, 73, 125); margin: 0px;"
                data-linkindex="0" title="mailto:corbys@anacorteswa."
                class="OWAAutoLink moz-txt-link-freetext"
                id="OWA39f5544c-084e-e3db-7e91-1ce5bdb7bfea"
                href="mailto:corbys@anacorteswa." moz-do-not-send="true">corbys@anacorteswa.</a>gov</span><span
style="font-family: "Calibri Light", sans-serif; font-size: 12pt; color: rgb(31, 73, 125);"> |
            </span><span
style="font-family: Aptos, sans-serif; font-size: 12pt; color: rgb(31, 73, 125);"><a
                style="color: rgb(31, 73, 125); margin: 0px;"
                data-auth="NotApplicable" data-linkindex="1"
                title="http://www.anacorteswa.gov/fiber"
                class="OWAAutoLink"
                id="OWA8dc66009-a735-f249-269b-a10dbea89bbf"
                href="http://www.anacorteswa.gov/fiber"
                moz-do-not-send="true">www.anacorteswa.gov/fiber</a></span><span
style="font-family: "Calibri Light", sans-serif; font-size: 12pt; color: rgb(31, 73, 125);">           </span></p>
          <p
style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0in;"
            class="elementToProof"> <span
style="font-family: Aptos, sans-serif; font-size: 12pt; color: rgb(36, 36, 36);"><img
style="width: 2.6041in; height: 2.6041in; max-width: 722px; min-width: auto; min-height: auto; margin: 0px;"
                height="249" width="249" id="x_Picture_x0020_1"
                data-outlook-trace="F:1|T:1"
                src="cid:part1.Y8wHQkkH.e5ZNW60H@fantinibakery.com"
                class=""></span></p>
        </div>
        <hr style="display:inline-block;width:98%" tabindex="-1">
        <div id="divRplyFwdMsg" dir="ltr"><font
            face="Calibri, sans-serif" style="font-size:11pt"
            color="#000000"><b>From:</b> Stork-users <a
              class="moz-txt-link-rfc2396E"
              href="mailto:stork-users-bounces@lists.isc.org"
              moz-do-not-send="true"><stork-users-bounces@lists.isc.org></a>
            on behalf of fbcadmin <a class="moz-txt-link-rfc2396E"
              href="mailto:fbcadmin@fantinibakery.com"
              moz-do-not-send="true"><fbcadmin@fantinibakery.com></a><br>
            <b>Sent:</b> Wednesday, July 23, 2025 7:05 PM<br>
            <b>To:</b> <a
              class="moz-txt-link-abbreviated moz-txt-link-freetext"
              href="mailto:stork-users@lists.isc.org"
              moz-do-not-send="true">stork-users@lists.isc.org</a> <a
              class="moz-txt-link-rfc2396E"
              href="mailto:stork-users@lists.isc.org"
              moz-do-not-send="true"><stork-users@lists.isc.org></a><br>
            <b>Subject:</b> Re: [stork-users] stork issue</font>
          <div> </div>
        </div>
        <div>
          <p align="center" style="text-align:center"><strong><span
style="font-size:10.0pt; font-family:"Arial",sans-serif; color:white; background:#BA4642">*****   
                This email is from outside the City of Anacortes
                network.  Please use caution when clicking on links,
                opening attachments, or replying.   *****</span></strong></p>
          <div>
            <p>also I can ping from stork to the dhcp servers.  and ssh
              in to them.<br>
            </p>
            <div class="x_moz-cite-prefix">On 7/23/25 20:15, fbcadmin
              wrote:<br>
            </div>
            <blockquote type="cite">
              <p>We have 10+ kea dhcp servers monitored by stork.   for
                some unknown yet reason 7 of them are out of
                communication with stork .    this issue started last
                night.  there were no communication issues for the last
                month.</p>
              <p>I've checked software version and configs.  The systems
                in comm with stork have the same software versions  and
                base configuration.  All systems are giving out dhcp
                addresses.  per systemclt all is well. </p>
              <p> I tried restoring a backup on one of the dhcp servers
                to no solution yet.</p>
              <p>A few hours before the issue started I did a hardware
                upgrade to one of our nodes in proxmox cluster.   So
                there could be something with our network etc.  I looked
                but will look again..<br>
              </p>
              <p>I did notice the following in /var/log/daemon.log on
                stork.   these repeat often - the log fills 2MB per
                hour.<br>
              </p>
              <p>looking at the logs,  does anyone notice something to
                be fixed in our configuration?</p>
              <p><br>
              </p>
              <p>Update -  I checked earlier in logs this started on : 
                so this may be unrelated.  However they are coming a lot
                faster  ' <span style="font-family:monospace"><span
                    style="color:#000000; background-color:#ffffff">Jul
                    21 15:41:33 stork stork-server[284003]:
                    time="2025-07-21 15:41:33" level="error" msg="</span><span
                    style="color:#ffffff; background-color:#000000">Problem
                    populating configuration review repo>'     </span><br>
                </span></p>
              <p><br>
              </p>
              <p><font face="monospace">Jul 23 19:58:34 stork
                  stork-server[132]: time="2025-07-23 19:58:34"
                  level="warning" msg="Problem getting state from Kea
                  CA: cannot find Kea app" file="          
                  appkea.go:303  "<br>
                </font></p>
              <p><font face="monospace">Jul 23 19:58:34 stork
                  stork-server[132]: time="2025-07-23 19:58:34"
                  level="warning" msg="Problem getting state from Kea
                  daemons: cannot find Kea app" file="          
                  appkea.go:309  "<br>
                </font></p>
              <p><font face="monospace"><br>
                </font></p>
              <p><font face="monospace">Jul 23 19:58:34 stork
                  stork-server[132]: time="2025-07-23 19:58:34"
                  level="error" msg="Problem populating configuration
                  review reports <br>
                  to the database for daemon 70103: ERROR #23502 null
                  value in column \"config_hash\" of relation
                  \"config_review\" violates not-null
                  constraint\nproblem upserting the configuration review
                  entry for daemon 70103\<a
href="https://linkprotect.cudasvc.com/url?a=https%3a%2f%2f%2f%2fnisc.org%2fstork%2fserver%2fdatabase%2fmodel.AddConfigReview%5cn%5ct%2fbuilds%2fisc-projects%2fstork%2fbackend%2fserver%2fdatabase%2fmodel%2fconfigreview.go%3a42%5cnisc.org%2fstork%2fserver%2fconfigreview&c=E,1,-zCAGUJ1rC38w68vivnRf1qZlrScUcJP4PR4jXAfV2LBQZbN2bZjZifw6Yr0ZSFWLsluExbxVvCtSdP67rdshhmHiutwYWQCKEBRc0J76v2EX-A,&typo=1&ancr_add=1"
                    moz-do-not-send="true">//nisc.org/stork/server/database/model.AddConfigReview\n\t/builds/isc-projects/stork/backend/server/database/model/configreview.go:42\nisc.org/stork/server/configreview</a>.(*dispatcherImpl).populateReports\n\t/builds/isc-projects/stork/backend/server/configreview/dispatcher.go:664\<a
href="https://linkprotect.cudasvc.com/url?a=https%3a%2f%2f%2f%2fnisc.org%2fstork%2fserver%2fconfigreview&c=E,1,hBKOHQkA5jShHrRxErNrtbGky6PsQAwLF_ZzuRf0r1WgRBuh1t2CZnW9SSthpJ7C6pvdIL0EPrCCCV7jWTPw8yWAq5I4bkMWx-RNVDHJCGdpktqfViTd20o,&typo=1&ancr_add=1"
                    moz-do-not-send="true">//nisc.org/stork/server/configreview</a>.(*dispatcherImpl).<br>
awaitReports\n\t/builds/isc-projects/stork/backend/server/configreview/dispatcher.go:359\nruntime.goexit\n\t/builds/isc-projects/stork/tools/golang/go/src/runtime/asm_amd64.s:1700"
                  file="       dispatcher.go:361  "</font><br>
              </p>
              <br>
              <fieldset class="x_moz-mime-attachment-header"></fieldset>
            </blockquote>
          </div>
        </div>
      </blockquote>
      <br>
      <fieldset class="moz-mime-attachment-header"></fieldset>
    </blockquote>
  </body>
</html>