[bind10-dev] it takes long time to IXFR

JINMEI Tatuya / 神明達哉 jinmei at isc.org
Mon Apr 1 03:57:44 UTC 2013


At Fri, 29 Mar 2013 09:00:42 -0700,
JINMEI Tatuya <jinmei at isc.org> wrote:

> > > Could you apply the patch attached and check full debug logs while
> > > updating the zone in IXFR?  It logs owner names, but I suspect it's
> > > not data dependent, so it should be removed or obscured if you report
> > > the results publicly.
> > Thank you for making the patch. We've applied it and ran IXFR.
> > Unfortunately the logfile has grown to over 1GB. Could you please let me
> > know directly what type of log are needed and how to pass it to you?
> 
> I'd be interested in these for a single IXFR session (that takes a
> long time):
> 
> % DATASRC_DATABASE_ADDDIFF updated diff table for add: %1 %2 %3
> % DATASRC_DATABASE_DELETEDIFF updated diff table for delete: %1 %2 %3
> % DATASRC_DATABASE_ADDNSEC3 added NSEC3 RR: %1 %2
> % DATASRC_DATABASE_ADDRR added RR: %1 %2 %3
> % DATASRC_DATABASE_DELETENSEC3 deleted NSEC3 RR: %1 %2
> % DATASRC_DATABASE_DELETERR deleted RR: %1 %2 %3
> 
> in addition to DATASRC_SQLITE_CONNOPEN and DATASRC_SQLITE_DROPCONN
> before and after them.
> 
> I guess most of the logs are for building in-memory zone after
> completing the IXFR.  They are not of our interest.

According to the more detailed logs unicasted to me, the bottleneck
seems to be "delete RR" operations.  And, in fact, I could reproduce it
at the sqlite3 level with a sample of large zone data.  In fact, an
sqlite3 like this took several (> 5) seconds:

sqlite> DELETE FROM records WHERE zone_id=1 AND name='example.com.' AND rdtype='NS' AND rdata='ns.example.com.';

when the records table contains several millions of records.  I
suspected it was because this statement internally performs search for
the records that match the condition and that should be the major
bottleneck.

So one possible solution would be to revise the statement so the
search part can use more efficient indexes, like this one:

sqlite> DELETE FROM records WHERE zone_id=1 AND rname='com.example.' AND rdtype='NS' AND rdata='ns.example.com.';

this should have the same effect as the previous one, and was actually
much faster in my experiment.

I'm attaching a patch to BIND 10 so we use the latter version of
SQLite3 statement for deleting records.  Could you try that and see if
that makes it faster?

---
JINMEI, Tatuya
Internet Systems Consortium, Inc.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: sqlite3-del.patch
Type: application/octet-stream
Size: 5694 bytes
Desc: not available
URL: <https://lists.isc.org/pipermail/bind10-dev/attachments/20130331/d307b041/attachment.obj>


More information about the bind10-dev mailing list