Performance Test Metrics for dns server performance.

[Bill Larson]

> > 	There is also the "netperf" tool developed by Rick Jones at HP (see 
> > <ftp://ftp.cup.hp.com/dist/networking/briefs/named_performance.txt> 
> > for more information).

Rick Jones use his netperf3 tool to benchmark BIND operations, not the
standard "netperf" tool.  Netperf3 does perform a complete DNS query.
Granted, I had a difficult time compiling netperf3, and this was under
HPUX-11. It was so bad that I gave up and ran a precompiled binary.
And even then, I talked to Rick about this only to discover that there
was an additional library necessary that I didn't have.

Given that, the idea that Rick has is correct.  You don't want to try
and benchmark recursive queries, you have no control over the other
servers, including the root servers, that you would be using.  This
makes a recursive query and extremely uncontrolled experiment, and
a poor benchmark.

Possibly, as a secondary experiment, you could measure the response
time for recursive queries, but be prepared for an extremely wide
variation in results.  For example, I have played around measuring the
response time for a domain name maintained by a server widely removed
from my own Internet access (cable modem at home, my ISP, and work).
This query resulted in response times from 0.15s to up to 5s and
followed a squashed bell curve distribution (can't remember the proper
name for the distribution anymore) with an average of around 0.25s.

This distribution was NOT under my control, therefore trying to account
for someone elses performance makes for a bad benchmark.  Then again,
maybe this whole topic is now more appropriate in the comp.benchmarks
newsgroup than in info-bind/cpdb.

Finally, another item to take from Rick's experiments.  You need to use
multiple client simulator machines.  You can't generate a large enought
load with a single machine without being limited by the client's
system.  Many years ago, I ran multiple copies of dig on a 60MHz
Pentium system.  I could hit my D.U.T. ("device under test" - the DNS
server) with about 100 queries/s, but no further than than.  To obtain
a greater query rate, I had to use multiple systems as the query
sources.  This was nothing fancy, but it did give me a "warm fuzzy
feeling" about the performance of my server.

I guess my question for you is what are your goals with this testing.
Are you trying to determine what server will provide faster responses?
Or, are you trying to determine is one server or another will provide
the performance that you require?

If you are using some form of a performance criteria (I would suggest
looking into RFC2010 for a suggestion), such as 500 queries/s with a
response time of less than 0.1sec, you have to have at least a guess as
to what query rate you are expecting and them bump it up by 50-100% as
a fudge factor.

You also may be interested in a "dns.ping" script that I wrote and
posted here a while back.  This is what I used to repeated query a
server for the same information to determine what sort of response time
distribution is returned.  Search the archives for this if you are
interested, or send me a message and I will forward you a copy of this
script - but let's keep this off this list.

You are performing what I believe is an extremely useful, and
interesting, test.  Good luck with your work, and keep us informed of
your results.  Many people would be interested in your results (at
least myself).

Bill Larson