[bind10-dev] auth testing

[Shane Kerr]

Jeremy,

Fantastic stuff!

Hopefully these scripts can be tidied up and used regularly to compare
the latest versions of servers over time.

On Tue, 2009-09-15 at 15:48 -0500, Jeremy C. Reed wrote:
> We now have a dedicated system for doing performance and benchmark 
> testing. I don't know what type of hardware it is. I think it has two 
> physical CPUs. The dmesg says:
> 
>  Memory: 4032428k/5242880k available (2493k kernel code, 152780k reserved, 
>     1263k data, 200k init
>  Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
>  Brought up 8 CPUs
> 
> The operating system is CentOS 5.3 running a x86_64 Linux 
> 2.6.18-128.7.1.el5 with SMP.

"lshw" can give you an embarrassing amount of information about the
hardware. It can be installed via "yum" on CentOS.

> My script to gather resource usage stores the /proc/$PID/status, stat, and 
> statm files, and ps output like:
> 
> %CPU  C     ELAPSED     TIME   TIME - -  UTIME %MEM   RSS   RES    SZ - 
> START    VSZ COMMAND
> 40.0 40       00:01 00:00:00   0:00 - -      -  0.0  1808     -  1512 - 
> 12:10   5736 named

I got better results for CPU time by looking at the output of the
"vmstat" command. It won't give you per-process values, but on a test
like this we don't really care, since we are looking at a dedicated
system.

> I have lots of data stored, but here is a summary:
> 
> Server		Queries/second	Virtual Size	cputime/realtime
> ------		--------------	------------	----------------
> bind-8.4.7	77582 qps	5736 kB		40.0%
> bind-9.6.1	37745 qps	8536 kB		79.0%
> pdns-2.9.22	89884 qps	148364 kB	0.0%
> nsd-3.2.3	83938 qps	14228 kB	0.0%
> python-test	29812 qps	104184 kB	101%

You can get a lot better performance from Python using psyco. ;)

I think you should probably use resident size (RSS) rather than virtual
size (VSZ).

Note to know how much memory a process is *actually* using is...
non-trivial. Good article about it here:

http://lwn.net/Articles/230975/

Actual documentation about the mechanism is in the kernel itself:

http://tinyurl.com/qkpyat

The author has been hard at work, and has a new tool this year:

http://lwn.net/Articles/329458/

Sadly his site appears to be down. 

Anyway, RSS is Good Enough(tm).

> (I don't know why the %CPU is 0.0 sometimes. And I don't know how it is 
> 101% either, must be very inefficient? Also I saw pdns reach 213900 kB 
> VSIZE and 32.0 PCPU. And I have see nsd reach 90450 qps. I did several 
> runs, but above summary is not an average.)

How long did the test run? Maybe we need to let it loop a bit more.

> My questions:
> 
> 1) Any types of tests you'd like me to run?

It might be nice to sign the zones, both with NSEC and NSEC3. Don't
forget to turn on DNSSEC in queryperf if you do that.

IPv6 testing might also be nice.

> 3) Other server software?

We're only missing proprietary authoritative servers I think (Microsoft,
Nominum, and so on).

> 7) Any suggestions or pointers for creating graphs? What do you want to 
> see? How should it be presented? Should I average results?

I'm quite happy with the table you gave, although an averaged values do
make sense.

--
Shane