[bind10-dev] Scaling the resolver across multiple cores

[Michal 'vorner' Vaner]

Hello

On Mon, Mar 11, 2013 at 11:31:51AM +0100, Michal 'vorner' Vaner wrote:
> Unfortunately, they are mostly feelings. Does anybody have an idea how to
> measure them reasonably (and without actually implementing the whole resolver)?

As talked yesterday on the call, here's a concrete proposal how to measure them.
We need something similar enough to a resolver, but we don't want to do real
resolution.

So, we replace the resolution with something dummy. Let's say a resolution of a
„query“ will contain these steps:
 • Receive: { for (size_t i = 0; i < 1000000; i ++) doNothing(); }
 • Parse: { for (size_t i = 0; i < 1000000000; i ++) doNothing(); }
 • Look into the cache:
   ◦ { for (size_t i = 0; i < 1000000000; i ++) doNothing(); }
   ◦ if (random(50) == 0) {
       Not found in cache → Proceed to upstream queries (do 1 + random(3)
       upstream queries)
     } else {
       Found, good
     }
 • Render: { for (size_t i = 0; i < 1000000000; i ++) doNothing(); }
 • Send: { for (size_t i = 0; i < 1000000; i ++) doNothing(); }

Upstream query would look like:
 • Some rendering, etc: { for (size_t i = 0; i < 1000000000; i ++) doNothing(); }
 • Schedule timeout after short time (random(30) ms).
 • After the time, update cache { for (size_t i = 0; i < 1000000000; i ++) doNothing(); }

Now, the doNothing() would be an empty function (in some other compilation
module, so the compiler will not want to remove it completely).

Then, we would implement this work in each of the models, with all the locking,
etc. We would then generate the „queries“ artificially and queue bunch of them
at the beginning and see how long it takes to answer all of them with reasonable
number of threads, tweaking the parameters (like amount of queries batched in
the landlords model), etc.

Do you think it would work? If so, we would have something like 4 tickets:
 • Write the functions to do the work described here that would be placed in the
   respective parts of the model. This would be to ensure every experiment
   measures the same „workload“.
 • 3 tickets, one for each model.

Now, I don't know if anybody has a coroutine library at hand, I don't know of
any and I don't think writing it with ucontext.h is reasonable approach.

With regards

-- 
Java: Write once, run everywhere
Perl: Write once, run away
Perl6: Don't write, it DWYW

Michal 'vorner' Vaner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <https://lists.isc.org/pipermail/bind10-dev/attachments/20130313/38a33540/attachment.bin>