Any reason not to split perl filtering?

[Russ Allbery]

bill davidsen <davidsen at tmr.com> writes:

> Single threaded innd is simply not keeping up with the incoming volume,
> and the obvious trouble spot is the perl filter. I have one CPU 98% used
> and the rest idle, is there any reason not to move filtering into a
> separate process to use some extra power. Looks cheaper than buying a
> new system :-(

> Has anyone done this, and if so are there records of how it worked or
> why it didn't?

> The actual reading of the articles is the next target, but the perl
> filter takes way too much CPU!

You take a fairly sizable I/O performance hit by doing this since you have
to start talking back and forth with a separate process (hence going
through the kernel).  Whether that's better or not depends a lot on the
system configuration.

The other obvious problem is that no one's submitted the code for it yet.

I have plans at some point to redo the filtering setup to provide a
cleaner API, which would make this much easier, but I have no idea when
I'll find the time.

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>

    Please send questions to the list rather than mailing me directly.
     <http://www.eyrie.org/~eagle/faqs/questions.html> explains why.