"the problem with threads"

[Brad Knowles]

At 4:56 PM +0000 2006-09-07, Paul Vixie quoted Shane Kerr:

>>  - - All concurrent programs are provably bad
>
>  that's not the way i read this article.  but speaking from bind9's 
>experience,
>  actually getting useful work out of N processors is very much more difficult
>  than "use threads".

Parallelism is hard.  Closely coupled IPC is hard.  The 
multiprocessor super computer guys can talk about those problems at 
length.  There are a surprising number of problems that do not 
parallelize very well, and I don't think it matters too much what 
your programming model is, although some do make solving this kind of 
problem harder than others do.

I think the mistake that a lot of people make is that they blindly 
throw solutions at a problem to see what sticks, and these solutions 
frequently amount to whatever the latest fad is to patch some old 
programming method.

I am not convinced that threads are inherently bad.  I am convinced 
that they are hard for many people to use, and very hard for most 
people to use correctly -- even in those situations where they are 
actually appropriate.

Of course, I'm not qualified to tell you which kinds of situations 
fall into which categories, or which type of people fall into which 
categories.  I'm just extrapolating from what I've seen in the 
various projects I've been involved with, and multithreaded 
programming seems to be something that trips up a hell of a lot of 
people, even a lot of people that I would otherwise consider to be 
very good programmers.


And this multicomputing problem just gets worse, as modern processors 
and operating systems get more complex.

>>  - - We don't really know what else to do
>
>  i think the VLIW people believe that they know what else we should do :-).
>  i do not myself know what else we should do, unless it's the apache fork()
>  model.

I've been a fan of the KISS principle for a long time, and I think 
the apache/postfix approach of having multiple cooperating 
intercommunicating processes (which are also mutually distrustful and 
operate at least possible privilege) may be one of the best 
real-world compromises that I have come across.

However, I'm not sure that I understand how even this model can be 
applied to many different types of problems, especially ones like 
those faced by named or ntpd, both of whom have to operate on 
potentially forged UDP packet data, and need to operate in a manner 
somewhat like many near real-time programs.


At this point, I can recognize some issues, and I can recognize some 
models that I have encountered which seemed to work at least 
reasonably well in certain circumstances in the past, but that's 
about the best I can do.

I was hoping that you folks would have some answers for me.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.