httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manoj Kasichainula <>
Subject Re: request for comments: multiple-connections-per-thread MPM design
Date Tue, 26 Nov 2002 04:36:59 GMT
On Mon, Nov 25, 2002 at 07:12:43AM -0800, Brian Pane wrote:
> On Mon, 2002-11-25 at 00:20, Manoj Kasichainula wrote:
> > I was actually wondering why the reader and writer were seperate
> > threads.
> It was a combination of several factors that convinced me
> to make them separate:
> * Take advantage of multiple CPUs more easily

Yeah, but as you noticed, once you get more than 2 CPUs, you have the
same problem.

I'm just guessing here, but I imagine most CPU effort wouldn't be
expended in the actual kernel<->user transitions that are polls and
non-blocking I/O.  And the meat of those operations could be handled by
other CPUs at the kernel level. So that separation onto multiple
CPUs might not help much.

> * Reduce the number of file descriptors that each poll call
>   is handling (important on platforms where we don't have
>   an efficient poll mechanism)

Has anyone read or benchmarked whether 2 threads polling 500 fds is
faster than 1 thread polling 1000?

> > For Linux 2.6, file notifications could be done entirely in userland in
> > the case where no blocking is needed, using "futexes".
> Thanks!  I'll check out futexes.

Note that futexes are just Fast User mUTEXES. Those are already in the
kernel (according to some threads I read yesterday anyway). But I
beleive the part about file notification using them is still in

> > But if you want to avoid the extra system calls, you could put a mutex
> > around maintenence of the pollset and just let the various threads dork
> > with it directly.
> > 
> > I do keep mentioning this mutex around the select/poll :). Is there a
> > performance reason that you're trying to avoid it? In my past skimmings,
> > I've seen you post a lot of benchmarks and such, so maybe you've studied
> > this.
> The real reason I don't like the mutex around the poll is that
> it would add too much latency if we had to wait for the current
> poll to complete before adding a new descriptor.  When the
> Listener accepts a new connection, or a Request Processor creates
> a new response brigade, it needs to get the corresponding socket
> added to the pollset immediately, which really requires interrupting
> the current poll.

Hmmm. That's a problem that needs solving even without the mutex though
(and it affects the design I proposed yesterday as well).  When you're
adding a new fd to the reader or writer, you have to write to a pipe or
send a signal. The mutex shouldn't affect that. 

BTW, ISTR Ryan commenting a while back that cross-thread signalling
isn't reliable, and it scares me in general, so I'd lean towards the

I'm pondering what else could be done about this; having to muck with a
pipe doesn't feel like the right thing to do. Perhaps I should actually
look at other people's code to see what they do. Other designs have
threads for disk I/O and such, so there should be a way. I believe
Windows doesn't have this problem, or at least hides it better, because
completion ports are independent entities that don't interact with each
other as far as the user is concerned.

View raw message