apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Bannert <aa...@clove.org>
Subject Re: [proposal] apr_thread_setconcurrency()
Date Sat, 15 Sep 2001 23:43:39 GMT
On Fri, Sep 14, 2001 at 06:33:47PM -0700, Justin Erenkrantz wrote:
> On Fri, Sep 14, 2001 at 04:21:51PM -0700, Aaron Bannert wrote:
> > Why would this circumvent the OS scheduler at all? In all cases it
> > is a *hint*. Please be more precise.
> > 
> > I think I showed you an example awhile ago where compute-bound threads
> > behave drastically different depending on the operating system. In
> > the case of solaris, a computationally intensive thread that makes no
> > system calls* will not automatically yield a mutex when entering/exiting
> > a critical section, unless pthread_setconcurrency() is called.
> That statement isn't necessarily correct.  What actually happens
> is that the user scheduler in Solaris never gets executed because
> none of the entry points as defined by the OS (i.e. system calls)
> get executed to trigger the user scheduler's activity during the
> compute-bound function call.  It isn't that it doesn't yield the
> mutex - it is that there is no other thread to yield to as the 
> scheduler on Solaris gives the thread a chance to run *before*
> launching the next thread.  This is a conscious decision on Sun's 
> part when designing their scheduler for Solaris (up to but not 
> including 9).

This may be a more accurate description, but I think we're talking
about the same thing here.

> If you create too many LWPs, you will lose a lot of optimizations 
> that are present in Solaris (i.e. handover of a mutex to another 
> thread in the same LWP - as discussed with bpane on dev@httpd 
> recently).

Of course, and that is something the caller needs to take into consideration.
I'm not forcing you to use it, I just think it needs to be available.

>             If you don't create enough LWPs, you may enter a 
> condition where the scheduler refuses to balance the processes 
> correctly (it also acts as a ceiling).

In consideration of your statement here I spend some time reading
the Solaris 8 libpthread source. On that platform your statement
here is false. Calling pthread_setconcurrency (or thr_setconcurrency
for that matter) can only change the number of multiplexed LWPs in
two ways: either not at all, or by increasing the number. I see
no way that it acts as a ceiling.

For the curious, the code in question begins at:
(pthread_* is built on top of thr_*)

>                                         0 lets the OS determine
> the concurrency (on Solaris).

Again, on solaris 8, calling pthread_setconcurrency(0) has absolutely
no effect on the number of LWPs (which, I might add, is what it states
on the man page).

> By setting a value, you are attempting to circumvent the OS 
> scheduler.  If you ask it to set the concurrency on Solaris, it 
> *will* create enough LWPs to equal that concurrency (as you
> create threads to be paired with LWPs).

"I do not think that means what you think it means."

I still don't know what you mean by "circumvent the OS scheduler",
but the second statement is correct, and my point is that is exactly
what I want it to do.

>                                          This is not a hint, but a 
> command.  (Yes, the man page for Solaris says that it is a hint, 
> but it treats it as a command.)

Sorry, but that's just BS, and I don't know where you get off making such
bold unfounded statements. Please just go read the source, they match
the man pages.

> Talking about other OSes besides Solaris is moot because they don't 
> implement a M*N scheduling strategy.  With a bound thread 
> implementation, pthread_setconcurrency is a no-op (what else can
> it do?).  It can only be effective in the case of a LWP-like 
> (multiplexing a kernel thread) scheduling strategy.

Great. This is what I said at the start of this thread. So do you have
a good reason to keep it out of APR or not?

> Furthermore, I think that any values that you may pass into
> pthread_setconcurrency are inherently wrong.  What values will
> you use to set this?  The number of threads?  The number of CPUs?
> Let the programmer decide?  Let the user decide?  IMHO, all of these 
> are bad choices:

[self-fulfilling answers omitted]

> - Let the programmer decide.  Awfully bad choice.  Who knows
> how the system is setup?  What are you optimizing for?

This is the only choice I proposed, I don't know what the heck you are
arguing about in these other things. Of course let the programmer decide,
that's why it's an API!

I just gave you an example where I would use it: in the worker MPM.
In that case it would be the number of simultaneous requests I expect
to serve.

> So, what do I think the correct solution is?  Let the OS decide
> (exactly what it does now).  The OS has access to much better 
> information to make these decisions (i.e. load averages, I/O wait, 
> other processes, num CPUs, etc.).  The goal of the OS is to balance 
> competing processes.  Circumventing the OS scheduler by forcing it 
> to create too many or too few LWPs is the wrong thing.

Oh, just stop that. You can't keep saying "circumventing the OS scheduler"
when that doesn't mean anything! You surely don't mean "somehow getting
around the process scheduler", so just quit it! We're not hacking the
kernel here, we're using fully published POSIX APIs!

> The case of a compute-bound thread merely falls into a specific 
> trap on a specific OS with a specific thread model.  This case
> is typically evident in benchmarks not the real-world.  Most
> applications will enter a system call at *some* point.
> > In a practical sense, when I was playing with the worker MPM I noticed
> > that under high load (maxing out the CPU) it took on the order of 10
> > seconds** for the number of LWPs to stablize.
> I'll live with that - this is due to inherent OS scheduler 
> characteristics.  After 10 seconds, the system stabilizes - the 
> OS has performed its job.  Is there any evidence that this value
> that it stabilized at is incorrect?  What formula would you have
> used to set that number?  Any "hint" that we may give it may end 
> up back-firing rather than helping.

There you go again with this "OS scheduler" thing that I've never heard
of. 10 seconds to stabilize is rather long when you consider I have
already served O(5000) requests.

> In fact, the best solution may be to provide a configure-time 
> option to help the user select the "right" thread model on
> Solaris (i.e. /usr/lib/libthread.so or /usr/lib/lwp/libthread.so).
> You can recommend using the "alternative" thread model for
> certain types of compute-bound applications.  (However, be careful
> on Solaris 9 as they are reversed.)  -- justin

The configure-time option you're talking about is LDFLAGS. You can
also do it at runtime with LD_PRELOAD.


View raw message