apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Stein <gst...@lyra.org>
Subject Re: thread locking within apr file io
Date Thu, 26 Apr 2001 19:45:13 GMT
On Thu, Apr 26, 2001 at 11:57:26AM -0700, Justin Erenkrantz wrote:
>...
> We are currently using prefork because pthread is completely useless at
> this point.  Hopefully, the signals stuff gets fixed soon...  Actually, 
> on FreeBSD, threaded MPMs are disabled, but no one tells APR, so it 
> defines APR_HAS_THREADS - we need to fix that.  I may submit a patch for
> that - just add --disable-threads to APRs configure for these platforms?

There should already be some logic here and there for disabling threads on
FreeBSD. I'm not sure whether we try to disable all threading, or just the
threaded MPM. In any case, APR's configure should detect the problem and
shut things down. Apache should then notice "hey. no threads." and disable
the selection of the threaded MPM.

>...
> Somehow there is an exclusive cross-process lock going on somewhere 
> (upon further review, we don't think it is fcntl - but it still might 
> be...).  The net result is that only one process is running at a time.

Maybe APR only has an exclusive lock available in your setup? Thus, SDBM
ends up using an exclusive lock because who knows if some *other* process
might end up trying to write to the thing while the reading is occurring.

Take a look at apr-util/dbm/sdbm/sdbm_lock.c for more info. The two
functions are called from sdbm.c. In apr_file_io.h, the SHARED lock is a
reader lock, and EXCLUSIVE is a writer lock.

As a side note: exporting the SDBM headers from apr-util has been deemed as
a mistake. Rough consensus has said to keep SDBM private and have people
only use the apr_dbm interfaces. (so we can try to avoid name conflicts on
the sdbm code)

> There seems to be a lot of lock contention going on within SDBM - we
> ONLY want read access.

Yup. But *somebody* is going to be writing to those dbms. Thus, you need
*some* kind of lock to say, "hey, I'm reading. nobody should write to the
file right now". Take a look at the APR file locking for your platform and
see what is happening.

> As Roy has pointed out, there should be very 
> little locking going on in the mainline read code ANYWHERE within Apache
> (and hence within APR).  This is why each request gets its own 
> thread/process/whatever-the-MPM-decides.  Requests shouldn't spawn
> threads themselves.  r->pool is allocated on a per-request-basis.  
> Therefore, it is pointless to lock that.  By its very nature, we
> know that we are exclusive.

That is true, and you can feel free to fix it. It is simply that nobody else
has to this point. I'd bet that we all agree with you.

Except for the bit about requests spawning threads. *That* should be doable
and should be allowed.

>...
> mod_mbox doesn't give a hoot for threads anywhere.  Actually IIRC, 
> SDBM doesn't either explicitly - but the pools, IO, and locks do.

Right. SDBM just relies on the file locking, which is provided by APR. It
doesn't care about threads because it assumes that you won't use the same
SDBM structure from two threads.

> I'm not exactly sure where to start with removing the locks.  I guess
> we could just remove ALL of the locking calls in read and pools (unless
> global pools) and see if we can get threaded APR to be closer to
> non-threaded APR.  Roy might have some ideas...  -- justin

I just looked at APR's pool locks. Basically, it is a mutex around the free
block list. The implementation sucks, though, because the locking is not
focused around access to "block_freelist", but instead to the new_block()
function on the assumption that whenever you call that, it needs the lock
for block_freelist. The "right" answer is to have new_block() manipulate the
mutex (just like free_blocks() does). There are two other accesses to
block_freelist, one is protected by the mutex, and one debug function which
isn't.

Now... given that the pools use the (global) block freelist all the time,
you can't escape the mutex without a lot of work. You can easily have
multiple threads accessing that free list.

One possible answer is to have each pool maintain a list of free blocks.
When a pool is destroyed, its blocks are returned to the *parent* rather
than a global. You can then add a flag to a pool to state whether it needs a
mutex to protect its accesses. All the subpools just operate with their
blocks, periodically requesting blocks from the parent (which may float
upwards to the global pool, who then malloc's a new block).

Overall, that should reduce lock contention, at the cost of a possible free
block imbalance -- one connection could have 1000 free blocks, and another
could have 2. With the global pool, this kind of imbalance doesn't happen.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/

Mime
View raw message