httpd-modules-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neil McKee <>
Subject Re: per-worker-thread counter question
Date Sun, 03 Jul 2011 05:41:33 GMT
Trying to iterate over worker threads that may be starting and stopping seemed just too darn
hard (for me anyway).  Reading the "scoreboard" code put me off.   I ended up using atomic
operations.  In the mod_sflow module this boils down to just 3 atomic operations in the critical
path.  That's two calls to apr_atomic_inc32() to bump global counters and one apr_atomic_dec32()
to see if the transaction should be sampled (see   This way the
module doesn't know or care how many threads there are.

Although "apr_atomic_inc32(&counter)" is much more expensive than "counter++"  my understanding
is that this is still much better than having threads stall as they wait for a contested mutex.
  Someone please tell me if I got that wrong(!)

For a truly global counter you still have to aggregate across multiple unix processes.  A
shared memory segment might be the  best way to do that  but I wasn't sure if that was really
going to work with atomic operations.   In any event I found it convenient to use a pipe.
 If you open a pipe before the child-forking starts then every child process automatically
sees the same pipe and they can all send messges up to the "master" who is still reading from
the other end of it.   As long as those messages are less than PIPE_BUF in length (sometimes
512,  but usually 4096 bytes,  depending on the OS) then they are atomic and don't get interleaved.
 This way the "master" doesn't  know or care how many child processes there are.


On Jun 30, 2011, at 2:31 AM, Massimo Manghi wrote:

> I found this problem conceptually interesting and worth of generalization in many contexts.
I'd like to know if Neil got around the problem with something that could be a satisfying
> thanks
>  -- Massimo
> On 06/28/2011 04:57 AM, Neil McKee wrote:
>> Hi,
>> Here's an easy question for someone who knows their way around...
>> I want to maintain a new global counter,  but for performance reasons I am reluctant
to use a mutex or atomic_increment to update it.  I would rather maintain a separate counter
for every worker-thread,  and only accumulate the global counter when required.  (If the per-worker-thread
counter is 32-bit then I shouldn't even need a mutex when accumulating the total across all
the current threads).
>> Obviously I shouldn't just declare something as "__thread apr_int32_t mycounter;"
and mince it together as a linux-only hack.  I'd like to find the portable apr-library way
to do it.   So I think I need to find the following:
>> * - a hook that is called whenever a worker thread is started
>> * - a hook that is called whenever a worker thread is about to die
>> * - a hook to find_or_create a 32-bit integer that is private to the current worker-thread
>> * - a fn to iterate (safely) over all the current worker threads
>> It's the last one that seems particularly elusive.  I could't find an ap_ or apr_
library call that seemed to do anything like that.
>> If this has all been done before,   please can you point me to the relevant module
sources?  I think it would save me a lot of time.   Alternatively,  if you think I should
just relax and use an atomic increment instead,  then let me know.
>> Thanks!
>> Neil

View raw message