Return-Path: Delivered-To: apmail-httpd-dev-archive@www.apache.org Received: (qmail 46647 invoked from network); 3 Feb 2004 20:35:43 -0000 Received: from daedalus.apache.org (HELO mail.apache.org) (208.185.179.12) by minotaur-2.apache.org with SMTP; 3 Feb 2004 20:35:43 -0000 Received: (qmail 28837 invoked by uid 500); 3 Feb 2004 20:35:21 -0000 Delivered-To: apmail-httpd-dev-archive@httpd.apache.org Received: (qmail 28812 invoked by uid 500); 3 Feb 2004 20:35:21 -0000 Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm Precedence: bulk Reply-To: dev@httpd.apache.org list-help: list-unsubscribe: list-post: Delivered-To: mailing list dev@httpd.apache.org Received: (qmail 28781 invoked from network); 3 Feb 2004 20:35:20 -0000 Received: from unknown (HELO hqvwall01.citrix.com) (12.8.192.38) by daedalus.apache.org with SMTP; 3 Feb 2004 20:35:20 -0000 Received: from ftlpexch5im01.citrix.com ([10.9.1.140]) by hqvwall01 with tre nd_isnt_name_B; Tue, 03 Feb 2004 15:35:23 -0500 Received: by ftlpexch5im01.citrix.com with Internet Mail Service (5.5.2656.5 9)id ; Tue, 3 Feb 2004 15:35:19 -0500 Message-ID: <423019046EC9F647AF13F367FB01C158029EC9D1@ftlpexch501> From: David Pope To: "'dev@httpd.apache.org'" Subject: thread termination vs. pool lifetime Date: Tue, 3 Feb 2004 15:35:17 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2656.59) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C3EA95.3C175698" X-imss-version: 2.0 X-imss-result: Passed X-imss-scores: Baseline:48.6968 C:17 M:1 S:5 R:5 X-imss-settings: Clean:4 C:4 M:4 S:4 R:1 (0.1000 0.4000) X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C3EA95.3C175698 Content-Type: text/plain Hello all, I'm writing a module that launches a background thread that periodically wakes up to do some maintenance work. This thread is launched for the child process (all references in this email are to the winnt version of the code, but the problem space may be more general). I need to gracefully terminate this thread as the child process exits. Here's the "obvious" solution (mentioned in December in response to a similar problem): Launch the thread in child_init, and register a cleanup with pchild to terminate the thread. AFAICS this doesn't really work, due to a combination of the way pool cleanups work and the way APR threads use pools: - A pool's cleanups are run AFTER all of its child pools have been destroyed. - apr_thread_create() creates a child pool for internal use of the thread logic. - apr_thread_exit() calls apr_pool_destroy() on this pool. So the problem is that by the time the cleanup is run, the per-thread pool has already been destroyed; apr_thread_exit() tries to destroy it again, and *boom*. The first question is, did I miss something obvious that could make this work? Assuming no: I can see several ways out of this problem, but I'm very much the Apache newbie and would appreciate some advice, and perhaps some history behind why things were done the way they were. There is frequently good reasoning hidden behind hard-to-follow implementation details. Possible solutions: - Use apr_pool_parent_get() to get pconf out of pchild, using that to launch the thread; but register the termination callback with pchild, which guarantees it will run while pconf is still valid. This solution is awkward and nonintuitive, and not really future-proof. (My familiarity with non-winnt architectures is limited, but I suspect this may not even an option, if pconf lives in shared memory or otherwise has out-of-proc limitations.) - Modify apr_pool_destroy to run cleanups BEFORE destroying child pools. This would pretty much reverse the order of cleanup execution in a pool hierarchy, which is probably a Bad Thing. I assume there is a good reason why this order (destroy children before running cleanups) was chosen; can someone provide some background? - I don't really need the return value from the thread, so I could technically get away with not calling apr_thread_exit(). I don't consider this to be a real solution, and probably won't work at all for pthreads-based systems. - Add a child_shutdown hook to the MPM that is run just before pchild is destroyed, which is used instead of the pool cleanup. The fourth solution is the one I prefer; it has the following advantages: - It is symmetrical. child_init is run immediately after pchild is created, so running child_shutdown immediately before its destruction makes sense. It guarantees that the entire pchild hierarchy is valid during the hook. Since pchild was valid during child_init, its makes sense for it to be valid during child_shutdown. - It removes ordering constraints for cleanups. In my implementation I create mutexes in the same pool that I create my thread in (seems like a reasonable thing to do, especially if it's pchild). If I use a pool cleanup for thread termination, I have to make sure my thread termination cleanup is registered LAST, so it's executed FIRST. Otherwise, the mutexes my thread uses will be gone by the time it tries to acquire them. This pool cleanup approach would be bad because it relies on the internal implementation of the cleanup logic, plus it's nonintuitive. - It doesn't touch anything else in the system, and is reasonably future-proof compared to the first solution; if someone someday decides to change the way pconf fits into the system, this part of the system won't be affected. Does this sound like a reasonable solution? I apologize for the length; this is a complicated problem. -- Dave ---- David Pope Citrix Systems, Inc. ------_=_NextPart_001_01C3EA95.3C175698 Content-Type: text/html thread termination vs. pool lifetime

Hello all,

I'm writing a module that launches a background thread that periodically
wakes up to do some maintenance work.  This thread is launched for the
child process (all references in this email are to the winnt version
of the code, but the problem space may be more general).  I need to
gracefully terminate this thread as the child process exits.

Here's the "obvious" solution (mentioned in December in response to a
similar problem):  Launch the thread in child_init, and register a
cleanup with pchild to terminate the thread.

AFAICS this doesn't really work, due to a combination of the way pool
cleanups work and the way APR threads use pools:

    - A pool's cleanups are run AFTER all of its child pools have been
      destroyed.
    - apr_thread_create() creates a child pool for internal use of
      the thread logic.
    - apr_thread_exit() calls apr_pool_destroy() on this pool.

So the problem is that by the time the cleanup is run, the per-thread
pool has already been destroyed; apr_thread_exit() tries to destroy
it again, and *boom*.

The first question is, did I miss something obvious that could make this
work?

Assuming no:  I can see several ways out of this problem, but I'm very
much the Apache newbie and would appreciate some advice, and perhaps
some history behind why things were done the way they were.  There is
frequently good reasoning hidden behind hard-to-follow implementation
details.

Possible solutions:

    - Use apr_pool_parent_get() to get pconf out of pchild, using that
      to launch the thread; but register the termination callback with
      pchild, which guarantees it will run while pconf is still valid.
      This solution is awkward and nonintuitive, and not really
      future-proof.  (My familiarity with non-winnt architectures is
      limited, but I suspect this may not even an option, if pconf
      lives in shared memory or otherwise has out-of-proc limitations.)

    - Modify apr_pool_destroy to run cleanups BEFORE destroying child
      pools.  This would pretty much reverse the order of cleanup
      execution in a pool hierarchy, which is probably a Bad Thing.
      I assume there is a good reason why this order (destroy children
      before running cleanups) was chosen; can someone provide some
      background?

    - I don't really need the return value from the thread, so I could
      technically get away with not calling apr_thread_exit().  I don't
      consider this to be a real solution, and probably won't work
      at all for pthreads-based systems.

    - Add a child_shutdown hook to the MPM that is run just before
      pchild is destroyed, which is used instead of the pool cleanup.

The fourth solution is the one I prefer; it has the following advantages:

    - It is symmetrical.  child_init is run immediately after pchild
      is created, so running child_shutdown immediately before its
      destruction makes sense.  It guarantees that the entire pchild
      hierarchy is valid during the hook.  Since pchild was valid
      during child_init, its makes sense for it to be valid during
      child_shutdown.

    - It removes ordering constraints for cleanups.  In my implementation
      I create mutexes in the same pool that I create my thread in
      (seems like a reasonable thing to do, especially if it's pchild).
      If I use a pool cleanup for thread termination, I have to make
      sure my thread termination cleanup is registered LAST, so it's
      executed FIRST.  Otherwise, the mutexes my thread uses will be
      gone by the time it tries to acquire them.  This pool cleanup
      approach would be bad because it relies on the internal
      implementation of the cleanup logic, plus it's nonintuitive.

    - It doesn't touch anything else in the system, and is reasonably
      future-proof compared to the first solution; if someone someday
      decides to change the way pconf fits into the system, this part
      of the system won't be affected.

Does this sound like a reasonable solution?

I apologize for the length; this is a complicated problem.

-- Dave
----
David Pope
Citrix Systems, Inc.

------_=_NextPart_001_01C3EA95.3C175698--