httpd-modules-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lazy <lazy...@gmail.com>
Subject Peruser-mpm race conditions
Date Mon, 28 Apr 2008 10:07:44 GMT
Hi,

Peruser mpm is based on prefork, the diference is that it can
chuid&chroot to diferent users before processing the requests.
I have been trying to get some info on the peruser mailing list but
without success.
The issue is quite simple. Many childs are left behind after a
graceful restart, sometimes they sagfault sometimes they stay until
they are killed by timeout
(this mpm enables uou to set max request time).
   Some processes lingering for fire minutes every graceful restart is
not a big issue by itself. But because of the
time limit long downloads are getting killed whitch is not acceptable.
If we disable the timeout zombie childs crash the machine in under 3
hours.

In short peruser works like this.
Some fixed number of childs called multiplexers, accepts incomming
http connections, reads requests and pass them to workers depending on
vhost configuration (user to suid and dir to chroot in to).

Each server env (many vhost may use a single server env) has own pool
of workers, which processes requests passed from multiplexers ( via a
recvmsg() on a controll socket).

Like in standard prefork mpm peruser uses poll() and if it gets some
data on input socket http socket(in case of a multiplexer) or controll
socket(in case child is a worker).

When request is read by multiplexer if passes it to worker by a
controll socket, all the workers are polling controll socket and
pipe_of_death.
If some data arives many of them try to read it. But only the fastest
one gets the data, others gets themselfs blocked on recvmsg(). When
graceful comes,
all the multiplexers get killed and there will be no more data on that
socket and they are blocked untill killed by a timeout. Thats why I
can't disable the timeout.

My idea was to make recvmsg() non blocking and in case of EAGAIN just
ignore the request (some other child must have got it first). Is this
a right way ?

Bellow some code.
used in workers:

static apr_status_t receive_from_multiplexer(
    void **trans_sock,  /* will be filled out w/ the received socket */
    ap_listen_rec *lr,  /* listener to receive from */
    apr_pool_t *ptrans  /* transaction wide pool */
)
{
...

    /* -- receive data from socket -- */
    apr_os_sock_get(&ctrl_sock_fd, lr->sd);
    _DBG("receiving from sock_fd=%d", ctrl_sock_fd);

    //before
    //ret = recvmsg(ctrl_sock_fd, &msg, 0);

  //after
    ret = recvmsg(ctrl_sock_fd, &msg, MSG_DONTWAIT);
  //after
   if(ret==-1 && (errno == EAGAIN) ) {
     _DBG("recvmsg EAGAIN, someone was faster");
      return APR_EAGAIN;
   }


//common main loop
static void child_main(int child_num_arg)
{
...

// poolset is set accordingly

while (!die_now) {
...
 for (;;) {
                apr_status_t ret;
                apr_int32_t n;

                ret = apr_poll(pollset, num_listensocks, &n, -1);
                if (ret != APR_SUCCESS) {
                    if (APR_STATUS_IS_EINTR(ret)) {
                        continue;
                    }
                    clean_child_exit(1);
                }
                /* find a listener */
                curr_pollfd = last_pollfd;
                do {
                    curr_pollfd++;
                    if (curr_pollfd >= num_listensocks) {
                        curr_pollfd = 0;
                    }
                    /* XXX: Should we check for POLLERR? */
                    if (pollset[curr_pollfd].rtnevents & APR_POLLIN) {
                        last_pollfd = curr_pollfd;
                        offset = curr_pollfd;
                        goto got_fd;
                    }
                } while (curr_pollfd != last_pollfd);

                continue;
            }
        }
    got_fd:
        _DBG("input available ... resetting socket.",0);
        sock = NULL;    /* important! */

        /* if we accept() something we don't want to die, so we have to
         * defer the exit
         */

        status = listensocks[offset].accept_func((void *)&sock,
&listensocks[offset], ptrans);
        SAFE_ACCEPT(accept_mutex_off());        /* unlock after "accept" */

        if (status == APR_EGENERAL) {
            /* resource shortage or should-not-occur occured */
            clean_child_exit(1);
        }
        else if (status != APR_SUCCESS || die_now) {
            continue;
        }

        if (CHILD_INFO_TABLE[my_child_num].type == CHILD_TYPE_PROCESSOR ||
            CHILD_INFO_TABLE[my_child_num].type == CHILD_TYPE_WORKER)
        {
          _DBG("CHECKING IF WE SHOULD CLONE A CHILD...");

          _DBG("total_processors = %d, max_processors = %d",
            total_processors(my_child_num),
            CHILD_INFO_TABLE[my_child_num].senv->max_processors);

          _DBG("idle_processors = %d, min_free_processors = %d",
            idle_processors(my_child_num),
            CHILD_INFO_TABLE[my_child_num].senv->min_free_processors);
:

Mime
View raw message