httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lazy <>
Subject Apache stuck on sendmsg() and recvmsg()
Date Thu, 30 Jun 2011 18:22:33 GMT

I'm trying to fix an issue in a custom mpm. It's called peruser. More
or less it's a prefork with pools of processes running on different
Additional pool of processes called Multiplexers is accepting
connections and sending them to workers. Each worker pool has it's own
pair of sockets (socketpair(PF_UNIX, SOCK_STREAM)) one for
Multiplexers and other for Workers. Multiplexer sends socket and
request data to Worker using blocking sendmsg(), Workers are using non

The code looks like this

in Workers
    // Don't block
    ret = recvmsg(ctrl_sock_fd, &msg, MSG_DONTWAIT);

    if (ret == -1 && errno == EAGAIN) {
        _DBG("receive_from_multiplexer recvmsg() EAGAIN, someone was faster");

        return APR_EAGAIN;
    else if (ret == -1) {
        _DBG("recvmsg failed with error \"%s\"", strerror(errno));
                return APR_EGENERAL;
    else _DBG("recvmsg returned %d", ret);

in Multiplexers

    if ((rv = sendmsg(processor->senv->output, &msg, 0)) == -1)
        ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, ap_server_conf,
                     "Writing message failed %d %d", rv, errno);
        return -1;

The problem is that sometimes Multiplexer is stuck on sendmsg(), and
Worker is stuck on recvmsg()
os is linux 2.6.32 on amd64

sendmsg(74, {msg_name(0)=NULL, msg_iov(5)=[{"y\1\0\0\0\0\0\0", 8},
{"\0\0\0\0\0\0\0\0", 8},
192}, {"GET /oglxxxxx.html HTTP/1.0\r\nHost: xxxxxxx \r\nU
ser-Agent: Mozilla/5.0 (compatible; Yahoo! Slurp;\r\nAccept:
en-us,en;q=0.5\r\nAccept-Encoding: gzip\r\nAccept-Charset:
0.7\r\n\r\n\0", 378}, {"", 0}], msg_controllen=20, {cmsg_len=20,
cmsg_level=SOL_SOCKET, cmsg_type=SCM_RIGHTS, {151}},
msg_flags=MSG_PROXY|MSG_DONTWAIT}, 0 <
unfinished ...>

Killing destination Workers frees all Multiplexers.

I think that the problem might be in receive_from_multiplexer(), if a
message gets ie half received the code isn't going back to reread
this, receive_from_multiplexer() is called after apr_pool() on
multiple Workers so there's no guarantee that the same one is going
back to reread the message, and this blocks this socket for other

I know that this is not httpd code, but perusers mailing list is dead,
and I don't have any other ideas where to go with this.

Michal Grzedzicki

View raw message