tomcat-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Schwartz <alexander.schwa...@gmx.net>
Subject failover-problem and session mixup: jakarta-tomcat-connectors: jk_ajp_common.c
Date Mon, 26 Jan 2004 21:04:58 GMT
Hello Tomcat-Developers,

we are currently experiencing some problems with mod_jk with
loadbalancing: 

   (A) when a post-request fails while receiving data from tomcat, the
loadbalancer tries to send the request to the other tomcat, but
"forgets" the post body content (i.e. login and password submitted by
the user). The size of the body content is only a few bytes (< 100), so
it's not the "known" problem with bodies longer than 8k. 

   (B) it seems that sometimes after a POST-failover body data from an
old request is sent to the other tomcat when the first connection
failed. This leads to a session mix-up & user mix-up!

The setup is a simple cluster configuration: one apache httpd with
mod_ssl in front, mod_jk with one configured loadbalance worker with
sticky sessions and two child ajp13 workers in the middle, two tomcats
in the back. 

Versions involved: mod_jk 1.2.4, Tomcat 4.1.27 (but also tested with
mod_jk 1.2.5 and Tomcat 4.1.29). Apache on AIX, version 1.3.29. The
latest version of jk_ajp_common.c in CVS don't seem to have relevant
changes. We have searched the mailinglists, the only source I have found
is in tomcat-user-mailinglist where Jean-Philippe Belanger describes a
similar problem in the mails from 14 Jan 2004 (the empty POST body) and
12 Jan 2004 (the empty POST body and the session mixup). He didn't
investigate further but got himself a cisco load balancer. Another post
is here, but I found no replies/solutions.

http://www.mail-archive.com/tomcat-dev@jakarta.apache.org/msg48833.html

(A) can be reproduced quite easily using "netcat" as a simulator of the
first tomcat and a second real tomcat. If you kill netcat after it has
accepted a POST-connection from a mod_jk, mod_jk will forward the
request to the other tomcat without the POST-body 

But we can't reproduce (B), but it has definitely happened. We suspect
that POST-Recovery code might send an old request body while executing
the following statements. Please note that the log statement refers to
op->reply while the following if statement refers to op->post! (this is
cut and paste from the orginal source): 

   jk_log(l, JK_LOG_DEBUG,
     "ajp_send_request 2: "
     "request body to send %d - request body to resend %d\n",
     ae->left_bytes_to_send, jk_b_get_len(op->reply) - AJP_HEADER_LEN);

   if (jk_b_get_len(op->post) > AJP_HEADER_LEN) {
       if(!ajp_connection_tcp_send_message(ae, op->post, l)) {
           jk_log(l, JK_LOG_ERROR, "Error resending request body\n");
           return JK_FALSE;
       }

As we think that POST-recovery is buggy, we tried to disable
POST-recovery completly. This way our version is a bit more pessimistic
about failover, but it seems to work. We don't understand why mod_jk
should try to recover after it i.e. already received data from the
tomcat -- it should assume that the transaction has already been
committed and shouldn't issue another one. 

We are still worried about the session mixup, the case when mod_jk
resent a header with a different (old) body. We can't reproduce it and
therefore we hesitate to use mod_jk in a production environment. 

Can you help reproducing this bug and/or fixing POST-recovery? We have
attached the original source, the patched source and the patch.

Thanks in advance, 

Alexander Schwartz.
 
-- 
Alexander Schwartz (alexander.schwartz@gmx.net)
http://www.ahus1.de

Mime
View raw message