tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hannaoui, Mo" <Mo.Hanna...@nielsen.com>
Subject RE: mod_jk 1.2.28 errors
Date Thu, 28 Oct 2010 07:27:49 GMT

André, thanks for the suggestions.

I removed all the time outs except for connection_pool_timeout which I set to 120. I changed
the connectionTimeout to 120000 and maxThreads to 200 in server.xml. Things are running much
smoother now than before. I still see the error messages but less frequently than before.
I used to see the messages every 1 to 2 minutes, now it is between 5 minutes and 1 hour. 

Where to go from here? How do I determine which timeout needs to be adjusted?

Another more serious issue that I have not solved is the sudden surge of http connections,
from ~30 to over 200 in a matter of few seconds causing all kind of read timeout and service
unavailable errors. 

Any suggestions to how to go about fixing or determining the root cause of either of the two
issues?

Here are the latest configurations:

 --- workers.properties-----
worker.list=loadbl,status
worker.template.port=8009
worker.template.type=ajp13
worker.template.lbfactor=1

worker.template.ping_mode=A
worker.template.connection_pool_timeout=120
worker.worker1.reference=worker.template
worker.worker1.host=jboss_server1
worker.worker2.reference=worker.template
worker.worker2.host= jboss_server2
worker.loadbl.type=lb
worker.loadbl.balance_workers=worker1,worker2
worker.loadbl.sticky_session=True
worker.status.type=status

----------------------------

---- server.xml ------
	<!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="8009" address="${jboss.bind.address}" protocol="AJP/1.3"
         emptySessionPath="true" enableLookups="false" redirectPort="8443" 
        maxThreads="200" connectionTimeout="120000"/>

---------------------


---------- error messages ----------
 [Thu Oct 28 05:51:57 2010][19938:3086886672] [info] ajp_service::jk_ajp_common.c (2540):
(worker1) sending request to tomcat failed (unrecoverable), because of client write error
(attempt=1)
[Thu Oct 28 05:51:57 2010][19938:3086886672] [info] service::jk_lb_worker.c (1388): service
failed, worker worker1 is in local error state
[Thu Oct 28 05:51:57 2010][19938:3086886672] [info] service::jk_lb_worker.c (1407): unrecoverable
error 200, request failed. Client failed in the middle of request, we can't recover to another
instance.
[Thu Oct 28 05:51:57 2010][19938:3086886672] [info] jk_handler::mod_jk.c (2611): Aborting
connection for worker=loadbl
[Thu Oct 28 06:03:06 2010][27490:3086886672] [info] ajp_process_callback::jk_ajp_common.c
(1882): Writing to client aborted or client network problems
[Thu Oct 28 06:03:06 2010][27490:3086886672] [info] ajp_service::jk_ajp_common.c (2540): (worker1)
sending request to tomcat failed (unrecoverable), because of client write error (attempt=1)
[Thu Oct 28 06:03:06 2010][27490:3086886672] [info] service::jk_lb_worker.c (1388): service
failed, worker worker1 is in local error state
[Thu Oct 28 06:03:06 2010][27490:3086886672] [info] service::jk_lb_worker.c (1407): unrecoverable
error 200, request failed. Client failed in the middle of request, we can't recover to another
instance.
[Thu Oct 28 06:03:06 2010][27490:3086886672] [info] jk_handler::mod_jk.c (2611): Aborting
connection for worker=loadbl
[Thu Oct 28 06:38:49 2010][25752:3086886672] [info] ajp_handle_cping_cpong::jk_ajp_common.c
(879): timeout in reply cpong
[Thu Oct 28 06:38:51 2010][25752:3086886672] [info] ajp_send_request::jk_ajp_common.c (1518):
(worker1) failed sending request, socket -1 prepost cping/cpong failure (errno=110)
[Thu Oct 28 06:38:51 2010][25752:3086886672] [info] ajp_send_request::jk_ajp_common.c (1574):
(worker1) all endpoints are disconnected, detected by connect check (0), cping (1), send (0)


Thanks,
-mo
-----Original Message-----
From: André Warnier [mailto:aw@ice-sa.com] 
Sent: Tuesday, October 26, 2010 3:15 AM
To: Tomcat Users List
Subject: Re: mod_jk 1.2.28 errors

Pid wrote:
> On 26/10/2010 00:05, Hannaoui, Mo wrote:
...

>>
>> worker.template.ping_mode=A
>>
>> worker.template.reply_timeout=30000
>> worker.template.socket_connect_timeout=10000
>> worker.template.socket_timeout=10
>> worker.template.connection_pool_timeout=600
> 
> I can't get to the jk docs docs at the moment, but that socket_timeout
> seems a little low.  Are those the defaults?
> 
> 
1) What happens when you just leave the line

 >> worker.template.ping_mode=A

and *remove* all the other timeout-related lines (to let the defaults be configured) ?


2) About the Tomcat-side configuration :

<!-- Define an AJP 1.3 Connector on port 8009 -->
 >
 >     <Connector port="8009" address="${jboss.bind.address}"
 > protocol="AJP/1.3"
 >
 >          emptySessionPath="true" enableLookups="false"
 > redirectPort="8443"
 >
 >         maxThreads="800" connectionTimeout="600000"/>
 >

You have MaxClients=250 at the Apache side, and maxThreads=800 at the Tomcat side (2 
times, because 2 Tomcats).  Unless each Apache client can issue several requests to 
Tomcat(s) at the same time, that seems a bit unbalanced.
Also, connectionTimeout="600000" means that when a client makes a connection but does not

send a request on it, Tomcat is going to keep a thread busy, waiting 600 seconds (10 
minutes) until the client deigns sending something.
After you change it, you should then go back to the explanation of connection_pool_timeout

in http://tomcat.apache.org/connectors-doc/reference/workers.html, to resynchronise that side.

Better yet probably, restart from the default values for everything, and start modifying 
from the defaults only if you really have a problem.  The default values are chosen 
sensibly, for a range of situations.  Playing around with them usually makes things worse

rather than better.
I would leave the
worker.template.ping_mode=A
as it is however.



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message