axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jay25 <jshr...@yahoo.com>
Subject Re: [jira] Commented: (AXIS2-3670) Problem using REUSE_HTTP_CLIENT - Application stalls
Date Fri, 01 May 2009 19:22:23 GMT

Thanks a lot for taking time during your vacation for this issue. Really
under lot of pressure to get this going. Yes, I could see that the generated
stub has
_messageContext.getTransportOut().getSender().cleanup(_messageContext); just
before send response.

I made little more progress on this issue.  
My settings: Java 1.5, Axis2 1.3, default values for axis settings - number
of connections to host, timeout.
This is what is happening:
default timeout for connection is getting set to 60000 and socket timeout to
30000. When the server takes more time to respond (I simulated this by
adding wait(70000) in service code ), axis fault exception is thrown: Also,
thread dump shows thread in stuck state; netstat shows some connections
lingering in close_wait for about 10 minutes. Looks like after the timeout
the connections or threads are not getting released to the connection pool.
So subsequent requests keep waiting.

 "xml version='1.0' encoding='UTF-8'?><soapenv:Envelope
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><soapenv:Fault><faultcode>soapenv:Client</faultcode><faultstring>current
thread not owner</faultstring><detail><Exception>org.apache.axis2.AxisFault:
current thread not owner&#xd;[\n]">
 "[0x9]at
org.apache.axis2.AxisFault.makeFault(AxisFault.java:381)&#xd;[\n]">
 "[0x9]at
com.rsa.pso.aa.services.AccountServiceMessageReceiverInOut.invokeBusinessLogic(AccountServiceMessageReceiverInOut.java:101)&#xd;[\n]">
 "[0x9]at
org.apache.axis2.receivers.AbstractInOutSyncMessageReceiver.receive(AbstractInOutSyncMessageReceiver.java:39)&#xd;[\n]">
 "[0x9]at
org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:144)&#xd;[\n]">
 "[0x9]at
org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:279)&#xd;[\n]">
 "[0x9]at
org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:116)&#xd;[\n]">
 "[0x9]at
javax.servlet.http.HttpServlet.service(HttpServlet.java:710)&#xd;[\n]">
 "[0x9]at
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)&#xd;[\n]">
 "[0x9]at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)&#xd;[\n]">
 "[0x9]at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)&#xd;[\n]">
 "[0x9]at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)&#xd;[\n]">
 "[0x9]at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)&#xd;[\n]">
 "[0x9]at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)&#xd;[\n]">
 "[0x9]at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)&#xd;[\n]">
 "[0x9]at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:548)&#xd;[\n]">
 "[0x9]at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)&#xd;[\n]">
 "[0x9]at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)&#xd;[\n]">
 "[0x9]at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:874)&#xd;[\n]">
 "[0x9]at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)&#xd;[\n]">
 "[0x9]at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)&#xd;[\n]">
 "[0x9]at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)&#xd;[\n]">
 "[0x9]at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)&#xd;[\n]">
 "[0x9]at java.lang.Thread.run(Unknown Source)&#xd;[\n]">
 "Caused by: java.lang.IllegalMonitorStateException: current thread not
owner&#xd;[\n]">
 "[0x9]at java.lang.Object.wait(Native Method)&#xd;[\n]">
 "[0x9]at com.xx.HelloWorld.hello(HelloWorld.java:49)&#xd;[\n]">
 "[0x9]at
com.xx.HelloWorld.HelloWorldMessageReceiverInOut.invokeBusinessLogic(HelloWorldMessageReceiverInOut.java:48)&#xd;[\n]">
 "[0x9]... 21 more&#xd;[\n]">
 "</Exception></detail></soapenv:Fault></soapenv:Body></soapenv:Envelope>">



Hi Jayashree,
you can check if:

_messageContext.getTransportOut().getSender().cleanup(_messageContext);

is really invoked. AFAI can remember there is an issue when an exception is
thrown into the stub. If this is the case the
_messageContext.getTransportOut().getSender().cleanup(_messageContext); is
not invoked.

Sorry for not being really responsive, but I am on a vacatio and will be
back on 7th of May.

hope this helps

Regards,
Dobri


On Wed, Apr 29, 2009 at 11:49 PM, jay25 <jshreek@yahoo.com> wrote:

>
> I am facing the same issue in production right now.  The axis stub already
> has cleanTransport --
> _messageContext.getTransportOut().getSender().cleanup(_messageContext);
>
> We have set reuse_option to true. Rest of values are set to default..
> Greatly appreciate help...
>
> The thread dump shows threads waiting like this:
> at java.lang.Object.wait(Native Method)
> at
>
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.doGetConnection(MultiThreadedHttpConnectionManager.java:509)
> at
>
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.getConnectionWithTimeout(MultiThreadedHttpConnectionManager.java:394)
> at
>
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:152)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
> at
>
> org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:558)
> at
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:176)
> at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:73)
> at
>
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:305)
> at
>
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:201)
> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:452)
> at
>
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:330)
> at
>
> org.apache.axis2.description.OutInAxisOperationClient.execute(OutInAxisOperation.java:294)
> at com.xxx.HelloStub.hello1(HelloStub.java:173)
>

-- 
View this message in context: http://www.nabble.com/-jira--Created%3A-%28AXIS2-3670%29-Probelm-using-REUSE_HTTP_CLIENT-tp16331279p23338385.html
Sent from the Axis - Dev mailing list archive at Nabble.com.


Mime
View raw message