tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rainer Jung <rainer.j...@kippdata.de>
Subject Re: Apache/mod_jk serves random files from tomcat
Date Tue, 15 Jul 2008 14:08:06 GMT
dave.smith schrieb:
> Yesterday, I upgraded our dev environment to mod_jk 1.2.26, which couldn't
> have been easier.  It will probably take me a couple of days before I can
> get this done in production, though.
> 
> I terminate all HTTPS requests before they get to the web server, so from
> what you have described, it is probably safe to disable the APR connector. 
> How do I disable it, though?  I will look into disabling this after I have
> updated mod_jk in production.

Locate the tcnative shared object file (tcnative.so or tcnative-1.so) 
and renme it, so that the linker loader does not find it (e.g. add an 
underscore at the end of the file name).

Then during the next startup, Tomcat should emit an info level log 
message telling you, that it couldn't find the lib.

> Here's the full stack trace for that exception, displayed in my Tomcat logs:
> 
> Jul 10, 2008 10:06:50 PM org.apache.catalina.connector.Request
> parseParameters
> WARNING: Exception thrown whilst processing POSTed parameters
> java.io.IOException: Socket read failed
>         at
> org.apache.coyote.ajp.AjpAprProcessor.read(AjpAprProcessor.java:1037)
>         at
> org.apache.coyote.ajp.AjpAprProcessor.readMessage(AjpAprProcessor.java:1158)
>         at
> org.apache.coyote.ajp.AjpAprProcessor.receive(AjpAprProcessor.java:1090)
>         at
> org.apache.coyote.ajp.AjpAprProcessor$SocketInputBuffer.doRead(AjpAprProcessor.java:1228)
>         at org.apache.coyote.Request.doRead(Request.java:419)
>         at
> org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:265)
>         at
> org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:403)
>         at
> org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:280)
>         at
> org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:193)
>         at
> org.apache.catalina.connector.Request.readPostBody(Request.java:2400)
>         at
> org.apache.catalina.connector.Request.parseParameters(Request.java:2379)
>         at
> org.apache.catalina.connector.Request.getParameterNames(Request.java:1047)
>         at
> org.apache.catalina.connector.RequestFacade.getParameterNames(RequestFacade.java:369)
>         at
> org.apache.struts.util.RequestUtils.populate(RequestUtils.java:1225)
>         at
> org.apache.struts.action.RequestProcessor.processPopulate(RequestProcessor.java:821)
>         at
> org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:254)
>         at
> org.apache.struts.action.ActionServlet.process(ActionServlet.java:1482)
>         at
> org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:525)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>         at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>         at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
>         at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>         at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
>         at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
>         at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
>         at
> org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:444)
>         at
> org.apache.coyote.ajp.AjpAprProtocol$AjpConnectionHandler.process(AjpAprProtocol.java:472)
>         at
> org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1286)
>         at java.lang.Thread.run(Thread.java:595)

OK, thanks. You are sure it is not 5.5.26?

Regards,

Rainer

> Rainer Jung-3 wrote:
>> Hi David,
>>
>> dave.smith schrieb:
>>> Hi Rainer,
>>>
>>> Thanks a lot for the reply.
>>>
>>> I am using Tomcat 5.5.25 (rpm from jpackage.org).  CentOS Linux 2.6.18.
>>>
>>> httpd was compiled in prefork mode. The prefork settings are:
>>>
>>> StartServers       8
>>> MinSpareServers    5
>>> MaxSpareServers   20
>>> ServerLimit      256
>>> MaxClients       256
>>> MaxRequestsPerChild  4000
>>>
>>> I have setup JMeter to run against a test environment, but was unable to
>>> reproduce.  These random responses occur in production about once every
>>> week
>>> or so/more.  The problem will often (temporarily) correct itself, but
>>> sometimes I will need to restart httpd if the problem persists --
>>> restarting
>>> tomcat also works to temporarily correct the problem.
>>>
>>> The only thing strange that I see in my logs are in the test_client.log:
>>>
>>> WARNING: Exception thrown whilst processing POSTed parameters 
>>> java.io.IOException: Socket read failed
>>>         at
>>> org.apache.coyote.ajp.AjpAprProcessor.read(AjpAprProcessor.java:1037)
>>>         ...
>> Thanks for the information. What is "test_client.log"? It looks like a 
>> Tomcat log file? Could you also post a larger part of the stack, or do 
>> you only get one line?
>>
>> Would you be able to do the following two things, maybe not both at the 
>> same time:
>>
>> - disable the apr connector (tcnative.so)
>> - upgrade jk to 1.2.26
>>
>> Concerning the apr connector: If you are using OpenSSL with apr and 
>> Tomcat or you have some similar reason you really need it, then don't 
>> switch. But if you use it without a very specific reason, disabling it 
>> for a week or two would help us isolate the problem.
>>
>> Concerning mod_jk upgrade: That should be very easy, apart from the 
>> following: if your httpd uses VirtualHost in the configuration, you have 
>> to include your JkMount inside the VirtualHost, not in the global part, 
>> or you add JkMountCopy On to the VirtualHost.
>>
>> Regards,
>>
>> Rainer
>>
>>> Rainer Jung-3 wrote:
>>>> dave.smith schrieb:
>>>>>> Wow. That's weird. Is Tomcat serving the file, or is httpd serving
it?
>>>>> Not too weird.  I am experiencing the same thing with Tomcat 5.5 and
>>>>> mod_jk
>>>>> 1.2.23.  I have Tomcat serving everything.
>>>>>
>>>>> I am also using a load balancer that sends an OPTION every 2 seconds
to
>>>>> each
>>>>> web server to make sure that the server is alive.
>>>>>
>>>>> This intermittent random response issue is really killing me.
>>>> Could you please also add some info:
>>>>
>>>> Tomcat version?
>>>>
>>>> And from my previous mail:
>>>>
>>>> What's you platform and which httpd MPM (prefork orworker or something 
>>>> else) do you use? For some platforms (e.g. AIX) the detection of 
>>>> multi-threading in httpd during mpod_jk build-time was broken. Starting 
>>>> with 1.2.24 we build always including multi-thread support unless 
>>>> explicitely stated via a configure option. If you 1.2.23 build is not 
>>>> thread safe, but your httpd uses threads (like with worker mpm), then 
>>>> such trouble is possible, although more likely you would see crashes 
>>>> etc. For most platforms like Linux and Solaris the threading detection 
>>>> was OK already before 1.2.24.
>>>>
>>>> Another possible (but not very likely) cause could be bug 44494 of 
>>>> Tomcat 6.0.16/5.5.26 which under certain circumstances could leave data 
>>>> in the request object after request handling completed. You could try 
>>>> either downgrading to 6.0.15/5.5.25 or upgrading to the soon to be 
>>>> expected 6.0.17/5.5.27.
>>>>
>>>> I would also add the access log on the Tomcat side. If you find the same

>>>> phenomenon there, then it's unlikely, that httpd/mod_jk are responsible 
>>>> and the reason should be inside Tomcat or the webapp.
>>>>
>>>> Can you reproduce the problem on a test system?
>>>>
>>>> Regards,
>>>>
>>>> Rainer

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message