tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pichels <scott.pichel...@weirminerals.com>
Subject Re: Tomcat: client disconnects?
Date Thu, 15 Jan 2009 20:20:52 GMT

Hi Chris or ?,

I appreciate the response - here goes with my answers and and more
questions!



Christopher Schultz-2 wrote:
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> SP,
> 
> pichels wrote:
>> We are experiencing users having issues connecting to our RH server and
>> getting disconnected with errors in their browsers that state there is a
>> DNS
>> error or page cannot be displayed.
> 
> Is this a browser error message, or something coming from the Tomcat side?
> 
> Browser message - "Page cannot be displayed" - in Firefox or IE6/7 - even
> using the IP address bypassing DNS of the Web server.
> 
> 
>> Users connect to Apache then Tomcat serves up our JSP's that sometimes
>> accesses specific MySQL server DB's.
> 
> Are you using a connection pool? If so, what is the configuration? If
> not, how are you connecting to the database (and is there a limit on the
> number of db connections you allow)?
> 
> 
> No - Pooling. JDBC connector. No limit.
> 
> 
>> Most of the pages seems to be index.jsp or index.php lately for both
>> websites(These are websites internal - our company web servers).
> 
> MediaWiki uses index.php as it's front-controller, so it's no surprise
> that you'd be getting many errors with that page in particular (because
> requests always go through that page). Is index.jsp similar to index.php
> in that it handles a great percentage of your servlet-server traffic?
> 
> 
> Yes, for Mediawiki we are setup to use index.php - understood. 
> With Tomcat we don't use index.jsp as a front-controller for servlet
> traffic.
> 
> 
> 
>> In Tomcat: it is a mess of exceptions that our developers try and keep
>> cleaned up. ( We do not use Log4j)
> 
> Would you care to post some of those? What are the most popular
> exceptions? What seem to be the worst-sounding?
> 
> 
> Most Popular below:
> 
> WARNING: Error sending end packet 
> java.net.SocketException: Broken pipe 
>         at java.net.SocketOutputStream.socketWrite0(Native Method) 
>         at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92) 
>         at java.net.SocketOutputStream.write(SocketOutputStream.java:136) 
>         at org.apache.jk.common.ChannelSocket.send(ChannelSocket.java:537) 
>         at
> org.apache.jk.common.JkInputStream.endMessage(JkInputStream.java:127) 
>         at org.apache.jk.core.MsgContext.action(MsgContext.java:305) 
>         at org.apache.coyote.Response.action(Response.java:183) 
>         at org.apache.coyote.Response.finish(Response.java:305) 
>         at
> org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:205) 
>         at
> org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283) 
>         at
> org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773) 
>         at
> org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703) 
>         at
> org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895) 
>         at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689) 
>         at java.lang.Thread.run(Thread.java:595) 
> 
> 
> 
>> In Mysql: there are lots of slow queries I've been logging - > 1 sec.
> 
> Are these mostly coming from your webapp or MediaWiki? Have you
> investigated these? Often, slow queries arise from inadequate or stale
> indexes. Try running REPAIR TABLE [tablename] on a table that sees a lot
> of slow queries. That should update the indexes in case you recently
> bulk-loaded a lot of data. If that's not the problem (and it's unlikely
> that it /is/ the problem), you should start running EXPLAIN on some of
> those queries to see if you have appropriate indexes on your tables.
> 
> 
> 
> Mostly coming from Tomcat/webapp. Yes, one we fixed but we are still
> having issues w/ slow queries.
> Ok, we'll look into running MySQL Explain.
> 
> 
> 
> 
>> The errors that occur are random browser disconnects - urls that end or
>> start with *.jsp/ or *.php/ pages.
> 
> Something like SocketException: Connection closed? If that's the case,
> the exceptions themselves are not a problem, but could be indicative of
> a larger problem: poor performance causes people to hit STOP on their
> browsers and re-try requests.
> 
> 
> Yes, see above for SocketExceoptions.
> Users are probably not hitting the Stop button - usually they refresh and
> most of the time the page reloads OK.
> We are looking at MySQL Performance.
> 
> 
> 
>> We have also tried using WinXP host files and using an IP address to
>> bypass
>> DNS altogether - no luck.
> 
> Are you actually performing DNS lookups?
> 
> Yes, with all the other users/clients.
> We have a pilot "test" group of users only using the IP address in the URL
> box of their browsers.
> 
> Two things might be happening if DNS is really your problem:
> 
> 1. Your DNS server is slow. Mitigations are a) get a
>    faster/closer/extra DNS server or b) disable DNS lookups
> 
> Maybe - perhaps our Active Directory DNS serves up a alot of requests from
> our users and servers.
> 
> 
> 2. Your DNS settings are misconfigured (e.g. wrong ip address,
>    etc.). Check your DNS configuration.
> 
> 
> Definitely not misconfiguration.
> 
> 
>> Problem around 4:15PM - Jan 9, 2009:
>> http://10.97.24.23/Sales/IndentedATPServlet?partNum=086EXUCCCXMA&qty=1&warehouse=M
> 
> What was the problem?
> 
> User was viewing a Servlet page that was querying the AS400 DB2 DB on the
> backend for a part number.
> Tomcat may have been waiting for a long time and then teh user recieved
> "page cannot be displayed" or "Server cannot be found" in IE7 browser.
>  
> 
>  http://www.nabble.com/file/p21485970/dns.jpeg 
> 
> 
> NOTE: The connection is thru Tomcat JDBC connector to AS400 DB2 DB.
> 
> 
>> Problem around 5:05PM - Jan 9, 2009:
>> http://wsi/Qoe/index.jsp 
>> (same thing....Page Cannot Be Displayed)  
> 
> Again... what was the problem? Look in catalina.out or some other log
> file in your Tomcat's logs directory.
> 
> We have Apache & Tomcat Debugging set very high. And have MySQL slow
> queries and error logs.
> Can't find any relevant to error. 
> Can we set Tomcat debugging with date/time stamps or will Log4j help more
> with debugging?
> 
> 
>> Our MysqL server has a tendency to ramp up or the load avg increases
>> using
>> massive CPU cycles - I restart Tomcat and the system usually calms down -
>> Mysql included.
> 
> This is why I was asking about connection pooling: it looks like Tomcat
> is murdering your DB. This could be due to too many connections to
> MySQL, overly high query volume, or sloppy code in your application
> leaking database resources.
> 
> 
> Java Developer is looking into code constantly and cleaning up code.
> He agreed that there is old sloppy code perhaps and possibly too many
> connections.
> Can we control the connections or help diagnose code errors in an easier
> way? 
> 
> 
>> We are currently using tcpdump on the server & Wireshark on the affected
>> clients to look at the packet level of my connections to the webserver.
> 
> Have you observed anything interesting using these tools?
> 
> 
> I've sent the logs to RHEL(RedHat Linux) support and they have been
> reviewing and think we have a network issue - router or VMware problems
> since this is a 
> 
> VMware guest.
> However, our clients are on a flat subnet with the server.
> Clients = 10.97.25.X
> Server = 10.97.24.23
> 
> I was also told by another tech at RHEL support that RHEL5 runs fine on
> WMWaare as is if it were a physical box - confusing.
> 
> 
> 
>> I know more details are probably needed - this is a complex problem.
>> And only seems to happen in the am right when users login to their
>> machines
>> and start up their browsers and then happens at the end of the day as
>> well.
> 
> Maybe you are just getting more peak traffic than your setup can handle.
> Have you performed any formal load testing?
> 
> 
> Perhaps we have too much traffic or our applications need to be further
> tuned or reinstalled from source code.
> I am using the RHEL5 RPM's from RH Network.
> 
> I see there are many ways to load test" MySQL - do you have any
> suggestions on where I can start?
> A free/Open source tool would be preferred. <smile>
> 
> 
> Thanks for all your questions and insight!
> 
> -SP
> 
> 
> 
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iEYEARECAAYFAklvYmUACgkQ9CaO5/Lv0PClcQCeOid/7wFe7/JvwpQHUE96xYft
> bagAn1MC7Rz+At5JitXUA8R4GHO/8BMf
> =Lxrm
> -----END PGP SIGNATURE-----
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Tomcat%3A-client-disconnects--tp21424139p21485970.html
Sent from the Tomcat - User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message