Hi Chris or ?,
I appreciate the response - here goes with my answers and and more
questions!
Christopher Schultz-2 wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> SP,
>
> pichels wrote:
>> We are experiencing users having issues connecting to our RH server and
>> getting disconnected with errors in their browsers that state there is a
>> DNS
>> error or page cannot be displayed.
>
> Is this a browser error message, or something coming from the Tomcat side?
>
> Browser message - "Page cannot be displayed" - in Firefox or IE6/7 - even
> using the IP address bypassing DNS of the Web server.
>
>
>> Users connect to Apache then Tomcat serves up our JSP's that sometimes
>> accesses specific MySQL server DB's.
>
> Are you using a connection pool? If so, what is the configuration? If
> not, how are you connecting to the database (and is there a limit on the
> number of db connections you allow)?
>
>
> No - Pooling. JDBC connector. No limit.
>
>
>> Most of the pages seems to be index.jsp or index.php lately for both
>> websites(These are websites internal - our company web servers).
>
> MediaWiki uses index.php as it's front-controller, so it's no surprise
> that you'd be getting many errors with that page in particular (because
> requests always go through that page). Is index.jsp similar to index.php
> in that it handles a great percentage of your servlet-server traffic?
>
>
> Yes, for Mediawiki we are setup to use index.php - understood.
> With Tomcat we don't use index.jsp as a front-controller for servlet
> traffic.
>
>
>
>> In Tomcat: it is a mess of exceptions that our developers try and keep
>> cleaned up. ( We do not use Log4j)
>
> Would you care to post some of those? What are the most popular
> exceptions? What seem to be the worst-sounding?
>
>
> Most Popular below:
>
> WARNING: Error sending end packet
> java.net.SocketException: Broken pipe
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
> at org.apache.jk.common.ChannelSocket.send(ChannelSocket.java:537)
> at
> org.apache.jk.common.JkInputStream.endMessage(JkInputStream.java:127)
> at org.apache.jk.core.MsgContext.action(MsgContext.java:305)
> at org.apache.coyote.Response.action(Response.java:183)
> at org.apache.coyote.Response.finish(Response.java:305)
> at
> org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:205)
> at
> org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
> at
> org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
> at
> org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
> at
> org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
> at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
> at java.lang.Thread.run(Thread.java:595)
>
>
>
>> In Mysql: there are lots of slow queries I've been logging - > 1 sec.
>
> Are these mostly coming from your webapp or MediaWiki? Have you
> investigated these? Often, slow queries arise from inadequate or stale
> indexes. Try running REPAIR TABLE [tablename] on a table that sees a lot
> of slow queries. That should update the indexes in case you recently
> bulk-loaded a lot of data. If that's not the problem (and it's unlikely
> that it /is/ the problem), you should start running EXPLAIN on some of
> those queries to see if you have appropriate indexes on your tables.
>
>
>
> Mostly coming from Tomcat/webapp. Yes, one we fixed but we are still
> having issues w/ slow queries.
> Ok, we'll look into running MySQL Explain.
>
>
>
>
>> The errors that occur are random browser disconnects - urls that end or
>> start with *.jsp/ or *.php/ pages.
>
> Something like SocketException: Connection closed? If that's the case,
> the exceptions themselves are not a problem, but could be indicative of
> a larger problem: poor performance causes people to hit STOP on their
> browsers and re-try requests.
>
>
> Yes, see above for SocketExceoptions.
> Users are probably not hitting the Stop button - usually they refresh and
> most of the time the page reloads OK.
> We are looking at MySQL Performance.
>
>
>
>> We have also tried using WinXP host files and using an IP address to
>> bypass
>> DNS altogether - no luck.
>
> Are you actually performing DNS lookups?
>
> Yes, with all the other users/clients.
> We have a pilot "test" group of users only using the IP address in the URL
> box of their browsers.
>
> Two things might be happening if DNS is really your problem:
>
> 1. Your DNS server is slow. Mitigations are a) get a
> faster/closer/extra DNS server or b) disable DNS lookups
>
> Maybe - perhaps our Active Directory DNS serves up a alot of requests from
> our users and servers.
>
>
> 2. Your DNS settings are misconfigured (e.g. wrong ip address,
> etc.). Check your DNS configuration.
>
>
> Definitely not misconfiguration.
>
>
>> Problem around 4:15PM - Jan 9, 2009:
>> http://10.97.24.23/Sales/IndentedATPServlet?partNum=086EXUCCCXMA&qty=1&warehouse=M
>
> What was the problem?
>
> User was viewing a Servlet page that was querying the AS400 DB2 DB on the
> backend for a part number.
> Tomcat may have been waiting for a long time and then teh user recieved
> "page cannot be displayed" or "Server cannot be found" in IE7 browser.
>
>
> http://www.nabble.com/file/p21485970/dns.jpeg
>
>
> NOTE: The connection is thru Tomcat JDBC connector to AS400 DB2 DB.
>
>
>> Problem around 5:05PM - Jan 9, 2009:
>> http://wsi/Qoe/index.jsp
>> (same thing....Page Cannot Be Displayed)
>
> Again... what was the problem? Look in catalina.out or some other log
> file in your Tomcat's logs directory.
>
> We have Apache & Tomcat Debugging set very high. And have MySQL slow
> queries and error logs.
> Can't find any relevant to error.
> Can we set Tomcat debugging with date/time stamps or will Log4j help more
> with debugging?
>
>
>> Our MysqL server has a tendency to ramp up or the load avg increases
>> using
>> massive CPU cycles - I restart Tomcat and the system usually calms down -
>> Mysql included.
>
> This is why I was asking about connection pooling: it looks like Tomcat
> is murdering your DB. This could be due to too many connections to
> MySQL, overly high query volume, or sloppy code in your application
> leaking database resources.
>
>
> Java Developer is looking into code constantly and cleaning up code.
> He agreed that there is old sloppy code perhaps and possibly too many
> connections.
> Can we control the connections or help diagnose code errors in an easier
> way?
>
>
>> We are currently using tcpdump on the server & Wireshark on the affected
>> clients to look at the packet level of my connections to the webserver.
>
> Have you observed anything interesting using these tools?
>
>
> I've sent the logs to RHEL(RedHat Linux) support and they have been
> reviewing and think we have a network issue - router or VMware problems
> since this is a
>
> VMware guest.
> However, our clients are on a flat subnet with the server.
> Clients = 10.97.25.X
> Server = 10.97.24.23
>
> I was also told by another tech at RHEL support that RHEL5 runs fine on
> WMWaare as is if it were a physical box - confusing.
>
>
>
>> I know more details are probably needed - this is a complex problem.
>> And only seems to happen in the am right when users login to their
>> machines
>> and start up their browsers and then happens at the end of the day as
>> well.
>
> Maybe you are just getting more peak traffic than your setup can handle.
> Have you performed any formal load testing?
>
>
> Perhaps we have too much traffic or our applications need to be further
> tuned or reinstalled from source code.
> I am using the RHEL5 RPM's from RH Network.
>
> I see there are many ways to load test" MySQL - do you have any
> suggestions on where I can start?
> A free/Open source tool would be preferred. <smile>
>
>
> Thanks for all your questions and insight!
>
> -SP
>
>
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iEYEARECAAYFAklvYmUACgkQ9CaO5/Lv0PClcQCeOid/7wFe7/JvwpQHUE96xYft
> bagAn1MC7Rz+At5JitXUA8R4GHO/8BMf
> =Lxrm
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
--
View this message in context: http://www.nabble.com/Tomcat%3A-client-disconnects--tp21424139p21485970.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
|