From user-return-13852-apmail-geronimo-user-archive=geronimo.apache.org@geronimo.apache.org Wed Sep 02 21:03:02 2009 Return-Path: Delivered-To: apmail-geronimo-user-archive@www.apache.org Received: (qmail 26133 invoked from network); 2 Sep 2009 21:03:02 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 2 Sep 2009 21:03:02 -0000 Received: (qmail 69289 invoked by uid 500); 2 Sep 2009 21:03:02 -0000 Delivered-To: apmail-geronimo-user-archive@geronimo.apache.org Received: (qmail 69270 invoked by uid 500); 2 Sep 2009 21:03:01 -0000 Mailing-List: contact user-help@geronimo.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: Reply-To: user@geronimo.apache.org List-Id: Delivered-To: mailing list user@geronimo.apache.org Received: (qmail 69262 invoked by uid 99); 2 Sep 2009 21:03:01 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Sep 2009 21:03:01 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of lists@nabble.com designates 216.139.236.158 as permitted sender) Received: from [216.139.236.158] (HELO kuber.nabble.com) (216.139.236.158) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Sep 2009 21:02:53 +0000 Received: from isper.nabble.com ([192.168.236.156]) by kuber.nabble.com with esmtp (Exim 4.63) (envelope-from ) id 1MiwyO-0006nx-O6 for user@geronimo.apache.org; Wed, 02 Sep 2009 14:02:32 -0700 Message-ID: <25265686.post@talk.nabble.com> Date: Wed, 2 Sep 2009 14:02:32 -0700 (PDT) From: Adam Ruggles To: user@geronimo.apache.org Subject: Re: Tomcat WADI under heaving load fails In-Reply-To: <4A9EBE70.7050507@apache.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: a.ruggles@gmail.com References: <25247803.post@talk.nabble.com> <4A9EBE70.7050507@apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Sorry about that. OS = Mac OS X 10.5.8 for both nodes. Geronimo = 2.1.4 w/Tomcat JVM = 64Bit 1.6.0_13 Both web nodes are running behind Apache HTTPD which is passing the requests to Geronimo via mod_proxy_ajp. Apache HTTPD is not doing the load balancing there is a F5 doing that job. Apache is handling static files only. I had a chance to do more testing and it doesn't seem to be CPU or Memory related as the cpu is between 30 - 80% on a quad core boxes (the way apple reports cpu usage we can go up to 400%). Memory is at about 1G when 3G is available. The configuration is rock solid stable until I reach about 150 active connections. I did try bumping up the number of threads available to AJP but that didn't make a difference. Donald Woods-2 wrote: > > Which version of Geronimo? Which OS? > > -Donald > > Adam Ruggles wrote: >> I recently begin enabling WADI clustering in our test environment with >> two >> Apache Geronimo Nodes behind a load balancer. My current setup does not >> have sticky sessions enabled. It seems to be working well until I start >> throwing a large load at the nodes. I start receiving the following >> errors: >> >> 2009-09-01 15:05:03,415 INFO [JkCoyoteHandler] Error servicing request >> R( >> /login.action) >> org.codehaus.wadi.replication.manager.ReplicationKeyNotFoundException: >> Key >> [F78530ECFF88F89F042C8590E68E59DE] does not exist >> at >> org.codehaus.wadi.replication.manager.basic.SyncReplicationManager.update(SyncReplicationManager.java:147) >> at >> org.codehaus.wadi.core.session.AbstractReplicableSession.onEndProcessing(AbstractReplicableSession.java:49) >> at >> org.codehaus.wadi.core.session.AtomicallyReplicableSession.onEndProcessing(AtomicallyReplicableSession.java:58) >> at >> org.apache.geronimo.clustering.wadi.WADISessionAdaptor.onEndAccess(WADISessionAdaptor.java:63) >> at >> org.apache.geronimo.tomcat.cluster.ClusteredManager$ClusteredSession.endAccess(ClusteredManager.java:127) >> at org.apache.catalina.connector.Request.recycle(Request.java:419) >> at >> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:322) >> at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190) >> at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283) >> at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:767) >> at >> org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:697) >> at >> org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:889) >> at >> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690) >> at java.lang.Thread.run(Thread.java:637) >> >> >> Even after the heaving activity is over the nodes are unable to recover. >> I >> have to restart both geronimo nodes before the cluster functions >> correctly >> again. Has anyone else run into this and if have you resolved it? > > -- View this message in context: http://www.nabble.com/Tomcat-WADI-under-heaving-load-fails-tp25247803s134p25265686.html Sent from the Apache Geronimo - Users mailing list archive at Nabble.com.