Return-Path: Delivered-To: apmail-tomcat-users-archive@www.apache.org Received: (qmail 81755 invoked from network); 30 Dec 2006 19:14:57 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 30 Dec 2006 19:14:57 -0000 Received: (qmail 73462 invoked by uid 500); 30 Dec 2006 19:14:48 -0000 Delivered-To: apmail-tomcat-users-archive@tomcat.apache.org Received: (qmail 73432 invoked by uid 500); 30 Dec 2006 19:14:48 -0000 Mailing-List: contact users-help@tomcat.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Tomcat Users List" Delivered-To: mailing list users@tomcat.apache.org Received: (qmail 73406 invoked by uid 99); 30 Dec 2006 19:14:48 -0000 Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 30 Dec 2006 11:14:48 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: domain of lists@nabble.com designates 72.21.53.35 as permitted sender) Received: from [72.21.53.35] (HELO talk.nabble.com) (72.21.53.35) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 30 Dec 2006 11:14:38 -0800 Received: from [72.21.53.38] (helo=jubjub.nabble.com) by talk.nabble.com with esmtp (Exim 4.50) id 1H0jes-000085-3I for users@tomcat.apache.org; Sat, 30 Dec 2006 11:14:18 -0800 Message-ID: <8100719.post@talk.nabble.com> Date: Sat, 30 Dec 2006 11:14:18 -0800 (PST) From: Peter Coppens To: users@tomcat.apache.org Subject: Re: session#getId changes during doGet invocation under heavy load In-Reply-To: <327858f40612300959v17c339c1rd3dc3953138c1621@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Nabble-From: pc.subscriptions@gmail.com References: <8081064.post@talk.nabble.com> <8085181.post@talk.nabble.com> <327858f40612290331y3a357d02h897f42eb5bda4b85@mail.gmail.com> <8097754.post@talk.nabble.com> <327858f40612300553q1f5c9858v96ea92b14a2cdb03@mail.gmail.com> <8099384.post@talk.nabble.com> <327858f40612300959v17c339c1rd3dc3953138c1621@mail.gmail.com> X-Virus-Checked: Checked by ClamAV on apache.org Thanks for the suggestions. I have been playing with the timeouts to trigger the problem. Typically I would like to have them around 15minutes....and yes that is apparently not sufficient for the load test I am doing. The setup is three machines. One with tomcat running that connects to a 2n= d one running MySQL. THe third is running jMeter (300 'users'). That said, I am baffled by the fact that some connections take >15minutes. >From the start, I included a dbcp pool to try and optimize the db resource usage, but I am starting to wonder whether that is not causing more problem= s than it is solving. I'll try to isolate the dbcp pool code and see how that behaves. In the end I guess it can also be the db server. For now I am going to try to not have timeouts during doGet processing, rather than try to deal with them. Something I did not understand from the suggestions is=20 > 1. You make DB requests not only from tomcat which is suboptimal > anyway, but also directly from the servlet, which would mean that you > utterly need to refactor your application anyway. I am not sure I understand the difference "from tomcat" versus "from the servlet" Again, all suggestions warmly welcomed! Peter Leon Rosenberg-3 wrote: >=20 > On 12/30/06, Peter Coppens wrote: >> >> Actually it just seems to be related to the fact that under heavy load >> the db >> connection starts to take longer than the timeout. >=20 > Thats a lot. What is your timeout configured to? >=20 >> >> Apparently, a call to request.getSession() somewhere in the middel of th= e >> doGet processing will also trigger invalidating the session, which is >> kind >> of a nuisance as one should then apparently constantly check whether the >> session has not expired during request processing. >> >> I assume this is Servlet spec compliant, but it does seem to make my lif= e >> rather complex. >> >> Would there be anyone having any suggestions to deal with this. >=20 > We might need some more details on your application to give you > meaningful suggestions, but just from the hip I'd say following: >=20 > 1. You make DB requests not only from tomcat which is suboptimal > anyway, but also directly from the servlet, which would mean that you > utterly need to refactor your application anyway. >=20 > 2. Apparently your timeouts are way to low. On the other hand, it > can't be healthy if your requests last longer than hours (default > timeout for the session are 2 hours) >=20 > 3. In case you can't change anything other, you could consider > starting a thread on the start of the request which just calls > session.get/put with dummy attributes periodically to prevent the > session from being timeouted. However this feature has to be > implemented carefully, since it can cause much harm with regard to > garbage collection if your thread instances sticks and keep references > to the session or similar. >=20 > By the way, what do you mean by "heavy load"? >=20 > Regards > Leon >=20 >> >> Thanks, >> >> Peter >> >> >> Leon Rosenberg-3 wrote: >> > >> > On 12/30/06, Peter Coppens wrote: >> >> >> >> I am gathering more evidence that this is related to a session >> expiring >> >> on >> >> one hand and a request being processed for that same session. >> >> >> >> I have been debugging the tomcat code a bit, and I have the >> *impression* >> >> that the expiration handling is not thread safe. It seems possible at >> >> first >> >> sight that the background processor decides a session is expired whil= e >> at >> >> the same time another thread starts a request for that same session. >> >> >> >> I will try to debug a bit more and if I find more I will let you know= . >> > >> > The question is whether the next request get the "right" session again >> > or not. I had the impression from your first post, that this is the >> > case: >> > Request A - Session 1 >> > Request B - Session 2 <-- which is wrong >> > Request C - Session 1 again. >> > >> > I observed this behaviour 3 years ago on a resin 2.1.x, but had not >> > enough time to debug it. >> > >> > regards >> > Leon >> >> >> >> Thanks, >> >> >> >> Peter >> >> >> >> PS What does "O/T" mean ? >> > >> > Off Topic >> > >> >> >> >> >> >> Martin Gainty wrote: >> >> > >> >> > Agreed >> >> > Once you have your use cases and test cases identified >> >> > >> >> > If you want the server to maintain its own side of the relationship >> >> > independent of client activity then you should consider container >> >> managed >> >> > persistence >> >> > More info avaialable at >> >> > >> >> >> http://www.javaworld.com/javaworld/jw-08-2006/jw-0828-persistence.html?p= age=3D6 >> >> > >> >> > Feel free to contact me offline as this is definitely O/T >> >> > Martin-- >> >> > >> >> >> ------------------------------------------------------------------------= --- >> >> > This e-mail message (including attachments, if any) is intended for >> the >> >> > use of the individual or entity to which it is addressed and may >> >> contain >> >> > information that is privileged, proprietary , confidential and >> exempt >> >> from >> >> > disclosure. If you are not the intended recipient, you are notified >> >> that >> >> > any dissemination, distribution or copying of this communication is >> >> > strictly prohibited. >> >> > >> >> >> ------------------------------------------------------------------------= --- >> >> > Le pr=C3=A9sent message =C3=A9lectronique (y compris les pi=C3=A8ce= s qui y sont >> >> annex=C3=A9es, >> >> > le cas =C3=A9ch=C3=A9ant) s'adresse au destinataire indiqu=C3=A9 et= peut contenir >> des >> >> > renseignements de caract=C3=A8re priv=C3=A9 ou confidentiel. Si vou= s n'=C3=AAtes >> pas >> >> le >> >> > destinataire de ce document, nous vous signalons qu'il est >> strictement >> >> > interdit de le diffuser, de le distribuer ou de le reproduire. >> >> > ----- Original Message ----- >> >> > From: "Leon Rosenberg" >> >> > To: "Tomcat Users List" >> >> > Sent: Friday, December 29, 2006 6:31 AM >> >> > Subject: Re: session#getId changes during doGet invocation under >> heavy >> >> > load >> >> > >> >> > >> >> >> Do I understand it right, that you made it a reproduceable >> testcase? >> >> >> If so, can we have a look on it? >> >> >> >> >> >> thank you >> >> >> Leon >> >> >> >> >> >> On 12/29/06, Peter Coppens wrote: >> >> >>> >> >> >>> Thanks Chuck. >> >> >>> >> >> >>> I have done some further research and I have the impression that >> >> there >> >> >>> is >> >> >>> some kind of race condition where a session that is being removed >> >> >>> because of >> >> >>> a timeout is also handling requests. >> >> >>> >> >> >>> The loggings indicate that a session is destroyed but then >> >> nevertheless >> >> >>> a >> >> >>> doGet is invoked with a request parameter that refers to that >> timed >> >> out >> >> >>> session. >> >> >>> >> >> >>> If I crank up the timeout, seriously reducing the chances a >> session >> >> >>> times >> >> >>> out before it has completed all the client requests it is suppose= d >> to >> >> >>> handle >> >> >>> during the test, the problem does no longer occur. >> >> >>> >> >> >>> I am not sure where that leaves me. I am still uncertain as to >> what >> >> the >> >> >>> servlet is doing wrong. >> >> >>> >> >> >>> Would you (or anyone else) have any other comments on this? >> >> >>> >> >> >>> Thanks, >> >> >>> >> >> >>> Peter >> >> >>> >> >> >>> >> >> >>> >> >> >>> Caldarale, Charles R wrote: >> >> >>> > >> >> >>> >> From: Peter Coppens [mailto:pc.subscriptions@gmail.com] >> >> >>> >> Subject: session#getId changes during doGet invocation under >> >> >>> >> heavy load >> >> >>> >> >> >> >>> >> THe problem I run into is that, under heavy load, during the >> >> >>> >> doGet invocation for the login request the session attached >> >> >>> >> to the request is changed by some other thread (the value >> >> >>> >> returned from getId changes and attributes that are originally >> >> >>> >> set are removed). >> >> >>> > >> >> >>> > This is most likely an application problem - storing session- o= r >> >> >>> > request-specific data in an inappropriately scoped structure >> (e.g., >> >> a >> >> >>> > servlet field). Under light load, this won't hurt, since the >> >> >>> insertion >> >> >>> > and retrieval for a given request don't overlap any other >> requests. >> >> >>> As >> >> >>> > the load increases, so does the probability of an overlap and >> >> >>> erroneous >> >> >>> > retrieval of data for another request. >> >> >>> > >> >> >>> > - Chuck >> >> >>> > >> >> >>> > >> >> >>> > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE >> >> >>> PROPRIETARY >> >> >>> > MATERIAL and is thus for use only by the intended recipient. If >> you >> >> >>> > received this in error, please contact the sender and delete th= e >> >> >>> e-mail >> >> >>> > and its attachments from all computers. >> >> >>> > >> >> >>> > >> >> --------------------------------------------------------------------- >> >> >>> > To start a new topic, e-mail: users@tomcat.apache.org >> >> >>> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org >> >> >>> > For additional commands, e-mail: users-help@tomcat.apache.org >> >> >>> > >> >> >>> > >> >> >>> > >> >> >>> >> >> >>> -- >> >> >>> View this message in context: >> >> >>> >> >> >> http://www.nabble.com/session-getId-changes-during-doGet-invocation-unde= r-heavy-load-tf2892448.html#a8085181 >> >> >>> Sent from the Tomcat - User mailing list archive at Nabble.com. >> >> >>> >> >> >>> >> >> >>> >> --------------------------------------------------------------------- >> >> >>> To start a new topic, e-mail: users@tomcat.apache.org >> >> >>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org >> >> >>> For additional commands, e-mail: users-help@tomcat.apache.org >> >> >>> >> >> >>> >> >> >> >> >> >> >> --------------------------------------------------------------------- >> >> >> To start a new topic, e-mail: users@tomcat.apache.org >> >> >> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org >> >> >> For additional commands, e-mail: users-help@tomcat.apache.org >> >> >> >> >> >> >> >> > >> >> >> >> -- >> >> View this message in context: >> >> >> http://www.nabble.com/session-getId-changes-during-doGet-invocation-unde= r-heavy-load-tf2892448.html#a8097754 >> >> Sent from the Tomcat - User mailing list archive at Nabble.com. >> >> >> >> >> >> --------------------------------------------------------------------- >> >> To start a new topic, e-mail: users@tomcat.apache.org >> >> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org >> >> For additional commands, e-mail: users-help@tomcat.apache.org >> >> >> >> >> > >> > --------------------------------------------------------------------- >> > To start a new topic, e-mail: users@tomcat.apache.org >> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org >> > For additional commands, e-mail: users-help@tomcat.apache.org >> > >> > >> > >> >> -- >> View this message in context: >> http://www.nabble.com/session-getId-changes-during-doGet-invocation-unde= r-heavy-load-tf2892448.html#a8099384 >> Sent from the Tomcat - User mailing list archive at Nabble.com. >> >> >> --------------------------------------------------------------------- >> To start a new topic, e-mail: users@tomcat.apache.org >> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org >> For additional commands, e-mail: users-help@tomcat.apache.org >> >> >=20 > --------------------------------------------------------------------- > To start a new topic, e-mail: users@tomcat.apache.org > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org > For additional commands, e-mail: users-help@tomcat.apache.org >=20 >=20 >=20 --=20 View this message in context: http://www.nabble.com/session-getId-changes-d= uring-doGet-invocation-under-heavy-load-tf2892448.html#a8100719 Sent from the Tomcat - User mailing list archive at Nabble.com. --------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org For additional commands, e-mail: users-help@tomcat.apache.org