tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guang Chao <guang.chao.1...@gmail.com>
Subject Re: Application hanging on Tomcat 7.0.54
Date Thu, 27 Sep 2018 03:45:05 GMT
On Thu, Sep 27, 2018 at 3:56 AM Louis Zipes <Louis.Zipes@gmcr.com> wrote:

> Problem just re-occurred and so I was able to at least get a JSTACK  (I
> assume it was Tomcat since it was the Java using the most memory on the
> machine).  Here is the reoccurring message.  I get more hits on but haven't
> dug through all of the Google hits yet (due to multi-tasking) so apologies
> up front if there is a simple answer to this.
>
> "Event_Manager_1413" daemon prio=6 tid=0x0000000024856000 nid=0x40c4
> waiting on condition [0x0000000042dae000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <0x00000005ab45f7b8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
> Source)
> at java.util.concurrent.LinkedBlockingQueue.poll(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
>
>
Do you lock from memory or database?  Make sure locking sequence is
consistent all across the application.  Meaning if you have

Lock A
Lock B
Lock C

You dont want another logic that locks something in another order.  E.g.

Lock B
Lock A
Lock C

Because that will cause deadlocks.



>    Locked ownable synchronizers:
> - None
>
> > Any comments/suggestions are appreciated!
>
> Your most likely problem is database connection pool mismanagement:
> connections aren't properly released and the pool empties. All threads
> are left waiting on available database connections which will never be
> replenished.
>
> I'm using the ojdbc6.jar if that is what you are referring to or is there
> a better setting somewhere.
>

What db connection pool are you using? Are you letting tomcat manage it via
Context.xml and get connection via JNDI?  Or are you managing inside the
application?


>
> -----Original Message-----
> From: Christopher Schultz [mailto:chris@christopherschultz.net]
> Sent: Wednesday, September 26, 2018 3:46 PM
> To: users@tomcat.apache.org
> Subject: Re: Application hanging on Tomcat 7.0.54
>
> - - - external message, proceed with caution - - -
>
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Louis,
>
> On 9/26/18 14:42, Louis Zipes wrote:
> > Hi all, Tomcat 7.0.54 running on Windows 2012
> >
> > We are running a third party application on Tomcat and today we
> > have intermittently run in issues where the application stops
> > working.  The big changes in our system is that we have added more
> > end users and we are at year end so of course everyone is hitting
> > the system hard. Even if we force a log out of all users and stop
> > all background jobs then the application doesn't recover.
> >
> > We see no active sessions on the database (our application is
> > connecting to an Oracle database) and I see no clear error messages
> > in either our third party application logs or the Tomcat logs (ex.
> > OutofMemory).  When we go to the Windows Task Manager we did not
> > see the machine's Memory max'd out but admittedly I didn't look at
> > the Java session to see if was reaching its Heap Max.  The only
> > thing that we noticed was that TCP connections went down right
> > after the restart.  I did open up Jconsole under Java and I did
> > force a garbage collection but that didn't seem to help.
> >
> > We do have an Oracle Grid Control and we did get an alert in
> > regards to Metric: [HTTP Transaction] Perceived Time per Page going
> > past thresholds but not sure if that was just an old alert with and
> > old range that was set up a long time ago or is a really valid
> > clue.    Since this is PRD we had to get it back up and running so
> > all I did was increase the Tomcat Xmx Heap size and restarted.  I'm
> > not really confident that is the solution since as mentioned you
> > tend to see a clear out of memory error if it was too small.
> >
> > So a few questions:
> >
> >
> > 1)     Does this sound like a known issue with this earlier version
> > of Tomcat?
>
> No.
>
> > 2)     Should I turn up any logging on Tomcat and if so which
> > ones?
>
> Not yet.
>
> > 3)     We didn't do a JSTACK dump while it was happening.  Would
> > that have been useful?
>
> Absolutely.
>
> > 4)     Do we need to play around with MaxThreads and/or
> > MaxConnections.  We do have maxThreads in our server.mxl but in DEV
> > when we turned it down to a value = 5  hoping to overwhelm it
> > nothing bad happened.
>
> Don't change anything, yet.
>
> > Once again, we are limited to what we could do and collect since it
> > was PRD and we needed to restart it.  We restarted the Tomcat
> > service and everything is processing fine for right now.  I will
> > note that that we did have that bad Windows patch that prevented it
> > from stopping and starting cleanly
> > (https://stackoverflow.com/questions/51498291/tomcat-lockup-on-shutdow
> n)
> > but we have taken the break fix patch and the daily restarts seem
> > to be fine since then.
> >
> > Any comments/suggestions are appreciated!
>
> Your most likely problem is database connection pool mismanagement:
> connections aren't properly released and the pool empties. All threads
> are left waiting on available database connections which will never be
> replenished.
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlur4fgACgkQHPApP6U8
> pFhq/RAAwZixHqbHxYdX3VCrTfvz0tnOmu7W4sbeFqhExV+M6NVL2LK1RO26eTq5
> OJB2o/RheCeajWHxqiJQY4ERMTOyyqZYCsRG9L901heW2UAW122zeX7hhXDB1IMo
> qIBVYQalg1j5e2Lw9MqT3ISj6U/GNA6VlirTAHGtuEBBpKXXyb6KtmOgpjHjjXS7
> mTYni2iTHO/NaGrS519alFPMBnF4Wq5NzRcLewNMqj9Nbx2uu3Suu95DhJ+WIIep
> DOmyy4UGHxGx2QqNUnVWMHApGGjFD4pTWIBwzcbbsL56kZDxRGsF0SsB0VR/jSMY
> HdI71RgjpQFSyar1rcCTPCRP97KnUC+oaKhn+i2jBMSRxs94GEOqk4LuKNo39HKP
> tXEMkYl0o/CR9QaFDjncy9M2M3/o50ooBvYTOu0SjmyZO+ab9tJpaXbnlf2ChxJI
> AWbhBGmwJ6kF5FvVbmzujV7EEF2YCRMBpWo2zjNd6zWvX9OWEOXrOaHuX7iBPCga
> YqEMQWyS7XWiBG7AI8+ka5x4s/oxWsbn/6pCdDXhfxl5p5jv7ajm7LbLXGut1N6a
> uV5hmLLgZywF68AYe6X3GWv6mygXMBYABZxEA6klWE7HpIzvLxmxu0+vFlYR8qsb
> FMAacJ/FYJ9nGRuMqG+V2Edr5U//JvrWqy4raPIvwXGtX+FsqEc=
> =Zavb
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
> ---------------------------------------
> CONFIDENTIALITY NOTICE: This message is for intended addressee(s) only and
> may contain information that is confidential, proprietary or exempt from
> disclosure. If you are not the intended recipient, please contact the
> sender immediately. Unauthorized use or distribution is prohibited and may
> be unlawful.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

-- 
Guang <http://javadevnotes.com/java-string-to-char-array>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message