cayenne-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ayhan Kondoz" <Ayhan.Kon...@freenet-ag.de>
Subject AW: possible bug / memory leak in DispatchQueue and EventManager?
Date Wed, 21 Feb 2007 13:37:40 GMT
Hi,

the JVM has 1 GB of memory and i already tried to run the GC manually but it doesn't make
a difference. I can start the GC from the profiler tool but the number of instances does not
change. 
I guess that there are some other references to this objects so that the weak ref. does not
expire.

And there are no EventManager exception in the log files.


ayhan

> -----Ursprüngliche Nachricht-----
> Von: Andrus Adamchik [mailto:andrus@objectstyle.org]
> Gesendet: Mittwoch, 21. Februar 2007 14:17
> An: user@cayenne.apache.org
> Betreff: Re: possible bug / memory leak in DispatchQueue and EventManager?
> 
> Seems like your assessment of the EventManager leaking is correct.
> 
> Now the cause is not that clear. A shot in the dark - this is due to
> a combination of lots of spare memory in JVM (so weak references are
> not collected fast enough) and slow custom 'equals' and 'hashCode'
> methods in invocation.
> 
> How much heap size do you have in your server? Also can you try to
> add a request filter that would do 'System.gc()' on every few
> thousands requests, and see if it makes any difference.
> 
> Also - do you see any EventManager exceptions in the logs?
> 
> Andrus
> 
> 
> On Feb 21, 2007, at 10:55 AM, Ayhan Kondoz wrote:
> > Hi,
> >
> >
> >
> > i think i found a possible bug / memory leak within the
> > DispatchQueue and EventManager.
> >
> >
> >
> >
> >
> > First a little bit about my setup:
> >
> >
> >
> > I have 3 servers. Each server runs an axis service. The service
> > uses cayenne 1.2.1 to connect to a database. It reads customer and
> > account information from the DB etc..
> >
> > The servers are using cayenne's shared caching with javagroups as
> > the messaging service so that changes made from one server are
> > dispatched to the other servers.
> >
> >
> >
> > The avarage connections per second is somewhere around 4-5.
> >
> >
> >
> > However I have a very strange problem with this setup so I startet
> > to search for the reason. The problem is that with nearly constant
> > and unchanging usage the load of each server increases over time.
> > To further test this I created a test server with a similar setup.
> > There I created a test program that creates totally constant usage.
> > But even with the unchanging usage the load of the server is
> > increasing until the cpu load is so high that the requests can not
> > be processed anymore.
> >
> >
> >
> > I installed a java profiler to trying to pinpoint the location of
> > this error and this is what I found out.
> >
> >
> >
> > I let the server run for 24 hours and then stopped the program
> > which creates the test usage.
> >
> > But even while the server was idle there where still a lot of
> > instances in the java heap after the GC run.
> >
> >
> >
> > http://img206.imageshack.us/img206/9769/memorysy5.jpg
> >
> >
> >
> > Please note the HashMap, WeakReference and Invocation counts. I
> > pressume that the $ObjectProvider_*** is cayenne aswell but I am
> > not sure.
> >
> >
> >
> > Now the following image shows the cpu profile with incoming
> > connections.
> >
> >
> >
> > http://img339.imageshack.us/img339/2441/cpuci0.jpg
> >
> >
> >
> > As you can see 58% of the cpu time is used within HashSet.add().
> >
> >
> >
> > So when I consider the two facts i think that there might be a
> > possible problem with the EventManager. The first table tells us
> > that there are over 1 million instances of HashSet's and cayenne
> > Invocations. So it seems like the set's within the DispatchQueue
> > are not recycled properly so that the object count rises over time
> > which result in extremly high processtime when trying to add new
> > HashSet's.
> >
> >
> >
> >
> >
> > Thanks
> >
> > Ayhan
> >
> >
> >
> >
> >
> >
> >
> > Ayhan Kondoz
> >
> >
> >
> > Software-Entwicklung
> >
> >
> >
> > ----------------------------------------------------------------------
> > ------------
> >
> > Telefon:    +49 (0) 40 513 06 616
> >
> > Telefax:    +49 (0) 40 513 06 998 616
> >
> > E-Mail:     ayhan.kondoz@freenet-ag.de <mailto:ayhan.kondoz@freenet-
> > ag.de>
> >
> > Website:  http://www.freenet.de <http://www.freenet.de/> ; http://
> > www.freenet-ag.de <http://www.freenet-ag.de/>
> >
> > ----------------------------------------------------------------------
> > ------------
> >
> > freenet.de AG
> >
> > Deelbögenkamp 4c
> >
> > 22297 Hamburg
> >
> > ----------------------------------------------------------------------
> > ------------
> >
> > Vorsitzender des Aufsichtsrates: Prof. Dr. Helmut Thoma
> >
> > Vorstand: Eckhard Spoerr (Vors.),
> > Axel Krieger, Stephan Esch, Eric Berger
> >
> > Amtsgericht Hamburg HRB 74048
> >
> >
> >
> >
> >
> > Diese Information ist ausschließlich für die adressierte Person
> > oder Organisation bestimmt und könnte vertrauliches und/oder
> > privilegiertes Material enthalten. Personen oder Organisationen,
> > für die diese Information nicht bestimmt ist, ist es nicht
> > gestattet, diese zu lesen, erneut zu übertragen, zu verbreiten,
> > anderweitig zu verwenden oder sich durch sie veranlasst zu sehen,
> > Maßnahmen irgendeiner Art zu ergreifen. Sollten Sie diese Nachricht
> > irrtümlich erhalten haben, bitten wir Sie, sich mit dem Absender in
> > Verbindung zu setzen und das Material von Ihrem Computer zu löschen.
> >
> >
> >
> > The information transmitted is intended only for the person or
> > entity to which it is addressed and may contain confidential and/or
> > privileged material. Any review, retransmission, dissemination or
> > other use of, or taking of any action in reliance upon, this
> > information by persons or entities other than the intended
> > recipient is prohibited. If you received this in error, please
> > contact the sender and delete the material from any computer.
> >
> >
> >


Mime
View raw message