openjpa-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Sutter <kwsut...@gmail.com>
Subject Re: memory leak? - simple question
Date Thu, 14 Jan 2010 19:57:55 GMT
We must have been crossing in the ether...  :-)  No need for the trace, I
can do that now.  Thanks anyway!

Kevin

On Thu, Jan 14, 2010 at 1:55 PM, Kurt T Stam <kurt.stam@gmail.com> wrote:

> OK :).
>
> 1. Unzip tomcat-1.0.SNAPSHOT.zip
>
> 2. cd tomcat-1.0.SNAPSHOT/bin
>
> 3. ./start.sh
>
> Now, to send in a request open your browser to url:
>
>
> http://localhost:8080/leak/test
>
> Hit refresh to send in another request.
>
> I'll try getting you a trace log. Which packages? Just
> "org.apache.openjpa"?
>
> Cheers,
>
> --Kurt
>
>
> Kevin Sutter wrote:
>
>> Hi Kurt,
>>
>> These instructions, of course, assume that I know something about Tomcat.
>> This would not be a good assumption...  :-)  If you really want me to see
>> this in action, I will need more information on how to start Tomcat and
>> how
>> to load applications.  In the mean time, a trace log of OpenJPA would be
>> an
>> excellent start.
>>
>> Thanks!
>> Kevin
>>
>> On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam <kurt.stam@gmail.com>
>> wrote:
>>
>>
>>
>>> Hi Kevin,
>>>
>>> I tried adding the javaagent stuff, and it still ends up leaking. I
>>> extracted the code into a tiny
>>> war with just one servlet which persist the authToken. The build creates
>>> a
>>> Tomcat distro backed by a
>>> derby database. You can download it from:
>>>
>>> http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz>
>>> <http://people.apache.org/%7Ekstam/leak.tgz>
>>>
>>>
>>> You can build it with: mvn install, you can find the final tomcat in
>>> tomcat/target/tomcat-1.0.SNAPSHOT.zip
>>>
>>> or if you don't want to build it:
>>> http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>> <http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>>
>>>
>>> After you bring up tomcat it persistent an authToken when you hit
>>> http://localhost:8080/leak/test
>>>
>>> In the profiler I see an org.apache.juddi.AuthToken left behind for each
>>> request.
>>>
>>> Hopefully this little example will pinpoint what's going on.
>>>
>>> Thx Kev for sticking with me :)
>>>
>>> --Kurt
>>>
>>>
>>>
>>> Kevin Sutter wrote:
>>>
>>>
>>>
>>>> Hi Kurt,
>>>> Since you have the environment to reproduce this, could you run with the
>>>> OpenJPA trace enabled?  I'm not even sure what I'm looking for at this
>>>> point.  :-)  But, maybe there's some tracing that would point us in the
>>>> right direction.
>>>>
>>>> <property name="openjpa.Log" value="DefaultLevel=TRACE"/>
>>>>
>>>> We have been always been on the watch for memory leaks, so this one is
>>>> puzzling.  You did mention blocked threads as well.  At this point, you
>>>> are
>>>> probably hitting blocked threads due to the GC and Memory problems.
>>>>  But,
>>>> we
>>>> did recently resolve some locking situations [1] in our
>>>> MetaDataRepository
>>>> that could affect scaling (under extreme load).  This was resolved in
>>>> the
>>>> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release,
>>>> so
>>>> that should be coming soon).
>>>>
>>>> Does your WebService version of the problem use EntityManager
>>>> transactions,
>>>> or is the transaction service coming from Tomcat?  And, does your
>>>> WebService
>>>> version also do the em.clear() and em.close()?  Basically, are you using
>>>> application-managed or container-managed persistence while running as a
>>>> WebService?
>>>>
>>>> I'm very interested in helping to resolve this issue.  Like I mentioned,
>>>> we
>>>> do a lot of memory leakage and scalability testing of the OpenJPA
>>>> solution
>>>> and we're not aware of anything.  Although there is always the
>>>> possibility
>>>> that something was resolved in the 1.2.x branch after the 1.2.1 release
>>>> that
>>>> I'm not remembering...
>>>>
>>>> Thanks for your help,
>>>> Kevin
>>>>
>>>> [1]  https://issues.apache.org/jira/browse/OPENJPA-250
>>>>
>>>> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <kurt.stam@gmail.com>
>>>> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Thanks Kevin,
>>>>>
>>>>> We're enhancing at build time:
>>>>>
>>>>>
>>>>>
>>>>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>>>>>
>>>>> Yeah we've been running load tests and things are nice and stable with
>>>>> Hibernate but with Openjpa we see increasing memory use, blocking
>>>>> threads
>>>>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our
>>>>> preference would be to ship with openjpa by default; but our build
>>>>> supports
>>>>> both hibernate and openjpa.
>>>>>
>>>>> And yes we use openjpa 1.2.1 (latest stable version).
>>>>>
>>>>> --Kurt
>>>>>
>>>>>
>>>>> Kevin Sutter wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Interesting detective work, Kurt.  Thanks.
>>>>>>
>>>>>> Why the WebService version of the app would behave differently as
far
>>>>>> as
>>>>>> GC
>>>>>> is concerned is a mystery.  And, you said that plugging in Hibernate
>>>>>> into
>>>>>> this scenario, everything works okay?  Very confusing.
>>>>>>
>>>>>> How are you performing the Entity enhancement processing?  Are you
>>>>>> pre-enhancing via your build process?  Or, are you using the
>>>>>> -javaagent
>>>>>> mechanism?  Or, are you falling back to the subclassing support within
>>>>>> OpenJPA?  (See [1] for more information on these questions in case
>>>>>> they
>>>>>> don't make sense.)
>>>>>>
>>>>>> This would be one area that is different between Hibernate and OpenJPA
>>>>>> --
>>>>>> enhancement processing.
>>>>>>
>>>>>> In the Tomcat environment, you may be falling back to the subclassing
>>>>>> support (which we do not recommend) and hitting a memory leak with
>>>>>> that.
>>>>>>
>>>>>> You said OpenJPA 1.2.x, right?
>>>>>>
>>>>>> Just a couple of thoughts on the subject...
>>>>>> Kevin
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>>
>>>>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <kurt.stam@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> The same code executed straight from a java client (inVM) shows
no
>>>>>>> memory
>>>>>>> leak.
>>>>>>>
>>>>>>> So is the fact that it is WebService significant then? What else
can
>>>>>>> be
>>>>>>> different? I think one thread remains up, and somehow this causes
>>>>>>> openjpa
>>>>>>> not being able to clean up after itself. What can I do to debug
this
>>>>>>> more? I
>>>>>>> can actually see in the profiler that the objects are allocated
by
>>>>>>> the
>>>>>>> WebService, but why aren't they cleaned up?
>>>>>>>
>>>>>>> Thx,
>>>>>>>
>>>>>>>
>>>>>>> --Kurt
>>>>>>>
>>>>>>>
>>>>>>> Kurt T Stam wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Thanks Kevin, thanks for your response.
>>>>>>>>
>>>>>>>> I just replaced the static call by:
>>>>>>>>
>>>>>>>>            apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>            apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>>>>>>            //MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>> apiAuthToken);
>>>>>>>>
>>>>>>>> which did not make a difference.
>>>>>>>>
>>>>>>>> I'm wondering if the fact that my class is a webservice makes
a
>>>>>>>> difference. I'll try extracting it into
>>>>>>>> a regular class with a main method and profile that. At least
I know
>>>>>>>> that
>>>>>>>> I didn't forget something
>>>>>>>> completely obvious..
>>>>>>>>
>>>>>>>> --Kurt
>>>>>>>>
>>>>>>>> Kevin Sutter wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Kurt,
>>>>>>>>> I agree that this is very common usage of the JPA programming
>>>>>>>>> model.
>>>>>>>>>  And,
>>>>>>>>> we are not aware of any memory leaks.  About the only
thing that
>>>>>>>>> jumps
>>>>>>>>> out
>>>>>>>>> at me is the following two lines:
>>>>>>>>>
>>>>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>             MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>> apiAuthToken);
>>>>>>>>>
>>>>>>>>> What do these do?  Can you comment these out and see
if the memory
>>>>>>>>> leak
>>>>>>>>> still exists?  Since you are passing the modelAuthToken
into this
>>>>>>>>> method,
>>>>>>>>> I
>>>>>>>>> don't know what it's doing with the reference and could
it be
>>>>>>>>> holding
>>>>>>>>> onto
>>>>>>>>> something to prevent the GC from cleaning up?
>>>>>>>>>
>>>>>>>>> The rest of your example seems very straight forward
with creating
>>>>>>>>> and
>>>>>>>>> persisting objects.
>>>>>>>>>
>>>>>>>>> Kevin
>>>>>>>>>
>>>>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <curtisr7@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> If you change the 1000 to something like 1000000...
does your
>>>>>>>>>> application
>>>>>>>>>> go
>>>>>>>>>> OOM? Are you running in a JSE environment? What is
>>>>>>>>>> PersistenceManager?
>>>>>>>>>>
>>>>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <kurt.stam@gmail.com
>>>>>>>>>> >
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> BTW I'm running with the cache off
>>>>>>>>>>>
>>>>>>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>>>>>>
>>>>>>>>>>> (that turns it off right?)
>>>>>>>>>>>
>>>>>>>>>>> --Kurt
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Kurt T Stam wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>>
>>>>>>>>>>>> [DESCRIPTION] The code below inserts a 1000
records in the
>>>>>>>>>>>> database.
>>>>>>>>>>>>
>>>>>>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>>>>>>       EntityManager em = PersistenceManager.getEntityManager();
>>>>>>>>>>>>       EntityTransaction tx = em.getTransaction();
>>>>>>>>>>>>       try {
>>>>>>>>>>>>           tx.begin();
>>>>>>>>>>>>               // Generate auth token and
store it!
>>>>>>>>>>>>           String authInfo = AUTH_TOKEN_PREFIX
+
>>>>>>>>>>>> UUID.randomUUID();
>>>>>>>>>>>>           org.apache.juddi.model.AuthToken
modelAuthToken = new
>>>>>>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>>>>>>           if (authInfo != null) {
>>>>>>>>>>>>               modelAuthToken.setAuthToken(authInfo);
>>>>>>>>>>>>               modelAuthToken.setCreated(new
Date());
>>>>>>>>>>>>               modelAuthToken.setLastUsed(new
Date());
>>>>>>>>>>>>               modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>>>>>>               modelAuthToken.setNumberOfUses(0);
>>>>>>>>>>>>               modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>>>>>>                 em.persist(modelAuthToken);
>>>>>>>>>>>>           }
>>>>>>>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>>>             MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>>>> apiAuthToken);
>>>>>>>>>>>>             tx.commit();
>>>>>>>>>>>>                     } finally {
>>>>>>>>>>>>           if (tx.isActive()) {
>>>>>>>>>>>>               tx.rollback();
>>>>>>>>>>>>           }
>>>>>>>>>>>>           em.clear();
>>>>>>>>>>>>           em.close();
>>>>>>>>>>>>       }
>>>>>>>>>>>>   }
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [ISSUE]
>>>>>>>>>>>> After it leaving this code I end up with
a 1000
>>>>>>>>>>>> org.apache.juddi.model.AuthToken objects
in memory. I've been
>>>>>>>>>>>> using
>>>>>>>>>>>> the
>>>>>>>>>>>> profiler, and these objects cannot be garbage
collected.
>>>>>>>>>>>>
>>>>>>>>>>>> This seems to be pretty the most common use
case of using an
>>>>>>>>>>>> OR-mapping
>>>>>>>>>>>> tool, so I find it hard to believe openjpa
has a memory leak
>>>>>>>>>>>> here.
>>>>>>>>>>>> Does
>>>>>>>>>>>> anyone see what I'm doing wrong? Or can someone
point me to an
>>>>>>>>>>>> example
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> that
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> does not exhibit this behavior? BTW same code
using hibernate
>>>>>>>>>>> does
>>>>>>>>>>> not
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> accumulate these objects.
>>>>>>>>>>>>
>>>>>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thx,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Kurt
>>>>>>>>>>>>
>>>>>>>>>>>> Apache jUDDI.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Thanks,
>>>>>>>>>> Rick
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message