openjpa-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Sutter <kwsut...@gmail.com>
Subject Re: memory leak? - simple question
Date Thu, 14 Jan 2010 19:57:02 GMT
Hey, this wasn't so difficult...  :-)  Maybe I can still learn new
things...  I have Tomcat up and running and I can get your app running.

On Thu, Jan 14, 2010 at 1:46 PM, Kevin Sutter <kwsutter@gmail.com> wrote:

> Hi Kurt,
> These instructions, of course, assume that I know something about Tomcat.
> This would not be a good assumption...  :-)  If you really want me to see
> this in action, I will need more information on how to start Tomcat and how
> to load applications.  In the mean time, a trace log of OpenJPA would be an
> excellent start.
>
> Thanks!
> Kevin
>
>
> On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam <kurt.stam@gmail.com> wrote:
>
>> Hi Kevin,
>>
>> I tried adding the javaagent stuff, and it still ends up leaking. I
>> extracted the code into a tiny
>> war with just one servlet which persist the authToken. The build creates a
>> Tomcat distro backed by a
>> derby database. You can download it from:
>>
>> http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz>
>>
>> You can build it with: mvn install, you can find the final tomcat in
>> tomcat/target/tomcat-1.0.SNAPSHOT.zip
>>
>> or if you don't want to build it:
>> http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>
>> After you bring up tomcat it persistent an authToken when you hit
>> http://localhost:8080/leak/test
>>
>> In the profiler I see an org.apache.juddi.AuthToken left behind for each
>> request.
>>
>> Hopefully this little example will pinpoint what's going on.
>>
>> Thx Kev for sticking with me :)
>>
>> --Kurt
>>
>>
>>
>> Kevin Sutter wrote:
>>
>>> Hi Kurt,
>>> Since you have the environment to reproduce this, could you run with the
>>> OpenJPA trace enabled?  I'm not even sure what I'm looking for at this
>>> point.  :-)  But, maybe there's some tracing that would point us in the
>>> right direction.
>>>
>>> <property name="openjpa.Log" value="DefaultLevel=TRACE"/>
>>>
>>> We have been always been on the watch for memory leaks, so this one is
>>> puzzling.  You did mention blocked threads as well.  At this point, you
>>> are
>>> probably hitting blocked threads due to the GC and Memory problems.  But,
>>> we
>>> did recently resolve some locking situations [1] in our
>>> MetaDataRepository
>>> that could affect scaling (under extreme load).  This was resolved in the
>>> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release, so
>>> that should be coming soon).
>>>
>>> Does your WebService version of the problem use EntityManager
>>> transactions,
>>> or is the transaction service coming from Tomcat?  And, does your
>>> WebService
>>> version also do the em.clear() and em.close()?  Basically, are you using
>>> application-managed or container-managed persistence while running as a
>>> WebService?
>>>
>>> I'm very interested in helping to resolve this issue.  Like I mentioned,
>>> we
>>> do a lot of memory leakage and scalability testing of the OpenJPA
>>> solution
>>> and we're not aware of anything.  Although there is always the
>>> possibility
>>> that something was resolved in the 1.2.x branch after the 1.2.1 release
>>> that
>>> I'm not remembering...
>>>
>>> Thanks for your help,
>>> Kevin
>>>
>>> [1]  https://issues.apache.org/jira/browse/OPENJPA-250
>>>
>>> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <kurt.stam@gmail.com>
>>> wrote:
>>>
>>>
>>>
>>>> Thanks Kevin,
>>>>
>>>> We're enhancing at build time:
>>>>
>>>>
>>>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>>>>
>>>> Yeah we've been running load tests and things are nice and stable with
>>>> Hibernate but with Openjpa we see increasing memory use, blocking
>>>> threads
>>>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our
>>>> preference would be to ship with openjpa by default; but our build
>>>> supports
>>>> both hibernate and openjpa.
>>>>
>>>> And yes we use openjpa 1.2.1 (latest stable version).
>>>>
>>>> --Kurt
>>>>
>>>>
>>>> Kevin Sutter wrote:
>>>>
>>>>
>>>>
>>>>> Interesting detective work, Kurt.  Thanks.
>>>>>
>>>>> Why the WebService version of the app would behave differently as far
>>>>> as
>>>>> GC
>>>>> is concerned is a mystery.  And, you said that plugging in Hibernate
>>>>> into
>>>>> this scenario, everything works okay?  Very confusing.
>>>>>
>>>>> How are you performing the Entity enhancement processing?  Are you
>>>>> pre-enhancing via your build process?  Or, are you using the -javaagent
>>>>> mechanism?  Or, are you falling back to the subclassing support within
>>>>> OpenJPA?  (See [1] for more information on these questions in case they
>>>>> don't make sense.)
>>>>>
>>>>> This would be one area that is different between Hibernate and OpenJPA
>>>>> --
>>>>> enhancement processing.
>>>>>
>>>>> In the Tomcat environment, you may be falling back to the subclassing
>>>>> support (which we do not recommend) and hitting a memory leak with
>>>>> that.
>>>>>
>>>>> You said OpenJPA 1.2.x, right?
>>>>>
>>>>> Just a couple of thoughts on the subject...
>>>>> Kevin
>>>>>
>>>>> [1]
>>>>>
>>>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <kurt.stam@gmail.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> The same code executed straight from a java client (inVM) shows no
>>>>>> memory
>>>>>> leak.
>>>>>>
>>>>>> So is the fact that it is WebService significant then? What else
can
>>>>>> be
>>>>>> different? I think one thread remains up, and somehow this causes
>>>>>> openjpa
>>>>>> not being able to clean up after itself. What can I do to debug this
>>>>>> more? I
>>>>>> can actually see in the profiler that the objects are allocated by
the
>>>>>> WebService, but why aren't they cleaned up?
>>>>>>
>>>>>> Thx,
>>>>>>
>>>>>>
>>>>>> --Kurt
>>>>>>
>>>>>>
>>>>>> Kurt T Stam wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Thanks Kevin, thanks for your response.
>>>>>>>
>>>>>>> I just replaced the static call by:
>>>>>>>
>>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>             apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>>>>>             //MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>> apiAuthToken);
>>>>>>>
>>>>>>> which did not make a difference.
>>>>>>>
>>>>>>> I'm wondering if the fact that my class is a webservice makes
a
>>>>>>> difference. I'll try extracting it into
>>>>>>> a regular class with a main method and profile that. At least
I know
>>>>>>> that
>>>>>>> I didn't forget something
>>>>>>> completely obvious..
>>>>>>>
>>>>>>> --Kurt
>>>>>>>
>>>>>>> Kevin Sutter wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Kurt,
>>>>>>>> I agree that this is very common usage of the JPA programming
model.
>>>>>>>>  And,
>>>>>>>> we are not aware of any memory leaks.  About the only thing
that
>>>>>>>> jumps
>>>>>>>> out
>>>>>>>> at me is the following two lines:
>>>>>>>>
>>>>>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>              MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>> apiAuthToken);
>>>>>>>>
>>>>>>>> What do these do?  Can you comment these out and see if the
memory
>>>>>>>> leak
>>>>>>>> still exists?  Since you are passing the modelAuthToken into
this
>>>>>>>> method,
>>>>>>>> I
>>>>>>>> don't know what it's doing with the reference and could it
be
>>>>>>>> holding
>>>>>>>> onto
>>>>>>>> something to prevent the GC from cleaning up?
>>>>>>>>
>>>>>>>> The rest of your example seems very straight forward with
creating
>>>>>>>> and
>>>>>>>> persisting objects.
>>>>>>>>
>>>>>>>> Kevin
>>>>>>>>
>>>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <curtisr7@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> If you change the 1000 to something like 1000000... does
your
>>>>>>>>> application
>>>>>>>>> go
>>>>>>>>> OOM? Are you running in a JSE environment? What is
>>>>>>>>> PersistenceManager?
>>>>>>>>>
>>>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <kurt.stam@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> BTW I'm running with the cache off
>>>>>>>>>>
>>>>>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>>>>>
>>>>>>>>>> (that turns it off right?)
>>>>>>>>>>
>>>>>>>>>> --Kurt
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Kurt T Stam wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hi guys,
>>>>>>>>>>>
>>>>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records
in the
>>>>>>>>>>> database.
>>>>>>>>>>>
>>>>>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>>>>>        EntityManager em = PersistenceManager.getEntityManager();
>>>>>>>>>>>        EntityTransaction tx = em.getTransaction();
>>>>>>>>>>>        try {
>>>>>>>>>>>            tx.begin();
>>>>>>>>>>>                // Generate auth token and store
it!
>>>>>>>>>>>            String authInfo = AUTH_TOKEN_PREFIX
+
>>>>>>>>>>> UUID.randomUUID();
>>>>>>>>>>>            org.apache.juddi.model.AuthToken modelAuthToken
= new
>>>>>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>>>>>            if (authInfo != null) {
>>>>>>>>>>>                modelAuthToken.setAuthToken(authInfo);
>>>>>>>>>>>                modelAuthToken.setCreated(new
Date());
>>>>>>>>>>>                modelAuthToken.setLastUsed(new
Date());
>>>>>>>>>>>                modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>>>>>                modelAuthToken.setNumberOfUses(0);
>>>>>>>>>>>                modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>>>>>                  em.persist(modelAuthToken);
>>>>>>>>>>>            }
>>>>>>>>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>>              MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>>> apiAuthToken);
>>>>>>>>>>>              tx.commit();
>>>>>>>>>>>                      } finally {
>>>>>>>>>>>            if (tx.isActive()) {
>>>>>>>>>>>                tx.rollback();
>>>>>>>>>>>            }
>>>>>>>>>>>            em.clear();
>>>>>>>>>>>            em.close();
>>>>>>>>>>>        }
>>>>>>>>>>>    }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [ISSUE]
>>>>>>>>>>> After it leaving this code I end up with a 1000
>>>>>>>>>>> org.apache.juddi.model.AuthToken objects in memory.
I've been
>>>>>>>>>>> using
>>>>>>>>>>> the
>>>>>>>>>>> profiler, and these objects cannot be garbage
collected.
>>>>>>>>>>>
>>>>>>>>>>> This seems to be pretty the most common use case
of using an
>>>>>>>>>>> OR-mapping
>>>>>>>>>>> tool, so I find it hard to believe openjpa has
a memory leak
>>>>>>>>>>> here.
>>>>>>>>>>> Does
>>>>>>>>>>> anyone see what I'm doing wrong? Or can someone
point me to an
>>>>>>>>>>> example
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> that
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> does not exhibit this behavior? BTW same code using
hibernate does
>>>>>>>>>> not
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> accumulate these objects.
>>>>>>>>>>>
>>>>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thx,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Kurt
>>>>>>>>>>>
>>>>>>>>>>> Apache jUDDI.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thanks,
>>>>>>>>> Rick
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message