openjpa-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alejandro Abdelnur <tuc...@gmail.com>
Subject Re: OOM with JDBCConfigurationImpl holding 1+ GB of heap
Date Tue, 28 May 2013 16:24:21 GMT
Rick,

Thanks for following up.

After you email I've looked again at the hprof dump and the number
of org.apache.openjpa.persistence.EntityManagerFactoryImpl isntances is 1.
I've also noticed the heap has
500+ org.apache.openjpa.persistence.EntityManagerImpl instances, but they
are referenced from a finalizer; thus in queue to be GC. So, it would
argue, the leak does not come from there.

I'll give it a ry using 2.2.0 to see if that makes a difference.

Thanks again.

Alejandro



On Fri, May 24, 2013 at 12:05 PM, Rick Curtis <curtisr7@gmail.com> wrote:

> After digging into the heapdump, I'm fairly certain that somewhere in your
> application EntityManagerFactories are being created, yet they are never
> closed. I know you pointed me at your JPA service source code, but there
> must be something else going on.  For giggles you could move to version
> 2.2.0, where we made some changes[1] to better tolerate applications that
> fail to close EMFs. The JIRA talks about additional diagnostics, but we
> just changed a bit of our code to use WeakReferences in hopes of avoiding
> these sorts of OOMs. If moving to 2.2.0 makes your problem go away, you'll
> need to look elsewhere in your code for EMF creation. Hope this helps.
>
> Thanks,
> Rick
>
>
> [1] https://issues.apache.org/jira/browse/OPENJPA-2042
>
>
> On Fri, May 24, 2013 at 10:49 AM, Alejandro Abdelnur <tucu00@gmail.com
> >wrote:
>
> > Hi Rick, a hprof dump is available at
> > http://people.apache.org/~tucu/tmp/openjpaleak.hprof.tar.gz
> >
> > Thank you very much for looking into this.
> >
> > Alejandro
> >
> >
> > On Fri, May 24, 2013 at 7:46 AM, Rick Curtis <curtisr7@gmail.com> wrote:
> >
> > > Can you post a heapdump somewhere for review?
> > >
> > > Thanks,
> > > Rick
> > >
> > >
> > > On Thu, May 23, 2013 at 6:53 PM, Alejandro Abdelnur <tucu00@gmail.com
> > > >wrote:
> > >
> > > > Rick,
> > > >
> > > > We create a single EMF:
> > > >
> > > >
> > > >
> > > >
> > >
> >
> https://github.com/apache/oozie/blob/trunk/core/src/main/java/org/apache/oozie/service/JPAService.java#L160
> > > >
> > > > And we use create/use/dispose all EMs using a command patter:
> > > >
> > > >
> > > >
> > > >
> > >
> >
> https://github.com/apache/oozie/blob/trunk/core/src/main/java/org/apache/oozie/service/JPAService.java#L209
> > > >
> > > > Any other idea/hint?
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Wed, May 22, 2013 at 11:32 AM, Rick Curtis <curtisr7@gmail.com>
> > > wrote:
> > > >
> > > > > Alejandro -
> > > > >
> > > > > It appears that your application is creating
> EntityManagerFactories,
> > > yet
> > > > > they are never getting closed.
> > > > >
> > > > > Thanks,
> > > > > Rick
> > > > >
> > > > >
> > > > > On Wed, May 22, 2013 at 12:40 PM, Alejandro Abdelnur <
> > tucu00@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Rick, by looking at the other thread memory analyzer image,
the
> > > issues
> > > > > seem
> > > > > > to be different.
> > > > > >
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > >
> > > > > > On Mon, May 20, 2013 at 9:47 AM, Rick Curtis <curtisr7@gmail.com
> >
> > > > wrote:
> > > > > >
> > > > > > > Alejandro --
> > > > > > >
> > > > > > > Is the problem that you're asking about the same as Hovan?
If
> so,
> > > > lets
> > > > > > > consolidate this conversation down to a single thread.
> > > > > > >
> > > > > > >
> > > > >
> > >
> http://openjpa.208410.n2.nabble.com/OOM-caused-by-OpenJPA-td7583869.html
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://openjpa.208410.n2.nabble.com/OOM-with-JDBCConfigurationImpl-holding-1-GB-of-heap-td7583862.html
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Rick
> > > > > > >
> > > > > > >
> > > > > > > On Mon, May 20, 2013 at 11:38 AM, Kevin Sutter <
> > kwsutter@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Alejandro,
> > > > > > > > It looks like something is holding onto references...
 How
> are
> > > you
> > > > > > > > enhancing your classes?  At build time?  Or, some
other
> > > mechanism?
> > > > > > > >
> > > > > > > > Does this memory leak happen with any of your configured
> > > databases?
> > > > > >  Or,
> > > > > > > > just select ones?
> > > > > > > >
> > > > > > > > This shouldn't make a difference, but specifying your
Entity
> > > > classes
> > > > > > via
> > > > > > > > both <class> and the openjpa.MetaDataFactory
property is
> > > overkill.
> > > > >  The
> > > > > > > > <class> elements are sufficient and is the standard,
so I'd
> > > suggest
> > > > > > > > sticking with that mechanism and removing the
> > > > openjpa.MetaDataFactory
> > > > > > > > property.
> > > > > > > >
> > > > > > > > We do considerable memory leak testing both as a standalone
> JPA
> > > > > > provider
> > > > > > > as
> > > > > > > > well as the JPA provider for WebSphere Application
Server.
>  We
> > > > could
> > > > > > have
> > > > > > > > missed something due to your configuration requirements,
but
> > this
> > > > > looks
> > > > > > > to
> > > > > > > > be a pretty major leak...
> > > > > > > >
> > > > > > > > Can you narrow down or simplify your scenario to the
point of
> > > > > sharing a
> > > > > > > > project that demonstrates the leak?  Or, maybe there
is
> memory
> > > data
> > > > > > dumps
> > > > > > > > that can be shared to help diagnose the issue?
> > > > > > > >
> > > > > > > > Kevin
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, May 17, 2013 at 4:50 PM, Alejandro Abdelnur
<
> > > > > tucu00@gmail.com
> > > > > > > > >wrote:
> > > > > > > >
> > > > > > > > > And .... I'm back.....
> > > > > > > > >
> > > > > > > > > It seems that removing runtime enhancement has
some effect,
> > but
> > > > the
> > > > > > > heap
> > > > > > > > > keeps building up in the same spot.
> > > > > > > > >
> > > > > > > > > The following link is show where is the memory
sink
> > > > > > > > >
> > > > > > > > >   http://people.apache.org/~tucu/tmp/OpenJPALeak.png
> > > > > > > > >
> > > > > > > > > I'd appreciate any ideas/suggestions.
> > > > > > > > >
> > > > > > > > > Thanks
> > > > > > > > >
> > > > > > > > > Alejandro
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, May 10, 2013 at 10:11 AM, Kevin Sutter
<
> > > > kwsutter@gmail.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > It shoudn't...  But, with that flag set
as "supported",
> we
> > > > can't
> > > > > be
> > > > > > > > sure
> > > > > > > > > > that the entities were actually enhanced
at build time.
>  By
> > > > > setting
> > > > > > > > this
> > > > > > > > > > property to "unsupported" or removing it
altogether, then
> > if
> > > an
> > > > > > > > > unenhanced
> > > > > > > > > > entity class is loaded, we'll log a message
and quit
> > > > processing.
> > > > > >  So,
> > > > > > > > > let's
> > > > > > > > > > go this route first and then we'll deal
with any
> potential
> > > > memory
> > > > > > > leak.
> > > > > > > > > >
> > > > > > > > > > Thanks, Kevin
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Fri, May 10, 2013 at 12:05 PM, Alejandro
Abdelnur <
> > > > > > > tucu00@gmail.com
> > > > > > > > > > >wrote:
> > > > > > > > > >
> > > > > > > > > > > I'm in the process of verifying if
Kevin's suggestion
> > fixes
> > > > the
> > > > > > > > > problem.
> > > > > > > > > > >
> > > > > > > > > > > While making the necessary changes,
I've notice we were
> > > > already
> > > > > > > > > enhancing
> > > > > > > > > > > the classes at build time.
> > > > > > > > > > >
> > > > > > > > > > > If the classes were enhanced at build
time, still would
> > be
> > > we
> > > > > see
> > > > > > > the
> > > > > > > > > > leak
> > > > > > > > > > > if the runtime enhancement flag is
enabled?
> > > > > > > > > > >
> > > > > > > > > > > Thanks
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Thu, May 9, 2013 at 9:15 AM, Alejandro
Abdelnur <
> > > > > > > tucu00@gmail.com
> > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Thanks Kevin, will try that.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, May 9, 2013 at 8:59 AM,
Kevin Sutter <
> > > > > > kwsutter@gmail.com
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > >> Don't use this:
> > > > > > > > > > > >>
> > > > > > > > > > > >> <property name="openjpa.RuntimeUnenhancedClasses"
> > > > > > > > > value="supported"/>
> > > > > > > > > > > >>
> > > > > > > > > > > >> That is probably your issue.
 Reference this page
> for
> > > more
> > > > > > > > > > information:
> > > > > > > > > > > >> http://openjpa.apache.org/entity-enhancement.html
> > > > > > > > > > > >>
> > > > > > > > > > > >> Good luck, Kevin
> > > > > > > > > > > >>
> > > > > > > > > > > >>
> > > > > > > > > > > >>
> > > > > > > > > > > >>
> > > > > > > > > > > >> On Thu, May 9, 2013 at 10:17
AM, Alejandro Abdelnur
> <
> > > > > > > > > tucu00@gmail.com
> > > > > > > > > > > >> >wrote:
> > > > > > > > > > > >>
> > > > > > > > > > > >> > Hi Rick,
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > Thanks for looking into
this, the persistence.xml
> > file
> > > > we
> > > > > > are
> > > > > > > > > using
> > > > > > > > > > > is:
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/oozie/blob/trunk/core/src/main/resources/META-INF/persistence.xml
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > The EMF is instatiated
here:
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/oozie/blob/trunk/core/src/main/java/org/apache/oozie/service/JPAService.java#L156
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > Thanks again.
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > Alejandro
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > On Thu, May 9, 2013 at
6:06 AM, Rick Curtis <
> > > > > > > curtisr7@gmail.com
> > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > > Can we see the contents
of your persistence.xml
> > > file?
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > On Wed, May 8, 2013
at 6:34 PM, Alejandro
> > Abdelnur <
> > > > > > > > > > > tucu00@gmail.com>
> > > > > > > > > > > >> > > wrote:
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > > We are seeing
this using OpenJPA 2.1.0 in
> Oozie.
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > A JDBCConfigurationImpl
seems to be holding a
> > > large
> > > > > > amount
> > > > > > > > of
> > > > > > > > > > > memory
> > > > > > > > > > > >> > via
> > > > > > > > > > > >> > > > its dataCacheTimeout
(then listeners) instance
> > > > > variable
> > > > > > > > > keeping
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > Class     
                    # Objects  Used
> > > Heap
> > > > > > > > Retained
> > > > > > > > > > > Heap
> > > > > > > > > > > >> > > >
> > > > > > > > > > >
> > > > > >
> -------------------------------------------------------------------
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > o.a.o.j.meta.MappingRepository
 1,902
> > 304,320
> > > > > > > > > > 904,410,184
> > > > > > > > > > > >> > > > o.a.o.j.meta.ClassMapping
      3,824
> > > 1,223,680
> > > > > > > > > > 105,073,624
> > > > > > > > > > > >> > > >
> > > > > > > > > > >
> > > > > >
> -------------------------------------------------------------------
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > We are using
a single EntityManagerFactory and
> > we
> > > > are
> > > > > > > > closing
> > > > > > > > > > all
> > > > > > > > > > > >> > > > EntityManager
instances correctly (via a
> command
> > > > > > pattern).
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > All caching
settings are default ones.
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > Any hint where
to look would be appreciated
> it.
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > Thanks
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > > > Alejandro
> > > > > > > > > > > >> > > >
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> > > --
> > > > > > > > > > > >> > > *Rick Curtis*
> > > > > > > > > > > >> > >
> > > > > > > > > > > >> >
> > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > *Rick Curtis*
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > *Rick Curtis*
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > *Rick Curtis*
> > >
> >
>
>
>
> --
> *Rick Curtis*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message