Mailing-List: contact users-help@openjpa.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@openjpa.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Message-ID: <49809FD2.40107@cyberspaceroad.com>
Date: Wed, 28 Jan 2009 18:11:30 +0000
From: Adam Hardy <adam.sql@cyberspaceroad.com>
User-Agent: Icedove 1.5.0.14eol (X11/20080724)
MIME-Version: 1.0
To: users@openjpa.apache.org
Subject: Re: out of memory issues
References: <497DF6F1.3000907@cyberspaceroad.com>
	 <72c1350f0901270831r74b6cafdnf81db9703e46c7f0@mail.gmail.com>
	 <498035D7.8000802@cyberspaceroad.com>
	 <c4a1048f0901280635j1f60e484s49fb4456431eb60d@mail.gmail.com>
	 <49807F0C.4020604@cyberspaceroad.com>
 <c4a1048f0901280835v5a82d724o31fbae044282b728@mail.gmail.com>
 <49809001.3080108@cyberspaceroad.com>
In-Reply-To: <49809001.3080108@cyberspaceroad.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Just to report back on my results using Java 6 class retransformation: the 
performance isn't any better.

I just double checked what was going on in terms of workload and I 
underestimated the number of child entities that I am creating. It's more in the 
area of 30,000 or 4 to 5 times more than I thought.


Adam Hardy on 28/01/09 17:04, wrote:
> Sorry, didn't mean compile-time enhancement, I was just mindlessly 
> parroting the words I was reading :$
> 
> I started off using nothing, making no changes. No javaagent, nada.
> 
> Now I've upgraded to Java 1.6 and I'm trying to get that working but 
> today's not being very productive. However "Java 6 class 
> retransformation" certainly sounds hopeful.
> 
> I do see on my workstation that I still get the warning in my test run that
> 
> " This means that your application will be less efficient than it would 
> if you ran the OpenJPA enhancer. (openjpa)"
> 
> although at least it doesn't tell me what it did with Java 1.5:
> 
> " Additionally, lazy loading will not be available for one-to-one and 
> many-to-one persistent attributes in types using field access; they will 
> be loaded eagerly instead."
> 
> 
> 
> Jeremy Bauer on 28/01/09 16:35, wrote:
>> Adam,
>>
>> Did you mean runtime enhancement is being used?  (Want to make sure I 
>> didn't
>> steer you down the wrong path - I don't think so, based on your statement
>> about adding enhancement to the maven build).
>> I'm not sure if 1.6 will help, but it may be worth a shot.  (Anyone?)
>>  Besides using compile-time enhancement you could also try to 
>> configure your
>> web app server to use the agent enhancer.
>>
>> -Jeremy
>>
>> On Wed, Jan 28, 2009 at 9:51 AM, Adam Hardy 
>> <adam.sql@cyberspaceroad.com>wrote:
>>
>>> Hi Jeremy,
>>>
>>> compile-time enhancement is being used.
>>>
>>> I figured it would have to be something like that. I just upgraded to 
>>> JDK
>>> 1.6 to see if the enhancement provided that way is better, so fingers
>>> crossed.
>>>
>>> Otherwise I'll have to build in the enhancement stage into the maven 
>>> build
>>> and from what it looks like, the current way of doing that hasn't been
>>> changed since OpenJPA v0.9.6.
>>>
>>>
>>> regards
>>> Adam
>>>
>>>
>>> Jeremy Bauer on 28/01/09 14:35, wrote:
>>>
>>>> Adam,
>>>> Are you using compile time enhancement or does the web container do
>>>> enhancement?  If not, runtime enhancement will be used and I've seen 
>>>> cases
>>>> where non-agent runtime enhancement can significantly increase memory
>>>> usage.
>>>>
>>>> If you aren't certain, an easy way to check is to disable runtime
>>>> enhancement via:
>>>>
>>>>          <property name="openjpa.RuntimeUnenhancedClasses"
>>>> value="unsupported"/>
>>>>
>>>> If your classes are not enhanced, the app will fail with an 
>>>> exception to
>>>> that effect.
>>>>
>>>> -Jeremy
>>>>
>>>> On Wed, Jan 28, 2009 at 4:39 AM, Adam Hardy 
>>>> <adam.sql@cyberspaceroad.com
>>>>> wrote:
>>>>  Hi Mike,
>>>>> thanks for the input.
>>>>>
>>>>> The child objects are mapped entities and they total around 25 * 
>>>>> 300, so
>>>>> 7500.
>>>>>
>>>>> This is their structure:
>>>>>
>>>>>   private Long id;
>>>>>   private BigDecimal total;
>>>>>   private DollarReturn dollarReturn;
>>>>>   private TestAnalysis testAnalysis;
>>>>>
>>>>> where DollarReturn and TestAnalysis are also mapped entities.
>>>>> TestAnalysis
>>>>> will have 7500 of these in its child list. They extend a superclass:
>>>>>
>>>>>   private Long version;
>>>>>   private Long ownerId;
>>>>>   private Date created;
>>>>>   private Date modified;
>>>>>
>>>>> For what it's worth, they also have several transient properties.
>>>>>
>>>>> I didn't get a heap dump before. Now I have changed the architecture a
>>>>> bit
>>>>> to try to streamline the process, and the process doesn't crash 
>>>>> now, it
>>>>> just
>>>>> takes a very long time.
>>>>>
>>>>> Regards
>>>>> Adam
>>>>>
>>>>>
>>>>>
>>>>> Michael Dick on 27/01/09 16:31, wrote:
>>>>>
>>>>>  Hi Adam, just some quick thoughts,
>>>>>> How many child objects are we talking about, and are they entities or
>>>>>> just
>>>>>> objects which are persisted to the database?
>>>>>>
>>>>>> If you have heap dumps it might be interesting to see which object(s)
>>>>>> are
>>>>>> taking up the most memory. It sounds like the likely culprit is the
>>>>>> number
>>>>>> of child objects and any OpenJPA objects associated with them.
>>>>>>
>>>>>> -mike
>>>>>>
>>>>>> On Mon, Jan 26, 2009 at 11:46 AM, Adam Hardy <
>>>>>> adam.sql@cyberspaceroad.com
>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>  The webapp I am working on uses OpenJPA (or Hibernate depending 
>>>>>> on bugs
>>>>>>
>>>>>>> blocking progress) and I have just hit a problem that currently 
>>>>>>> looks
>>>>>>> like
>>>>>>> it will only be solved by violating the current architecture.
>>>>>>>
>>>>>>> I'm hoping I'm wrong and thought I'd ask first.
>>>>>>>
>>>>>>> The scenario is that the user can choose a whole range of financial
>>>>>>> instruments to put in a portfolio, and the webapp grabs them all on
>>>>>>> submission of the request and builds the object graph.
>>>>>>>
>>>>>>> The Manager class which is also the transactional interface then
>>>>>>> creates
>>>>>>> an
>>>>>>> Analysis entity to hold performance stats, and which may create for
>>>>>>> itself a
>>>>>>> large number of small child objects representing past performance.
>>>>>>>
>>>>>>> When the Analysis is done, the Manager call finishes and the
>>>>>>> transaction
>>>>>>> commits (or I commit the transaction in my unit test), and I get an
>>>>>>> out-of-memory exception.
>>>>>>>
>>>>>>> Presumably it's all the child object inserts causing the problem.
>>>>>>>
>>>>>>> Obviously I would like to do a flush before I run out of memory, but
>>>>>>> the
>>>>>>> Analysis entity object has no access to the entity manager. Or at 
>>>>>>> least
>>>>>>> it
>>>>>>> shouldn't have.
>>>>>>>
>>>>>>> The other problem is that the Analysis entity can't really be saved
>>>>>>> until
>>>>>>> the child objects are all created, so I would have to think of a 
>>>>>>> dirty
>>>>>>> work-around to allow me to save it first, to allow me to flush the
>>>>>>> child
>>>>>>> objects.
>>>>>>>
>>>>>>>
>>
> 
>