Mailing-List: contact user-help@cayenne.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cayenne.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <4F44C4B9.20308@lindesay.co.nz>
Date: Wed, 22 Feb 2012 23:34:33 +1300
From: Andrew Lindesay <apl@lindesay.co.nz>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7;
 rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2
MIME-Version: 1.0
To: user@cayenne.apache.org
Subject: Re: Question on ObjectContext and Deserialization
References: <4F3A37C9.2080708@lindesay.co.nz>
 <C99D0DBC-72A6-46D7-9F4F-8BB4E616E4EB@objectstyle.org>
 <199079C1-27EA-4F09-A9B2-736A45DBF5D4@objectstyle.org>
 <4F3E299E.8040807@lindesay.co.nz>
 <7C266DED-920A-4D14-BE77-ACD29C4907D0@objectstyle.org>
 <4F3ED9C8.2070906@lindesay.co.nz>
In-Reply-To: <4F3ED9C8.2070906@lindesay.co.nz>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hi there;

It does appear that the problem is in an Apache Commons abstract 
superclass.  What is happening is that whilst running;

AbstractReferenceMap.doReadObject(..)

...the method...

AbstractReferenceMap.checkCapacity(..)

...is invoked and it seems to be doubling the size of the Map's entry 
array on de-serialization.  Through sequential serialization + 
de-serialization this yields the exponential growth in the memory 
consumption.

It seems that the problem is caused by the 'threshold' value being 
calculated after to populating the Map during de-serialization.

I have re-built 3.0-STABLE with commons 3.2.1 and it still seems to 
exhibit the same problem.

The release cycles for commons seems to be quite long these days.  Hmmm.....

Regards;

On 18/02/12 11:50 AM, Andrew Lindesay wrote:
> Hi Andrus;
>
> It does seem rather strange.
>
> I'm starting the application with maven (from Eclipse) and
> jetty-maven-plugin. No hot redeploy. No actual container per se. The
> class-loader should be the same so shouldn't be re-creating "isolated
> singletons or statics". I'm using a Filter to serialize-->deserialize
> the attributes on the HttpSession in order to test how some
> infrastructure handles serialization-->deserialization of the session
> attributes.
>
> If I explicitly don't serialize-->deserialize the ObjectContext then
> there is no problem; so serializing ObjectContext as the root object
> does seem to cause the problem.
>
> The incremental increase in the memory consumption seems to be roughly
> exponential in relation to the number of deserializations of the
> ObjectContext. That may be a clue; I may download Apache commons and see
> if I can see anything obvious there in the LRU map. Like you say though;
> it probably shouldn't be serializing this stuff anyway though.
>
> It looks like "SQLTemplateResourceManager" is created via reflection so
> it may be hard to profile for memory usage.
>
> I'll log a Jira when I get a moment.
>
> Regards;
>
> On 17/02/12 11:29 PM, Andrus Adamchik wrote:
>> Interesting... I didn't realize we have a static Velocity runtime in
>> the app. This must to be changed to instance variable with correct
>> scope, (and if needed shut down properly when ServerRuntime is
>> shutdown). Could you please log a Jira - we need to investigate.
>>
>> BTW, are you doing hot redeploy of your app? This in combination with
>> static Velocity runtime might produce the observed behavior.
>>
>> Andrus
>>
>>
>> On Feb 17, 2012, at 1:19 PM, Andrew Lindesay wrote:
>>
>>> Hi;
>>>
>>> It looks like it is holding onto lots (hundreds of megs) of;
>>>
>>> org.apache.commons.collections.map.AbstractHashedMap$HashEntry[]
>>> > org.apache.commons.collecitons.map.LRUMap
>>> >> org.apache.cayenne.access.jdbc.SQLTemplateResourceManager
>>> >>> ...velocity
>>>
>>> Not quite understanding yet... as it seems like this wouldn't be
>>> something stored on an object context.
>>>
>>> Regards;
>>>
>>> On 16/02/12 7:50 PM, Andrus Adamchik wrote:
>>>>>> an OutOfMemory -- even with substantial quantities of memory
>>>>>> available.
>>>>
>>>> This is a hint actually.. Maybe the problem is not heap size, but
>>>> perm gen size instead? Try increasing that:
>>>>
>>>> -XX:MaxPermSize=256m
>>>>
>>>> On Feb 16, 2012, at 9:44 AM, Andrus Adamchik wrote:
>>>>
>>>>> This is strange indeed. Haven't seen such behavior. May need some
>>>>> profiling on your end I guess.
>>>>>
>>>>> Andrus
>>>>>
>>>>>
>>>>> On Feb 14, 2012, at 1:30 PM, Andrew Lindesay wrote:
>>>>>> Hello;
>>>>>>
>>>>>> I have a little question on the ObjectContext in 3.0.2.
>>>>>>
>>>>>> I'm keeping an ObjectContext in an HttpSession instance's
>>>>>> attributes for re-use between request-response cycles. When I
>>>>>> serialize this ObjectContext instance, it (+ other objects) stores
>>>>>> out to< 8k in my test case, but then deserializing the
>>>>>> ObjectContext causes an OutOfMemory -- even with substantial
>>>>>> quantities of memory available.
>>>>>>
>>>>>> Has anybody else seen this or is there anything to be weary of
>>>>>> before I look into it further?
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> --
>>>>>> Andrew Lindesay
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Andrew Lindesay
>>> www.silvereye.co.nz
>>>
>>
>>
>
>


-- 
Andrew Lindesay
www.silvereye.co.nz