cayenne-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Baldwin <jfbald...@earthlink.net>
Subject Re: Visibility & Monitoring
Date Fri, 09 Apr 2010 00:12:51 GMT
Michael,

thanks for your input.  I have started analyzing this in detail and the problem has become
multidimensional.

Assuming a webstore pattern:

First, if the DataContext never goes out of scope, and the DataContext never releases references
to DataObjects, then eventually *all* of the products will be loaded into the DataContext.
 I can't imagine that this would be the default behavior.

Second: webstore reality trumps delegation to DBMS of filtering & sort-ordering, since
there are not only search-based fetches, but productList relationships to "Category" and "Vendor"
objects.   I was told by your team in a previous email that I (as designer) have no access
to modifying or overriding relationship select-queries.   Therefore if I use the Cayenne relationship
(for a Vendor or Category) to get a list of Products, I would would be forced to apply a list-based
filter or sort-ordering.

*If* the DataContext never releases DataObjects, then eventually (as a result of the relationship
imposed list-based filtering/ordering), every Product will be faulted and loaded into memory.

Michael, this can't be correct because then it would mimic the behavior of a memory leak.

On the other hand, if it is correct, then my only option left is to completely disable all
Cayenne relationships, and do custom select queries which replace the relationships while
including the required filtering and sort ordering.  This kind-of sounds screwy as well.

Therefore, I am now more confused about this than when I started. :)

My hope is that the DataContext now allows more control over garbage collecting objects.

Joe



On Apr 8, 2010, at 7:32 PM, Michael Gentry wrote:

> My Q4 comment may be based more upon my usage of Cayenne 2 than 3.  I
> know there is some weak reference stuff in 3 that I haven't looked
> into yet and perhaps it will automatically clean things up now (which
> would be wonderful).  I've always taken the approach of evicting
> things I no longer wanted so I could control my memory footprint
> better.  Maybe Andrus/Andrey/etc will chime in with better info here
> if I am incorrect on this one.
> 
> mrg
> 
> 
> On Thu, Apr 8, 2010 at 5:49 PM, Joe Baldwin <jfbaldwin@earthlink.net> wrote:
>> Hi Michael,
>> 
>> Thank you for your methodical answers.
>> 
>> RE Q3
>> I suspected that this might be one of my problems, and I am trying to redesign so
that I only filter & order small result lists.
>> 
>> RE Q4
>> Oops!   OK, this one totally threw me.  Given my understanding of the behavior and
usage of the DataContext is somewhere between "wow" to "hmmm, I didn't know that",   Perhaps
I need to review a primer. :)
>> 
>> But seriously:
>> Since I am deploying a web-app, my understanding is that in that type of scenario
the container (via the Tomcat CayenneFilter) is instantiating and controlling the DataContext
(and I am, presumably now accessing it via BaseContext.getThreadObjectContext();)  However,
beyond that piece of understanding, I am not sure what to do with it in order to manage objects.
>> 
>> I read your docs once more and it says that the DataContext uses "weak references
to store registered objects".  I *thought* this meant that (assuming I use the default configuration),
that the DataContext would allow the DataObjects to be garbage collected, if there is no reference
(Java Reference) and if I did not implement query caching.
>> 
>> I am confused by reading you Q4 comment and then the "Memory Management Strategy"
documentation.
>> 
>> (Note: I am attempting to learn more about Cayenne so that I can optimize (and re-factor)
my code.  I am still uncertain whether I should be implementing caching, not implementing
caching, initiating garbage collection (after freeing objects), or just sitting back and letting
Cayenne DataContext handle it.)
>> 
>> Thanks,
>> Joe
>> 
>> 
>> 
>> 
>> On Apr 8, 2010, at 5:18 PM, Michael Gentry wrote:
>> 
>>> Hi Joe,
>>> 
>>> Assuming 2000 products fetched with a paginated query and a page size
>>> of 50, I believe the answers are ...
>>> 
>>> Q1: 2000 hollow objects are created.   Cayenne fetches the primary
>>> keys of the objects first in a paginated query.
>>> 
>>> Q2: Whichever page (or pages) you access in the paginated list will
>>> fault those objects for that page (grouped by 50 at a time) into
>>> memory.
>>> 
>>> Q3: If you try to filter or sort in-memory, then it will have to fetch
>>> all of the objects.  Might be better to re-issue a query to the DB.
>>> 
>>> Q4: If your list pointers go out-of-scope, but the DataContext is
>>> still in-scope, then the objects will still be in the DataContext
>>> until you evict them.
>>> 
>>> mrg
>>> 
>>> 
>>> On Thu, Apr 8, 2010 at 4:24 PM, Joe Baldwin <jfbaldwin@earthlink.net> wrote:
>>>> Andrus,
>>>> 
>>>> In this same context, I have a question about what behavior I should expect
to see.
>>>> 
>>>> Let us assume for this scenario:
>>>> 1. a web deployment with Tomcat, typical webstore pattern
>>>> 2. 2000-5000 products with an attribute "name"
>>>> 3. custom search engine built on top of cayenne
>>>>        a. mostly standard configuration
>>>>        b. set PageSize to 50
>>>>        c. using (presumably) default caching
>>>>        d. custom factory class method used to create a select query
>>>> 4. a user searches for "guitar" substring in the name field
>>>> 
>>>> Questions:
>>>> 1. Based on my tests, it appears that "hollow" objects are created but no
faults are fired. (true?)
>>>> 2. If I display any product data for a subset less than PageSize, then only
<PageSize> number of objects are faulted (true ?)
>>>> 3. If I execute "filterObjects" or "orderList" on the resulting list, I assume
the fault is fired for all objects in the list. (true ?)
>>>> 4. What normally happens to these objects if my list pointers go out of scope?
 (Are they GarbageCollected per Java rules?)
>>>> 
>>>> (Note: I have not been able to get much visibility in understanding (3) &
(4), but I think my tests show that 1 & 2 are correct.)
>>>> Joe
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Apr 8, 2010, at 12:42 PM, Andrus Adamchik wrote:
>>>> 
>>>>> BTW, we started on some monitoring hooks inside Cayenne runtime in 3.0
per http://issues.apache.org/jira/browse/CAY-1173 , however then it was decided that it'll
work much better with the 3.1 DI-based stack. So this is waiting for its time.
>>>>> 
>>>>> Andrus
>>>>> 
>>>>> 
>>>>> On Apr 7, 2010, at 6:16 PM, Joe Baldwin wrote:
>>>>> 
>>>>>> I am attempting to do more performance monitoring (with an eye towards
optimizing my design & use of DataObject lists and lifetime).
>>>>>> 
>>>>>> The first thing I would like to do is monitor how many DataObjects
are created and which JSP session they belong.
>>>>>> 
>>>>>> My idea was to simply create a constructor for the DataObject of
interest and place monitoring hooks inside. I would assume that I could also create a finalize
method to do similar things.
>>>>>> 
>>>>>> Is this the best way to create custom monitoring using Cayenne DataObjects
or is there a better way?
>>>>>> 
>>>>>> (BTW, even after reading the docs, I am still somewhat mystified
by how my configuration specifications control the life cycle and performance.  So I am hoping
these custom hooks will make my design decisions more visible, especially with multiple users.)
>>>>>> 
>>>>>> Thanks,
>>>>>> Joe
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>> 
>> 


Mime
View raw message