openjpa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Craig L Russell <Craig.Russ...@Sun.COM>
Subject Re: [jira] Commented: (OPENJPA-370) LoadFetchGroup annotation was not recognized during the fetch1
Date Thu, 01 Nov 2007 15:22:39 GMT
Hi Pinaki,

On Oct 31, 2007, at 9:46 PM, Pinaki Poddar wrote:

> Hi Craig,
> Let us keep two issues namely a) correctness and b) performance
> separate.
> The root question is
>   a) is current implementation of LoadFetchGroup incorrect?
> The secondary question is
>   b) does current implementation of LoadFetchGroup result in optimal
> performance?
>
> My current position on (a) is: no evidence conclusively suggests it to
> be incorrect.

The evidence I have is that the test case that's checked in runs and  
the test case is incorrect according to my interpretation of the load  
fetch group. In fact, there is no test method in TestFetchGroup that  
will exercise the load fetch group.

Are there tests in the BEA suite (perhaps written to JDO interface)  
that we can use to see how it works with Kodo? That might give us a  
clue as to where the anomaly lies.
>
> My current position on (b) is: Given fundamental data structures and
> strategies,  the current implementation is optimal but not minimal in
> SQL generated. To determine all the requisite fields within a single
> project of SQL query as in proposed alternative to minimize SQL may
> result in overall performance reduction as per Patrick's observation.

I'm speculating here, but I don't believe that you can explain a 20%  
performance regression by some additional code running when loading  
instances. You can explain it by executing more than the number of  
SQL statements. So my working hypothesis is that we're generating too  
much SQL when running with the patch compared to without the patch.
>
> I will try to support (a) with my own test cases (as positive example)
> and (b) analyze Teresa's test cases (as negative example) to ascertain
> my position or otherwise, eat my own words.

I'd like to see how your analysis of the currently checked in test  
case compares with mine (it's a comment on issue https:// 
issues.apache.org/jira/browse/OPENJPA-370 ) and see if we agree on  
the proper behavior. Then we can talk about implementation.

Craig
>
> Even if (a) is provably true, we should seek solution within current
> pathways than to introduce changes that may impair overall  
> performance.
>
>
>
> Pinaki Poddar
> 972.834.2865
>
>
>> -----Original Message-----
>> From: Craig.Russell@Sun.COM [mailto:Craig.Russell@Sun.COM]
>> Sent: Wednesday, October 31, 2007 11:14 PM
>> To: dev@openjpa.apache.org
>> Subject: Re: [jira] Commented: (OPENJPA-370) LoadFetchGroup
>> annotation was not recognized during the fetch1
>>
>> Hi Pinaki,
>>
>> On Oct 31, 2007, at 8:55 PM, Pinaki Poddar wrote:
>>
>>> Hi Craig,
>>>   I value your observation that we should be careful in using three
>>> terms: 'accessed', 'loaded' and 'fetched' in this context. Let me
>>> state my (limited or even inaccurate) understanding of these terms:
>>>   accessed : a field f is accessed when user application attempts to
>>> get its value for a managed instance X.
>>
>> ok
>>
>>>   loaded: a field f is loaded in X if the value for the field is
>>> considered to be the same as that of the datastore.
>>
>> I'd prefer to consider a field loaded after it's been fetched
>> from the datastore. The issue with using loaded in our
>> discussion is that "load fetch group" loses its context.
>>
>>>   fetched: a field f is fetched when its value is read from
>> datastore
>>> and set to the instance X
>>>
>>>   You may correct the above by my following statements are based on
>>> the above understanding.
>>>
>>>> Activating a fetch group doesn't do anything.
>>> A more accurate statement would be "execution of a query or find()
>>> operation with a fetch configuration that includes fetch group named
>>> "f4f5".
>>>
>>>> This is wrong. Only one SQL should be issued, to fetch a specific
>>>> instance.
>>> I beg to differ on qualifying it as 'wrong'. Number of SQL issued to
>>> meet a user contract can not be mandated. Trying to issue
>> only one SQL
>>> in this case within current implementaion/data structure strategy
>>> perhaps will lead to us to a solution similar ro what Teresa
>> outlines.
>>
>> Ok, but I'd say a key design decision is to limit the number
>> of SQL statements issued in order to retrieve fields from a
>> single instance to exactly one.
>>>
>>>> So this isn't exactly a good example, because it matters whether p1
>>>> (the instance of PObject) is already loaded into memory or not.
>>> I should have underlined that everything is hollow as a pre-
>>> condition. I agree that if f3 is pre-loaded then this must
>> not result
>>> in fetching f4 and f5 from datastore.
>>
>> Here's where I think we are having a problem. If p1 is hollow,
>> then one SQL statement is all you should need to fetch  the
>> non- relationship fields. The current fetch plan will result
>> in fetching f1, f2, and f3. If you get these fields and then
>> decide to go back and get f4 and f5, it's wrong. It's not
>> acceptable to go to the datastore to fetch fields and then
>> after you get them, to go back and get more because of a "load
>> fetch group" analysis.
>>
>> Craig
>>>
>>>
>>> Pinaki Poddar
>>> 972.834.2865
>>>
>>>
>>>> -----Original Message-----
>>>> From: Craig.Russell@Sun.COM [mailto:Craig.Russell@Sun.COM]
>>>> Sent: Wednesday, October 31, 2007 10:02 PM
>>>> To: dev@openjpa.apache.org
>>>> Subject: Re: [jira] Commented: (OPENJPA-370) LoadFetchGroup
>>>> annotation was not recognized during the fetch1
>>>>
>>>> Hi Pinaki,
>>>>
>>>> On Oct 31, 2007, at 6:19 PM, Pinaki Poddar wrote:
>>>>
>>>>> The implementation is ensuring that the user instance is
>> loaded with
>>>>> requisite fields that can be specified by
>>>>> A) fetch configuration
>>>>> B) fields that are directly accessed
>>>>> C) fields that are included because they are member of a
>>>>> load-fetch-group of another field which is being fetched/accessed.
>>>>
>>>> Not exactly. Accessed, yes. Fetched, no. This distinction
>> is critical
>>>> to understanding how load fetch groups are supposed to work.
>>>>>
>>>>> For concreteness, let us consider the following
>>>>>
>>>>> @Entity
>>>>> public class PObject {
>>>>> 	@Id
>>>>> 	private long id;	
>>>>> 	private Integer f2;
>>>>> 	@LoadFetchGroup("f4f5")
>>>>> 	private Integer f3;
>>>>> 	private Integer f4;
>>>>> 	private Integer f5;
>>>>>
>>>>> Where FetchGroup "f4f5" consists of (you guessed it right:) {"f4",
>>>>> "f5"}.
>>>>>
>>>>> Consider f3 being accessed either by direct load such as
>>>>> Pobject.getF3()
>>>>> or activating a FetchGroup {"f3"}.
>>>>
>>>> Activating a fetch group doesn't do anything. Going to the back end
>>>> to fetch instances causes the fetch plan to be analyzed and fetch
>>>> groups to actually make something happen.
>>>>
>>>> So this isn't exactly a good example, because it matters whether p1
>>>> (the instance of PObject) is already loaded into memory or not.
>>>>
>>>> So if you em.find(PObject.class, 1) then the current fetch plan is
>>>> analyzed and if f3 is not in it, then f3 isn't fetched. Full stop.
>>>>
>>>>> The resultant PObject instance with f1,f2 unloaded and f3,f4,f5
>>>>> loaded.
>>>>> However, this will result in two separate SQL being issued.
>>>>> SELECT t0.id, t0.f3 FROM pobject t0
>>>>> SELECT t0.f4, t0.f5 FROM pobject t0 WHERE t0.id = ?
>>>>
>>>> This is wrong. Only one SQL should be issued, to fetch a specific
>>>> instance.
>>>>>
>>>>> The first SQL is resulted because f3 is included in current fecth
>>>>> configuration while f4 and f5 are not.
>>>>
>>>> If this is the case, then you should stop here. The load
>> fetch group
>>>> for f3 must be ignored.
>>>>
>>>>> The second SQL is resulted of StateManagerImpl.load() that
>>>> detects f3
>>>>> has a fetch group "f4f5". It is part of the current fecth
>>>> session and
>>>>> not a result of post-fetch analysis.
>>>>
>>>> Here's where the trouble is. If f3 is not part of the fetch plan,
>>>> then it should not be fetched.
>>>>
>>>>> However the way it is done is by
>>>>> adding "f4f5" to the active fetch configuration temporarily
>>>> and going
>>>>> through similar cycle that caused the first SQL.
>>>>
>>>> This should only occur if p1 is loaded, f3 is not loaded, and
>>>> f3 is accessed.
>>>>>
>>>>> The concern is I am observing via few test cases this mechanics
>>>>> working to fulfill the user contract. However, Teresa/Kevin
>>>>> reported/observed that LoadFetchGroup is not working. Further
>>>>> investigation on this failure is required before introducing
>>>>> alternative solution.
>>>>
>>>> So we need to look at the test cases to validate them
>> before we "fix"
>>>> anything.
>>>>
>>>> Craig
>>>>>
>>>>>
>>>>> Regards --
>>>>>
>>>>>
>>>>> Pinaki Poddar
>>>>> 972.834.2865
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Craig Russell (JIRA) [mailto:jira@apache.org]
>>>>>> Sent: Wednesday, October 31, 2007 6:52 PM
>>>>>> To: dev@openjpa.apache.org
>>>>>> Subject: [jira] Commented: (OPENJPA-370) LoadFetchGroup
>> annotation
>>>>>> was not recognized during the fetch1
>>>>>>
>>>>>>
>>>>>>    [
>>>>>> https://issues.apache.org/jira/browse/OPENJPA-370?page=com.atla
>>>>>> ssian.jira.plugin.system.issuetabpanels:comment-tabpanel#action
>>>>>> _12539221 ]
>>>>>>
>>>>>> Craig Russell commented on OPENJPA-370:
>>>>>> ---------------------------------------
>>>>>>
>>>>>>> Before the instance is returned to the caller, the fields
>>>>>> are checked again if they cause any other field be loaded because
>>>>>> LoadFecthGroup. At that point, if field f has a
>>>> LoadFetchGroup L and
>>>>>> L is not part of the active fetch configuration then
>> *temporarily*
>>>>>> add L to the active FetchConfiguration and go for another 'fetch'
>>>>>> (i.e. from the datastore).
>>>>>>
>>>>>> This sounds wrong. The effect of the load fetch group
>>>> should be part
>>>>>> of the fetch strategy, and no post-fetch analysis should be
>>>> done. The
>>>>>> only time the load fetch group is used is if a field f is
>> accessed
>>>>>> and it's not already fetched.
>>>>>>
>>>>>> The intent of the load fetch group is to augment the fetch
>>>> plan under
>>>>>> which the persistent instance was fetched. It's designed
>> to provide
>>>>>> an intelligent fetch strategy for the lower-usage cases
>> where some
>>>>>> use needs field f1 (not in any fetch group in the current
>>>> fetch plan)
>>>>>> and when using field f1 you want to also fetch fields f2,
>>>> f3, and f4,
>>>>>> that are also not part of the current fetch plan.
>>>>>>
>>>>>>
>>>>>>> LoadFetchGroup annotation was not recognized during the fetch1
>>>>>>> --------------------------------------------------------------
>>>>>>>
>>>>>>>                 Key: OPENJPA-370
>>>>>>>                 URL:
>>>>>> https://issues.apache.org/jira/browse/OPENJPA-370
>>>>>>>             Project: OpenJPA
>>>>>>>          Issue Type: Bug
>>>>>>>          Components: kernel
>>>>>>>    Affects Versions: 1.0.1, 1.1.0
>>>>>>>            Reporter: Teresa Kan
>>>>>>>            Assignee: Teresa Kan
>>>>>>>             Fix For: 1.0.1, 1.1.0
>>>>>>>
>>>>>>>         Attachments: OPENJPA_370_2.patch, TestFetchGroup.zip
>>>>>>>
>>>>>>>
>>>>>>> Employee class has a LoadFetchGroup annotation defined on
>>>>>> the Rating field, when getRating was called, the address
>> should be
>>>>>> returned also. However, openjpa did not handle the LoadFetchGroup
>>>>>> correctly, therefore, address was not eargly fetched.
>>>>>>> public class FGEmployee{
>>>>>>>     @Id
>>>>>>>     private int id;
>>>>>>>
>>>>>>>     @OneToOne(fetch=FetchType.LAZY)
>>>>>>>     private FGAddress address;
>>>>>>>
>>>>>>>     @Basic(fetch=FetchType.LAZY)
>>>>>>>     @LoadFetchGroup("AddressFetchGroup")
>>>>>>>     private String rating;
>>>>>>>
>>>>>>>     @ManyToOne(fetch=FetchType.LAZY)
>>>>>>>     private FGManager manager;
>>>>>>> ..
>>>>>>> }
>>>>>>
>>>>>> --
>>>>>> This message is automatically generated by JIRA.
>>>>>> -
>>>>>> You can reply to this email to add a comment to the issue online.
>>>>>>
>>>>>>
>>>>>
>>>>> Notice:  This email message, together with any attachments, may
>>>>> contain information  of  BEA Systems,  Inc.,  its
>> subsidiaries and
>>>>> affiliated entities,  that may be confidential,  proprietary,
>>>>> copyrighted  and/or legally privileged, and is intended
>>>> solely for the
>>>>> use of the individual or entity named in this message. If
>>>> you are not
>>>>> the intended recipient, and have received this message in error,
>>>>> please immediately return this by email and then delete it.
>>>>
>>>> Craig Russell
>>>> Architect, Sun Java Enterprise System http://java.sun.com/products/
>>>> jdo
>>>> 408 276-5638 mailto:Craig.Russell@sun.com P.S. A good JDO? O, Gasp!
>>>>
>>>>
>>>
>>> Notice:  This email message, together with any attachments, may
>>> contain information  of  BEA Systems,  Inc.,  its subsidiaries
>>> and  affiliated entities,  that may be confidential,  proprietary,
>>> copyrighted  and/or legally privileged, and is intended
>> solely for the
>>> use of the individual or entity named in this message. If
>> you are not
>>> the intended recipient, and have received this message in error,
>>> please immediately return this by email and then delete it.
>>
>> Craig Russell
>> Architect, Sun Java Enterprise System http://java.sun.com/products/ 
>> jdo
>> 408 276-5638 mailto:Craig.Russell@sun.com P.S. A good JDO? O, Gasp!
>>
>>
>
> Notice:  This email message, together with any attachments, may  
> contain information  of  BEA Systems,  Inc.,  its subsidiaries   
> and  affiliated entities,  that may be confidential,  proprietary,   
> copyrighted  and/or legally privileged, and is intended solely for  
> the use of the individual or entity named in this message. If you  
> are not the intended recipient, and have received this message in  
> error, please immediately return this by email and then delete it.

Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!


Mime
View raw message