Return-Path: Delivered-To: apmail-openjpa-users-archive@minotaur.apache.org Received: (qmail 29433 invoked from network); 14 Apr 2009 03:52:24 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 14 Apr 2009 03:52:24 -0000 Received: (qmail 70710 invoked by uid 500); 14 Apr 2009 03:52:24 -0000 Delivered-To: apmail-openjpa-users-archive@openjpa.apache.org Received: (qmail 70632 invoked by uid 500); 14 Apr 2009 03:52:23 -0000 Mailing-List: contact users-help@openjpa.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@openjpa.apache.org Delivered-To: mailing list users@openjpa.apache.org Received: (qmail 70622 invoked by uid 99); 14 Apr 2009 03:52:23 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Apr 2009 03:52:23 +0000 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [69.49.111.68] (HELO mail168c2.megamailservers.com) (69.49.111.68) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Apr 2009 03:52:13 +0000 X-Authenticated-User: paul.copelandz.com Received: from [127.0.0.1] (ellen.pc.ashlandfiber.net [66.241.90.23] (may be forged)) (authenticated bits=0) by mail168c2.megamailservers.com (8.13.6/8.13.1) with ESMTP id n3E3plY8021643 for ; Mon, 13 Apr 2009 23:51:50 -0400 Message-ID: <49E40854.6040905@jotobjects.com> Date: Mon, 13 Apr 2009 20:51:48 -0700 From: Paul Copeland Organization: JOT Object Technologies User-Agent: Thunderbird 2.0.0.5 (Windows/20070716) MIME-Version: 1.0 To: users@openjpa.apache.org Subject: Re: Does OpenJPA replace Collections? References: <49DC3FE8.2020103@jotobjects.com> <1239185774592-2604361.post@n2.nabble.com> <49DCD415.2060500@jotobjects.com> <49DCDD24.407@jotobjects.com> <340F1664-0382-43C6-A55F-833BC547D484@SUN.com> <49DE21A1.1070702@jotobjects.com> <49DE24ED.9010804@jotobjects.com> <1762AF7D-39D0-475D-9C29-7F4F64164B1F@SUN.com> <49DE4B01.5070601@jotobjects.com> <49E36297.1070901@jotobjects.com> <47637B6C-F61F-44BE-869F-8547C4814C8B@SUN.com> <49E3D59F.1020301@jotobjects.com> <31AE21DA-85D1-447F-9DB0-06FA176AAFF8@SUN.com> In-Reply-To: <31AE21DA-85D1-447F-9DB0-06FA176AAFF8@SUN.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Hi Craig - Do you mean a JIRA about the case where "OpenJPA will decide to replace the collection"? I agree it is not a bug, just a potential issue for the unwary. Would it be useful to create a small example and post it here? - Paul On 4/13/2009 7:41 PM, Craig L Russell wrote: > Hi Paul, > > On Apr 13, 2009, at 5:15 PM, Paul Copeland wrote: > >> Craig - Thanks for the responses. This confirms that for a new Entity >> a collection field may be null unless the application initializes it. >> >> When you say "flushed" does that include calling >> EntityManager.flush() before the transaction is committed? The spec >> says the field can be null until it is "fetched". My expectation is >> that the field may remain null even after calling EntityManager.flush(). >> >> The surprising thing is that when you add elements to such an >> application initialized empty collection, in some situations OpenJPA >> will decide to replace the collection. At that point if you are >> holding a reference to the value returned by getMyPcList() that >> collection will then be stale, possibly leading to inconsistent >> results for the caller. > > This is worth a JIRA if only to clarify the code and the behavior that > the code exposes. > > Craig >> >> >> - Paul >> >> (other comments below) >> >> On 4/13/2009 4:18 PM, Craig L Russell wrote: >>> Hi Paul, >>> >>> On Apr 13, 2009, at 9:04 AM, Paul Copeland wrote: >>> >>>> Are there any responses from the OpenJPA experts on my two >>>> assertions below? If the assertions seem wrong I will put together >>>> examples that demonstrate the behavior. If the assertions are >>>> correct that is not necessary. >>>> >>>> From JPA Spec Section 2.1.7 - "If there are no associated entities >>>> for a multi-valued relationship of an entity fetched from the >>>> database, >>>> the persistence provider is responsible for returning an empty >>>> collection as the value of the relationship." >>>> >>>> Note the words "fetched from the database". My reading of this is >>>> that if the Entity is new and has not been flushed to the database >>>> (even though persist() has been called) the value could be null >>>> rather than an empty collection. So the behavior of OpenJPA >>>> returning null (assertion #1) would be consistent with the spec. >>> >>> That's how I read it as well. Until the new entity is flushed, >>> there's no reason to have the entity manager provider messing with >>> the values. >>>> >>>> >>>> - Paul >>>> >>>> On 4/9/2009 12:22 PM, Paul Copeland wrote: >>>>> Thanks for the assistance Craig - >>>>> >>>>> Here are two assertions that I have observed in my testing with >>>>> OpenJPA 1.2.1 - >>>>> >>>>> (1) A Field Access persistent collection has a null value when the >>>>> field is accessed if the collection is empty. This is the state of >>>>> the field in the transaction after the entity is first persisted >>>>> before the transaction is committed (these are the conditions that >>>>> occur in my process). Corollary - the null field is NOT >>>>> automatically changed to an empty Collection when first accessed. >>>>> A method returning the collection field will return null. >>> >>> This is discussed above. The entity is not "fetched" but rather >>> newly persisted. >>>>> >>>>> >>>>> (2) The value of a null collection field (state as in #1 above) >>>>> that has been assigned to an initialized non-null value may be >>>>> automatically replaced before the transaction is committed at >>>>> which point references to the assigned value will be stale and no >>>>> longer updated (for instance when entity's are added to the >>>>> collection). >>> >>> This is discussed above. Until flush, any user changes to the >>> collection should be reflected in the database. >>> >>> But one other thing to consider. It's the application's >>> responsibility to manage both sides of a relationship to be >>> consistent at commit. So if you're looking to update only the other >>> side of a relationship you're in trouble unless you use some OpenJPA >>> special techniques. >>> >> Good point about updating both sides of the relation. In this case I >> am using the OpenJPA API to detect if the other side has not been >> loaded yet and only updating the other side when necessary. This is >> to avoid loading a potentially very large collection that is not >> going to be used during the life of that EntityManager. If and when >> the other side is loaded OpenJPA will include the new elements then. >> >> This does not change the question about null or empty collections >> however. >> >>> Craig >>>>> >>>>> >>>>> If the experts believe either of these assertions are incorrect >>>>> then I definitely want to investigate further. >>>>> >>>>> - Paul >>>>> >>>>> (further comments below) >>>>> >>>>> >>>>> On 4/9/2009 11:13 AM, Craig L Russell wrote: >>>>>> Hi Paul, >>>>>> >>>>>> On Apr 9, 2009, at 9:40 AM, Paul Copeland wrote: >>>>>> >>>>>>> Couple of clarifications - >>>>>>> >>>>>>> A lazily loaded FIELD ACCESS collection is a null value when >>>>>>> initially accessed if the Collection is EMPTY (I said "null" >>>>>>> incorrectly below). >>>>>> >>>>>> My comment below was intended to compare your "if null then >>>>>> initialize" paradigm with my "initialize to an empty collection >>>>>> during construction". So if the first time you access the >>>>>> collection it is null your code sets the value to an empty >>>>>> collection. My recommended code would never encounter a null >>>>>> collection. >>>>> >>>>> Your way works (as do other ways). :-) >>>>> >>>>>>> >>>>>>> The test I have shows this behavior for a newly persisted Entity >>>>>>> during the same transaction where em.persist(entity) is called. >>>>>>> This is with a LAZY loaded collection. >>>>>> >>>>>> During persist, the provider should not replace fields. Replacing >>>>>> fields behavior should happen at commit (flush) time. So if you >>>>>> never explicitly initialize a field, it should have its Java >>>>>> default value until flush. >>>>> >>>>> This is NOT what I am seeing. In fact the replacement happens >>>>> during the transaction under certain conditions where the proxy is >>>>> apparently created during the transaction some time after the call >>>>> to em.persist(entity) and before commit. >>>>> >>>>>> >>>>>> If you're talking about wrapping the persistent collection with >>>>>> an unmodifiable collection then you're talking about adding more >>>>>> objects. I thought you were trying to avoid any object construction? >>>>> >>>>> I would construct the unmodifiable collection (if the idiom >>>>> worked) only if and when the value is accessed and has already >>>>> been loaded. Other things being equal, I don't want to construct >>>>> tens of thousands of Collections in a tight loop that are never >>>>> used. Given database latencies it is a small point in overall >>>>> performance. As I said, there are good arguments either way and >>>>> your recommendation is one reasonable approach, but apparently not >>>>> a JPA requirement. >>>>> >>>>>> >>>>>> In some applications there is a difference between an empty >>>>>> collection and a null collection. There are properties that allow >>>>>> that behavior to be implemented as well, although that's >>>>>> non-standard and a bit more complicated. >>>>>> >>>>>> It might be easier to look at a test case because I think we're >>>>>> talking past each other. >>>>>> >>>>>> Craig >>>>>>> >>>>>>> >>>>>>> On 4/9/2009 9:26 AM, Paul Copeland wrote: >>>>>>>> Hi Craig - >>>>>>>> >>>>>>>> My experience is not what you are describing. A lazily loaded >>>>>>>> FIELD ACCESS collection is a null value when initially accessed >>>>>>>> if the Collection is null (possibly a PROPERTY ACCESS >>>>>>>> collection behaves differently as mentioned by Pinaki , I >>>>>>>> haven't tested that). >>>>>>>> >>>>>>>> To repeat what is below - >>>>>>>> >>>>>>>> getMyPcList() >>>>>>>> >>>>>>>> returns null if the Collection is empty unless you initialize >>>>>>>> the value with "new ArrayList()". This is what my testing >>>>>>>> shows with 1.2.1 - I wish it weren't this way since that might >>>>>>>> it make it possible to use the Collections.unmodifiedList() >>>>>>>> idiom (as it is that idiom has unreliable behavior). If the >>>>>>>> experts are pretty sure that I am wrong about this then I >>>>>>>> definitely want to investigate it further. I'd like to hear more. >>>>>>>> >>>>>>>> I don't think you have given a reason to require initializing >>>>>>>> the Collection at construction time or at first access -- there >>>>>>>> are reasonable aesthetic and performance arguments either way. >>>>>>>> >>>>>>>> - Paul >>>>>>>> >>>>>>>> >>>>>>>> On 4/9/2009 7:01 AM, Craig L Russell wrote: >>>>>>>>> Hi Paul, >>>>>>>>> >>>>>>>>> I like to think of entities as POJOs first, so I can test them >>>>>>>>> without requiring them to be persistent. So if you want code >>>>>>>>> to be able to add elements to collections, the collections >>>>>>>>> must not be null. >>>>>>>>> >>>>>>>>> If you construct the field as null and then "lazily" >>>>>>>>> instantiate an empty collection, then anyway you end up with >>>>>>>>> an empty collection the first time you access the field. And >>>>>>>>> constructing an empty collection should not be even a blip on >>>>>>>>> your performance metric. >>>>>>>>> >>>>>>>>> Considering everything, I still recommend that you instantiate >>>>>>>>> an empty collection when you construct an entity. >>>>>>>>> >>>>>>>>> Craig >>>>>>>>> >>>>>>>>> On Apr 8, 2009, at 10:21 AM, Paul Copeland wrote: >>>>>>>>> >>>>>>>>>> Pinaki - >>>>>>>>>> >>>>>>>>>> I tried your suggestion of not initializing the value of >>>>>>>>>> myPcList and I get a null pointer exception when adding to an >>>>>>>>>> empty list. >>>>>>>>>> >>>>>>>>>> I noticed your example was for Property access and Russell >>>>>>>>>> (and I) were talking about Field access. Do you agree that >>>>>>>>>> it is necessary to initialize an empty list when using Field >>>>>>>>>> access? >>>>>>>>>> >>>>>>>>>> On Craig's advice to always construct a new ArrayList(), why >>>>>>>>>> is that necessary instead of just constructing it in the >>>>>>>>>> getter when it tests to null? Otherwise you are constructing >>>>>>>>>> an ArrayList that is unnecessary when the List is NOT empty >>>>>>>>>> (usually) and also unnecessary in the case of LAZY loading if >>>>>>>>>> the List is never accessed (perhaps also a frequent case). >>>>>>>>>> In some applications you might create lots of these objects >>>>>>>>>> and normal optimization is to avoid calling constructors >>>>>>>>>> unnecessarily. Just want to be clear about whether it is >>>>>>>>>> necessary. >>>>>>>>>> >>>>>>>>>> - Paul >>>>>>>>>> >>>>>>>>>> On 4/8/2009 9:43 AM, Paul Copeland wrote: >>>>>>>>>>> Thanks Pinaki - >>>>>>>>>>> >>>>>>>>>>> I think you are saying that at some point the proxy object >>>>>>>>>>> does replace the local List. Is that right? >>>>>>>>>>> >>>>>>>>>>> I have seen that model - if (myPcList == null) myPcList = >>>>>>>>>>> new ArrayList() - in various examples (not sure where now). >>>>>>>>>>> Thanks for clearing that up. But then Craig Russell >>>>>>>>>>> contradicts you in his reply (below) where he recommends >>>>>>>>>>> always initializing the Collection in the constructor (which >>>>>>>>>>> seems like a performance anti-pattern of wasted constructor >>>>>>>>>>> calls since usually it will be replaced by the proxy). Are >>>>>>>>>>> you and Craig saying opposite things here? >>>>>>>>>>> >>>>>>>>>>> In my testing when the List is empty - (myPcList == null) - >>>>>>>>>>> does indeed evaluate to true. >>>>>>>>>>> >>>>>>>>>>> getMyPcList().add(new MyPcObject()) >>>>>>>>>>> >>>>>>>>>>> Therefore I thought the above would cause a null pointer >>>>>>>>>>> exception when the List is empty. You say that won't happen >>>>>>>>>>> so I'll give it a try! >>>>>>>>>>> >>>>>>>>>>> - Paul >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 4/8/2009 3:16 AM, Pinaki Poddar wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> According to JPA spec: >>>>>>>>>>>> "If there are no associated entities for a multi-valued >>>>>>>>>>>> relationship of an entity fetched from the database, >>>>>>>>>>>> the persistence provider is responsible for returning an >>>>>>>>>>>> empty collection as the value of the relationship." >>>>>>>>>>>> >>>>>>>>>>>> That is what OpenJPA does. So the application do not need >>>>>>>>>>>> to return an empty list for a null (initialized) list. >>>>>>>>>>>> >>>>>>>>>>>> OpenJPA proxies all returned collections. So application >>>>>>>>>>>> code can simply do the following >>>>>>>>>>>> >>>>>>>>>>>> // In the domain class >>>>>>>>>>>> private List myPcList = null; // never >>>>>>>>>>>> explictly initialized >>>>>>>>>>>> >>>>>>>>>>>> @OneToMany (mappedBy="ownerSide", fetch=FetchType.LAZY, >>>>>>>>>>>> cascade=CascadeType.PERSIST) >>>>>>>>>>>> public List getMyPcList() { >>>>>>>>>>>> return myPcList; // return as it is >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> // In the application >>>>>>>>>>>> List list = owner.getMyPcList(); >>>>>>>>>>>> assertNotNull(list); >>>>>>>>>>>> assertTrue(java.util.List.class.isInstance(list)); >>>>>>>>>>>> assertNotSame(java.util.ArrayList.class, list.getClass()); >>>>>>>>>>>> list.add(new MyPcObject()); >>>>>>>>>>>> owner.setMyPcList(list); >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Apr 7, 2009, at 11:10 PM, Paul Copeland wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Can OpenJPA replace a Collection when it is loaded? >>>>>>>>>>>>> >>>>>>>>>>>>> With the code below when the list is initially empty you >>>>>>>>>>>>> need to create a List (ArrayList) so you can add elements >>>>>>>>>>>>> to it. When I persisted new objects on the ManyToOne side >>>>>>>>>>>>> and added them to the List that worked. But the first >>>>>>>>>>>>> time the List was loaded it seemed to replace my >>>>>>>>>>>>> ArrayList with the newly loaded data and made an older >>>>>>>>>>>>> reference to the ArrayList stale (no longer updated when >>>>>>>>>>>>> more elements were added to myPcList). This was all in >>>>>>>>>>>>> one transaction. >>>>>>>>>>>>> >>>>>>>>>>>>> So now I wonder if the initial null List is a special case >>>>>>>>>>>>> or if OpenJPA might replace the Collection anytime it >>>>>>>>>>>>> decides to load it again. Anyone know the answer? >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> If the list is persistent and the class is enhanced, the >>>>>>>>>>>> collection will always reflect what's in the database. >>>>>>>>>>>> >>>>>>>>>>>>> If I don't create an initial ArrayList how can I add >>>>>>>>>>>>> elements when the List is empty? >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I'd recommend always having a non-empty list. Initialize it >>>>>>>>>>>> in the constructor to an empty list and don't check it >>>>>>>>>>>> after that. >>>>>>>>>>>> >>>>>>>>>>>> Here's what it would look like: >>>>>>>>>>>> >>>>>>>>>>>>> @OneToMany (mappedBy="ownerSide", fetch=FetchType.LAZY, >>>>>>>>>>>>> cascade=CascadeType.PERSIST) >>>>>>>>>>>>> private List myPcList = new >>>>>>>>>>>>> ArrayList(); >>>>>>>>>>>>> >>>>>>>>>>>>> List getMyPcList() >>>>>>>>>>>>> { >>>>>>>>>>>>> return myPcList; >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> Craig >>>>>>>>>>>> Craig L Russell >>>>>>>>>>>> Architect, Sun Java Enterprise System http://db.apache.org/jdo >>>>>>>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com >>>>>>>>>>>> P.S. A good JDO? O, Gasp! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> ----- >>>>>>>>>>>> Pinaki Poddar >>>>>>>>>>>> http://ppoddar.blogspot.com/ >>>>>>>>>>>> >>>>>>>>>>>> http://www.linkedin.com/in/pinakipoddar >>>>>>>>>>>> OpenJPA PMC Member/Committer >>>>>>>>>>>> JPA Expert Group Member >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> Craig L Russell >>>>>>>>> Architect, Sun Java Enterprise System http://db.apache.org/jdo >>>>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com >>>>>>>>> P.S. A good JDO? O, Gasp! >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> Craig L Russell >>>>>> Architect, Sun Java Enterprise System http://db.apache.org/jdo >>>>>> 408 276-5638 mailto:Craig.Russell@sun.com >>>>>> P.S. A good JDO? O, Gasp! >>>>>> >>>>> >>>>> >>>>> >>>> >>> >>> Craig L Russell >>> Architect, Sun Java Enterprise System http://db.apache.org/jdo >>> 408 276-5638 mailto:Craig.Russell@sun.com >>> P.S. A good JDO? O, Gasp! >>> >> > > Craig L Russell > Architect, Sun Java Enterprise System http://db.apache.org/jdo > 408 276-5638 mailto:Craig.Russell@sun.com > P.S. A good JDO? O, Gasp! >