Return-Path: Delivered-To: apmail-cayenne-user-archive@www.apache.org Received: (qmail 42766 invoked from network); 20 Nov 2007 08:49:17 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 20 Nov 2007 08:49:17 -0000 Received: (qmail 7946 invoked by uid 500); 20 Nov 2007 08:49:04 -0000 Delivered-To: apmail-cayenne-user-archive@cayenne.apache.org Received: (qmail 7933 invoked by uid 500); 20 Nov 2007 08:49:04 -0000 Mailing-List: contact user-help@cayenne.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cayenne.apache.org Delivered-To: mailing list user@cayenne.apache.org Received: (qmail 7924 invoked by uid 99); 20 Nov 2007 08:49:04 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Nov 2007 00:49:04 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [17.148.16.80] (HELO smtpoutm.mac.com) (17.148.16.80) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Nov 2007 08:49:06 +0000 Received: from mac.com (asmtp001-s [10.150.69.64]) by smtpoutm.mac.com (Xserve/smtpout017/MantshX 4.0) with ESMTP id lAK8mi7l015030 for ; Tue, 20 Nov 2007 00:48:44 -0800 (PST) Received: from [192.168.1.33] (45-37.5-85.cust.bluewin.ch [85.5.37.45]) (authenticated bits=0) by mac.com (Xserve/asmtp001/MantshX 4.0) with ESMTP id lAK8m71j014650 for ; Tue, 20 Nov 2007 00:48:39 -0800 (PST) Message-Id: <466927B4-C36B-406D-8A12-9BD6E51FB3C6@mac.com> From: "Alexander Lamb (dev)" To: user@cayenne.apache.org In-Reply-To: <80DBE39A-1342-47CC-B3A8-BC591E1C6FD0@mac.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Apple Message framework v915) Subject: Re: Batch faulting with Cayenne 3: small add-on to correct problem Date: Tue, 20 Nov 2007 09:48:07 +0100 References:

<4900F6C7-FCE3-42C0-9FD6-7C77429753B9@mac.com> <6897D13C-FC52-4E1D-9857-3132971B9160@objectstyle.org> <80DBE39A-1342-47CC-B3A8-BC591E1C6FD0@mac.com> X-Mailer: Apple Mail (2.915) X-Virus-Checked: Checked by ClamAV on apache.org Hello, In my code example, there is actually a bug. Indeed, if I do: >>> List roles =3D getObjectContext().performQuery(query); >>> ToManyList tml =3D new ToManyList(this,"roles"); >>> tml.addAll(roles); >>> writePropertyDirectly("roles",tml); >> I will end up with twice the number of roles... Indeed, doing "addAll" =20= will first fire the to-many which was just created before adding the =20 objects... The only way to avoid that initial fetch is to do: tml.setObjectList(roles) then continue with the writePropertyDirectly. Alex Le 15 nov. 07 =E0 10:45, Alexander Lamb (dev) a =E9crit : > > >> The code below should work. Another way is to use multi-step =20 >> prefetching on a root of to-many relationship. > > Well, that's a good news so we shall probably implement my piece of =20= > code for most relationships. However, I am afraid I didn't =20 > understand your second sentence:-( >> >> >>> Now, it would still be cool if we could have batch faulting for =20 >>> the odd places where we didn't set up the prefetching. >> >> While I used batch faulting in webobjects days and found it quite =20 >> useful, I could never understand how to make it work predictably =20 >> (i.e. fault the objects that I care about). I wouldn't object =20 >> though to somebody (or even myself) implementing it at the =20 >> framework level if whoever that is could explain me the algorithm =20 >> used to select which objects to fault. IIRC EOF builds internal =20 >> "fault chains". Wonder how much overhead this would incur in Cayenne. > I don't think there is a particular order in which the faults are =20 > fired. Actually, it is not a problem since gradually in a few =20 > queries, all outstanding faults will be fired. My guess is that upon =20= > firing the first fault of a to-one registered to batch fault, you =20 > simply (ok, not that simple:-) look at all entities of same class in =20= > the DataContext and take the first X to be fired at random. X being =20= > the size of the batch. Of course, it some how means if there is a to-=20= > one which needs to be batch faulted it has to be flagged somewhere =20 > so you quickly know which objects to take into account. >> >> BTW relationship prefetching policies can be specified per JPA spec =20= >> (and hence will be implemented in Cayenne). However my =20 >> understanding is that JPA specifies a different kind of prefetch - =20= >> which attributes/relationships to resolve eagerly when an object is =20= >> fetched. > > I am afraid I don't know anything about JPA, but it probably means =20 > in the future some kind of interface in the modeler to be able to =20 > specify those prefetches? > > Thanks! > > Alex > >> >> >> Andrus >> >> >> >> On Nov 14, 2007, at 9:33 AM, Alexander Lamb (dev) wrote: >> >>> Well, yes it is possible up to a point. >>> >>> Usually it is through the "to-many" relationship I get my objects. =20= >>> Some other times it might be through a custom query meaning I have =20= >>> to do it each time. >>> >>> However, as I said in the second email I sent about prefetching, =20 >>> the solution is maybe the following: >>> >>> @SuppressWarnings("unchecked") >>> public List getRoles () { >>> =20 >>> if=20 >>> (org=20 >>> .apache=20 >>> .cayenne=20 >>> .Fault.class.isInstance(this.readPropertyDirectly("roles"))) { >>> Expression exp =3D ExpressionFactory.matchExp("person", this); >>> SelectQuery query =3D new SelectQuery(Role.class, exp); >>> query.addPrefetch("profile"); >>> query.addPrefetch("person"); >>> List roles =3D getObjectContext().performQuery(query); >>> ToManyList tml =3D new ToManyList(this,"roles"); >>> tml.addAll(roles); >>> writePropertyDirectly("roles",tml); >>> } >>> return super.getRoles(); >>> } >>> >>> The advantage of this is that it does the prefetch but also sets =20 >>> correctly the "to-many" relationship, meaning it will not refetch =20= >>> everything if I do an addToRoles or removeFromRoles. >>> >>> If I want to refault the relationship, I do: >>> >>> =20 >>> if=20 >>> (org=20 >>> .apache=20 >>> .cayenne=20 >>> .access=20 >>> .ToManyList.class.isInstance(this.readPropertyDirectly("roles"))) { >>> = ((org.apache.cayenne.access.ToManyList)getRoles()).invalidate(); >>> } >>> >>> Is this the correct way of doing it? >>> >>> If so, could there be a way to add this in a generic way to the =20 >>> model? >>> >>> Now, it would still be cool if we could have batch faulting for =20 >>> the odd places where we didn't set up the prefetching. >>> >>> Alex >>> >>> Le 14 nov. 07 =E0 14:45, Andrus Adamchik a =E9crit : >>> >>>> Can you use prefetching instead? You got a list of users vis some =20= >>>> sort of query - just add prefetch to that query. >>>> >>>> Andrus >>>> >>>> >>>> On Nov 14, 2007, at 8:11 AM, Alexander Lamb (dev) wrote: >>>> >>>>> Hello list, >>>>> >>>>> One thing is killing performance of our application: it is the =20 >>>>> resolving of individual to-one faults in lists. >>>>> >>>>> For example, we can have 200 roles each refering to a person. >>>>> >>>>> When we loop through the roles, for each role where we do a =20 >>>>> role.getPerson() there will be a return trip to the database. >>>>> >>>>> In the EOF days, there was a possibility to define a batch =20 >>>>> faulting strategy for the entity. In that we would say for =20 >>>>> example "batch fault 20 for person" and the first time a to-one =20= >>>>> fault to person from role would be found, it would look in the =20 >>>>> data context for up to 19 more to build a single SQL statement =20 >>>>> and fetch in one go the person objects and resolve up to 20 =20 >>>>> faults. >>>>> >>>>> Is this feature available somewhere in Cayenne 3m2 or planned in =20= >>>>> the near future? >>>>> >>>>> If not, is there some kind of callback or hook wich would allow =20= >>>>> us to do the same thing? >>>>> >>>>> Thanks, >>>>> >>>>> Alex >>>>> >>>> >>> >> > -- Alexander Lamb alamb@mac.com