openjpa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Linskey" <plins...@bea.com>
Subject RE: Proposal: Optimizing empty collection fetch. Meta Column in ContainerFieldMappling
Date Fri, 06 Oct 2006 14:02:10 GMT
Bear in mind that even if it doesn't ship in Kodo 4.1, Kodo 4.1 will
ship against a known OpenJPA build, so you could just download OpenJPA
at that revision number, patch the file, rebuild, drop in the patched
jar.

Of course, this becomes a management nightmare moving forward. But
submitting the patch to this list should help resolve that, by getting
it into the main OpenJPA codebase.

-Patrick

-- 
Patrick Linskey
BEA Systems, Inc. 

_______________________________________________________________________
Notice:  This email message, together with any attachments, may contain
information  of  BEA Systems,  Inc.,  its subsidiaries  and  affiliated
entities,  that may be confidential,  proprietary,  copyrighted  and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it. 

> -----Original Message-----
> From: Marc Prud'hommeaux [mailto:mprudhomapache@gmail.com] On 
> Behalf Of Marc Prud'hommeaux
> Sent: Friday, October 06, 2006 12:25 AM
> To: open-jpa-dev@incubator.apache.org
> Subject: Re: Proposal: Optimizing empty collection fetch. 
> Meta Column in ContainerFieldMappling
> 
> Alex-
> 
> > - Any chance to get this feature implemented in 4.1? Then we might  
> > just
> > migrate to 4.1
> 
> 4.1 is so close that I seriously doubt that it would make it in.
> 
> > assuming it is a solid release (I hope so very much)
> 
> It certainly will be a solid release!
> 
> > - If not in 4.1 is there any way to get some advise from you guys  
> > (like
> > in old Solarmetric times :-) on nuances of 3.4 
> implementation so I can
> > do it myself?
> 
> Nothing springs to mind that would make it complex ... the 
> interfaces  
> are more of less the same, so you might be able to get 90% of 
> the way  
> there by just trying to compile it against Kodo 3.4 and see what  
> happens...
> 
> 
> On Oct 5, 2006, at 6:14 PM, Roytman, Alex wrote:
> 
> > Mark,
> >
> > Actually, I developed my prototype for 3.4 not for 4.0 - we are not
> > using 4.0 - due to some problems with it.
> >
> > My implementation is not very good - it is just a quick prototype/ 
> > proof
> > of concept built for Kodo 3.4. It lacks transparency (for 
> example one
> > must declare a java field to hold the attribute) and might not be
> > completely correct.
> >
> > So my questions are:
> >
> > - Any chance to get this feature implemented in 4.1? Then we might  
> > just
> > migrate to 4.1 assuming it is a solid release (I hope so very much)
> > - If not in 4.1 is there any way to get some advise from you guys  
> > (like
> > in old Solarmetric times :-) on nuances of 3.4 
> implementation so I can
> > do it myself?
> >
> > I will file a JIRA request
> >
> > Alex
> >
> >
> > -----Original Message-----
> > From: Marc Prud'hommeaux [mailto:mprudhomapache@gmail.com] On  
> > Behalf Of
> > Marc Prud'hommeaux
> > Sent: Thursday, October 05, 2006 9:01 PM
> > To: open-jpa-dev@incubator.apache.org
> > Subject: Re: Proposal: Optimizing empty collection fetch. Meta  
> > Column in
> > ContainerFieldMappling
> >
> > Alex-
> >
> > You are right about jdbc-container-meta ... I had forgotten about
> > that attribute.
> >
> >> I can certainly file a JIRA request if you'd like me to.
> >
> > That's the standard mechanism for entering enhancement requests for
> > OpenJPA itself, so it is useful to have as a reference point.
> >
> >> I wonder if there is any chance to get some advice on implementing
> >> it in
> >> Kodo JDO 3.4?
> >
> > Do you mean in addition to your implementation of it for Kodo 4.0?
> > The custom field mapping mechanism is similar to 4.0, so any work
> > that you have done for this implementation in 4.0 will probably be
> > similar to how you would do it in 3.4.
> >
> >> Do you think it is reasonable to expect this feature in 
> 4.1 release?
> >
> > It probably would not make it in in time.
> >
> >> Is 4.1 coming soon?
> >
> > Yes, very soon.
> >
> >> Will it have solid JDO2 implementation?
> >
> > Yes it will.
> >
> >
> >
> > On Oct 5, 2006, at 5:55 PM, Roytman, Alex wrote:
> >
> >> Hello Mark,
> >>
> >> There are two mechanisms in Kodo - jdbc-null-indicator for 1-1
> >> embedded
> >> and jdbc-container-meta for collections/maps. In general I was  
> >> talking
> >> about jdbc-container-meta one. The docs state:
> >>
> >> " 6.2.3.7. jdbc-container-meta
> >> Container metadata is used to record non-essential 
> information about
> >> collection and map fields. If this extension is set to true,
> >> collections
> >> and maps will be able to distinguish between the empty 
> state and the
> >> null state. If this extension is set to false or is unset, then it
> >> will
> >> not be possible for Kodo to differentiate between these two  
> >> states. In
> >> this situation, all collections and maps in persistent 
> objects loaded
> >> from the database will be non-null"
> >>
> >> Almost exactly what I want and I was kind of surprised by 
> the limited
> >> implementation of the feature.
> >>
> >> I can certainly file a JIRA request if you'd like me to.
> >>
> >> I wonder if there is any chance to get some advice on implementing
> >> it in
> >> Kodo JDO 3.4?
> >>
> >> Do you think it is reasonable to expect this feature in 4.1
> >> release? Is
> >> 4.1 coming soon? Will it have solid JDO2 implementation?
> >>
> >> Thank you
> >>
> >> alex
> >>
> >>
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: Marc Prud'hommeaux [mailto:mprudhomapache@gmail.com] On
> >> Behalf Of
> >> Marc Prud'hommeaux
> >> Sent: Thursday, October 05, 2006 8:19 PM
> >> To: open-jpa-dev@incubator.apache.org
> >> Subject: Re: Proposal: Optimizing empty collection fetch. Meta
> >> Column in
> >> ContainerFieldMappling
> >>
> >> Alex-
> >>
> >> That does sound like a good feature to add. Note that I think the
> >> "null-indicator" attribute is only available for embedded mappings,
> >> not for container mappings (although I could be wrong about this).
> >>
> >> I'd recommend opening a JIRA issue as a reference for the 
> enhancement
> >> request, and we can build on that.
> >>
> >>
> >>
> >> On Oct 5, 2006, at 3:52 PM, Roytman, Alex wrote:
> >>
> >>> Hello Abe,
> >>>
> >>> I would like to present a valid use case and a very useful
> >>> performance
> >>> enhancement.
> >>>
> >>> The idea is that, if we know that a collection field is empty
> >>> there is
> >>> no need to fetch it.
> >>>
> >>> It can provide a truly dramatic performance improvement when in a
> >>> large
> >>> set of instance only some of them have non-empty collection field.
> >>> Consider a very common case - composite (tree like) data 
> structures.
> >>> Unlike true composite pattern typical tree structure does 
> not have a
> >>> special leaf class that is any node of a tree can potentially have
> >>> sub-nodes. When traversing such a tree as many as 70% of 
> fetches of
> >>> child nodes will yield empty collection because obviously leaf
> >>> level is
> >>> the larges in a tree structure :-)
> >>>
> >>> I wrote a prototype custom 1-N mapping which allow to 
> store "empty"
> >>> flag
> >>> (whether the collection is empty) on commit and will store empty
> >>> collection into StateManager on collection field load if the flag
> >>> is set
> >>> to true (empty) instead of going to database to fetch it.
> >>>
> >>> The results were dramatic - when traversing 800-node tree 
> number of
> >>> "fetch-sub-nodes" SQL statements was cut from 800 to 130.
> >>>
> >>> Non-Tree cases when objects have sparsely populated collection
> >>> field can
> >>> be even more dramatic.
> >>>
> >>> If concurrency of the collection field is controlled on 
> owned class
> >>> level (default) I think there is no dander of this flag 
> being out of
> >>> synch with actual collection content without entering concurrent
> >>> modification state.
> >>>
> >>> I have not had chance to think through transaction commit
> >>> implications
> >>> if any.
> >>>
> >>> There is a very nice facility in ContainerFieldMappling for
> >>> indicating
> >>> null container fields. I wonder why it so much hard wired 
> to empty/
> >>> null
> >>> and does not allow non-empty/empty/null differentiation and
> >>> optimization.
> >>> Any reason it is so restrictive? Any plans to make it a bit more
> >>> flexible or directly implementing the behavior I outlined above?
> >>>
> >>> I would greatly appreciate if you could comment on this and may be
> >>> suggest the best approach implementing this. Or may be it 
> is already
> >>> implemented and I am missing it :-)
> >>>
> >>> Best Regards
> >>>
> >>> Alex Roytman
> >>> Peace Technology, Inc
> >>>
> >>>
> >>
> >
> 
> 

Mime
View raw message