openjpa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marc Prud'hommeaux <mprud...@apache.org>
Subject Re: Proposal: Optimizing empty collection fetch. Meta Column in ContainerFieldMappling
Date Fri, 06 Oct 2006 07:24:41 GMT
Alex-

> - Any chance to get this feature implemented in 4.1? Then we might  
> just
> migrate to 4.1

4.1 is so close that I seriously doubt that it would make it in.

> assuming it is a solid release (I hope so very much)

It certainly will be a solid release!

> - If not in 4.1 is there any way to get some advise from you guys  
> (like
> in old Solarmetric times :-) on nuances of 3.4 implementation so I can
> do it myself?

Nothing springs to mind that would make it complex ... the interfaces  
are more of less the same, so you might be able to get 90% of the way  
there by just trying to compile it against Kodo 3.4 and see what  
happens...


On Oct 5, 2006, at 6:14 PM, Roytman, Alex wrote:

> Mark,
>
> Actually, I developed my prototype for 3.4 not for 4.0 - we are not
> using 4.0 - due to some problems with it.
>
> My implementation is not very good - it is just a quick prototype/ 
> proof
> of concept built for Kodo 3.4. It lacks transparency (for example one
> must declare a java field to hold the attribute) and might not be
> completely correct.
>
> So my questions are:
>
> - Any chance to get this feature implemented in 4.1? Then we might  
> just
> migrate to 4.1 assuming it is a solid release (I hope so very much)
> - If not in 4.1 is there any way to get some advise from you guys  
> (like
> in old Solarmetric times :-) on nuances of 3.4 implementation so I can
> do it myself?
>
> I will file a JIRA request
>
> Alex
>
>
> -----Original Message-----
> From: Marc Prud'hommeaux [mailto:mprudhomapache@gmail.com] On  
> Behalf Of
> Marc Prud'hommeaux
> Sent: Thursday, October 05, 2006 9:01 PM
> To: open-jpa-dev@incubator.apache.org
> Subject: Re: Proposal: Optimizing empty collection fetch. Meta  
> Column in
> ContainerFieldMappling
>
> Alex-
>
> You are right about jdbc-container-meta ... I had forgotten about
> that attribute.
>
>> I can certainly file a JIRA request if you'd like me to.
>
> That's the standard mechanism for entering enhancement requests for
> OpenJPA itself, so it is useful to have as a reference point.
>
>> I wonder if there is any chance to get some advice on implementing
>> it in
>> Kodo JDO 3.4?
>
> Do you mean in addition to your implementation of it for Kodo 4.0?
> The custom field mapping mechanism is similar to 4.0, so any work
> that you have done for this implementation in 4.0 will probably be
> similar to how you would do it in 3.4.
>
>> Do you think it is reasonable to expect this feature in 4.1 release?
>
> It probably would not make it in in time.
>
>> Is 4.1 coming soon?
>
> Yes, very soon.
>
>> Will it have solid JDO2 implementation?
>
> Yes it will.
>
>
>
> On Oct 5, 2006, at 5:55 PM, Roytman, Alex wrote:
>
>> Hello Mark,
>>
>> There are two mechanisms in Kodo - jdbc-null-indicator for 1-1
>> embedded
>> and jdbc-container-meta for collections/maps. In general I was  
>> talking
>> about jdbc-container-meta one. The docs state:
>>
>> " 6.2.3.7. jdbc-container-meta
>> Container metadata is used to record non-essential information about
>> collection and map fields. If this extension is set to true,
>> collections
>> and maps will be able to distinguish between the empty state and the
>> null state. If this extension is set to false or is unset, then it
>> will
>> not be possible for Kodo to differentiate between these two  
>> states. In
>> this situation, all collections and maps in persistent objects loaded
>> from the database will be non-null"
>>
>> Almost exactly what I want and I was kind of surprised by the limited
>> implementation of the feature.
>>
>> I can certainly file a JIRA request if you'd like me to.
>>
>> I wonder if there is any chance to get some advice on implementing
>> it in
>> Kodo JDO 3.4?
>>
>> Do you think it is reasonable to expect this feature in 4.1
>> release? Is
>> 4.1 coming soon? Will it have solid JDO2 implementation?
>>
>> Thank you
>>
>> alex
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: Marc Prud'hommeaux [mailto:mprudhomapache@gmail.com] On
>> Behalf Of
>> Marc Prud'hommeaux
>> Sent: Thursday, October 05, 2006 8:19 PM
>> To: open-jpa-dev@incubator.apache.org
>> Subject: Re: Proposal: Optimizing empty collection fetch. Meta
>> Column in
>> ContainerFieldMappling
>>
>> Alex-
>>
>> That does sound like a good feature to add. Note that I think the
>> "null-indicator" attribute is only available for embedded mappings,
>> not for container mappings (although I could be wrong about this).
>>
>> I'd recommend opening a JIRA issue as a reference for the enhancement
>> request, and we can build on that.
>>
>>
>>
>> On Oct 5, 2006, at 3:52 PM, Roytman, Alex wrote:
>>
>>> Hello Abe,
>>>
>>> I would like to present a valid use case and a very useful
>>> performance
>>> enhancement.
>>>
>>> The idea is that, if we know that a collection field is empty
>>> there is
>>> no need to fetch it.
>>>
>>> It can provide a truly dramatic performance improvement when in a
>>> large
>>> set of instance only some of them have non-empty collection field.
>>> Consider a very common case - composite (tree like) data structures.
>>> Unlike true composite pattern typical tree structure does not have a
>>> special leaf class that is any node of a tree can potentially have
>>> sub-nodes. When traversing such a tree as many as 70% of fetches of
>>> child nodes will yield empty collection because obviously leaf
>>> level is
>>> the larges in a tree structure :-)
>>>
>>> I wrote a prototype custom 1-N mapping which allow to store "empty"
>>> flag
>>> (whether the collection is empty) on commit and will store empty
>>> collection into StateManager on collection field load if the flag
>>> is set
>>> to true (empty) instead of going to database to fetch it.
>>>
>>> The results were dramatic - when traversing 800-node tree number of
>>> "fetch-sub-nodes" SQL statements was cut from 800 to 130.
>>>
>>> Non-Tree cases when objects have sparsely populated collection
>>> field can
>>> be even more dramatic.
>>>
>>> If concurrency of the collection field is controlled on owned class
>>> level (default) I think there is no dander of this flag being out of
>>> synch with actual collection content without entering concurrent
>>> modification state.
>>>
>>> I have not had chance to think through transaction commit
>>> implications
>>> if any.
>>>
>>> There is a very nice facility in ContainerFieldMappling for
>>> indicating
>>> null container fields. I wonder why it so much hard wired to empty/
>>> null
>>> and does not allow non-empty/empty/null differentiation and
>>> optimization.
>>> Any reason it is so restrictive? Any plans to make it a bit more
>>> flexible or directly implementing the behavior I outlined above?
>>>
>>> I would greatly appreciate if you could comment on this and may be
>>> suggest the best approach implementing this. Or may be it is already
>>> implemented and I am missing it :-)
>>>
>>> Best Regards
>>>
>>> Alex Roytman
>>> Peace Technology, Inc
>>>
>>>
>>
>


Mime
View raw message