openjpa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Roytman (JIRA)" <j...@apache.org>
Subject [jira] Created: (OPENJPA-64) Optimizing empty collection fetch. Better use of jdbc-container-meta in ContainerFieldMappling
Date Fri, 06 Oct 2006 01:53:19 GMT
Optimizing empty collection fetch. Better use of jdbc-container-meta in ContainerFieldMappling
----------------------------------------------------------------------------------------------

                 Key: OPENJPA-64
                 URL: http://issues.apache.org/jira/browse/OPENJPA-64
             Project: OpenJPA
          Issue Type: Improvement
          Components: jdbc
            Reporter: Alex Roytman


The idea of this enhancement is that, if we know that a collection field is empty there is
no need to fetch it. Therefore an optimization can be done if we store a flag in collection
owner table indicating whether it is null//empty/not empty and awoid fetching null and empty
collections.

It can provide a truly dramatic performance improvement when in a large set of instance only
some of them have non-empty collection field. Consider a very common case - composite (tree
like) data structures. Unlike true composite pattern typical tree structure does not have
a special leaf class that is any node of a tree can potentially have sub-nodes. When traversing
such a tree as many as 70% of fetches of child nodes will yield empty collection because obviously
leaf level is the larges in a tree structure :-)  

I wrote a prototype custom 1-N mapping for Kodo 3.4 which allow to store "empty" flag (whether
the collection is empty) on commit and will store empty collection into StateManager on collection
field load if the flag is set to true (empty) instead of going to database to fetch it.

The results were dramatic - when traversing 800-node tree number of "fetch-sub-nodes" SQL
statements was cut from 800 to 130.

Non-Tree cases when objects have sparsely populated collection field can be even more dramatic.

If concurrency of the collection field is controlled on owned class level (default) I think
there is no dander of this flag being out of synch with actual collection content without
entering concurrent modification state.

I have not had chance to think through transaction commit implications if any.

There is a very nice facility in ContainerFieldMappling for indicating null container fields.
I wonder why it so much hard wired to empty/null and does not allow non-empty/empty/null differentiation
and optimization.

The documentation states:

" 6.2.3.7. jdbc-container-meta
Container metadata is used to record non-essential information about collection and map fields.
If this extension is set to true, collections and maps will be able to distinguish between
the empty state and the null state. If this extension is set to false or is unset, then it
will not be possible for Kodo to differentiate between these two states. In this situation,
all collections and maps in persistent objects loaded from the database will be non-null"

Actual implementation of jdbc-container-meta is very restrictive (at least in 3.4 branch)
and does not seem to be easily adaptable for not-fetching empty collections
It would be great if it were enhanced and no-fetch optimization for null/empty collections
could be performed

Best Regards

Alex Roytman
Peace Technology, Inc




-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message