jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jukka Zitting (JIRA)" <j...@apache.org>
Subject [jira] Updated: (JCR-2835) Poor performance of ISDESCENDANTNODE on SQL 2 queries
Date Wed, 08 Dec 2010 19:17:02 GMT

     [ https://issues.apache.org/jira/browse/JCR-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jukka Zitting updated JCR-2835:
-------------------------------

      Component/s: query
                   jackrabbit-core
      Description: 
Using the latest source code, I have noticed very bad performance on SQL-2 queries that use
the ISDESCENDANTNODE constraint on a large sub-tree. For example, the query : 

select * from [jnt:news] as news where ISDESCENDANTNODE(news,'/root/site') order by news.[date]
desc 

executes in 600ms 

select * from [jnt:news] as news order by news.[date] desc

executes in 4ms

>From looking at the problem in the Yourkit profiler, it seems that the culprit is the
constraint building, that uses recursive Lucene searches to build the list of descendant node
IDs : 

    private Query getDescendantNodeQuery(
            DescendantNode dn, JackrabbitIndexSearcher searcher)
            throws RepositoryException, IOException {
        BooleanQuery query = new BooleanQuery();

        try {
            LinkedList<NodeId> ids = new LinkedList<NodeId>();
            NodeImpl ancestor = (NodeImpl) session.getNode(dn.getAncestorPath());
            ids.add(ancestor.getNodeId());
            while (!ids.isEmpty()) {
                String id = ids.removeFirst().toString();
                Query q = new JackrabbitTermQuery(new Term(FieldNames.PARENT, id));
                QueryHits hits = searcher.evaluate(q);
                ScoreNode sn = hits.nextScoreNode();
                if (sn != null) {
                    query.add(q, SHOULD);
                    do {
                        ids.add(sn.getNodeId());
                        sn = hits.nextScoreNode();
                    } while (sn != null);
                }
            }
        } catch (PathNotFoundException e) {
            query.add(new JackrabbitTermQuery(new Term(
                    FieldNames.UUID, "invalid-node-id")), // never matches
                    SHOULD);
        }

        return query;
    }

In the above example this generates over 2800 Lucene queries, which is the culprit. I wonder
if it wouldn't be faster to retrieve the IDs by using the JCR to retrieve the list of child
IDs ?

This was probably also missed because I didn't seem to find any performance tests on this
constraint.

  was:

Using the latest source code, I have noticed very bad performance on SQL-2 queries that use
the ISDESCENDANTNODE constraint on a large sub-tree. For example, the query : 

select * from [jnt:news] as news where ISDESCENDANTNODE(news,'/root/site') order by news.[date]
desc 

executes in 600ms 

select * from [jnt:news] as news order by news.[date] desc

executes in 4ms

>From looking at the problem in the Yourkit profiler, it seems that the culprit is the
constraint building, that uses recursive Lucene searches to build the list of descendant node
IDs : 

    private Query getDescendantNodeQuery(
            DescendantNode dn, JackrabbitIndexSearcher searcher)
            throws RepositoryException, IOException {
        BooleanQuery query = new BooleanQuery();

        try {
            LinkedList<NodeId> ids = new LinkedList<NodeId>();
            NodeImpl ancestor = (NodeImpl) session.getNode(dn.getAncestorPath());
            ids.add(ancestor.getNodeId());
            while (!ids.isEmpty()) {
                String id = ids.removeFirst().toString();
                Query q = new JackrabbitTermQuery(new Term(FieldNames.PARENT, id));
                QueryHits hits = searcher.evaluate(q);
                ScoreNode sn = hits.nextScoreNode();
                if (sn != null) {
                    query.add(q, SHOULD);
                    do {
                        ids.add(sn.getNodeId());
                        sn = hits.nextScoreNode();
                    } while (sn != null);
                }
            }
        } catch (PathNotFoundException e) {
            query.add(new JackrabbitTermQuery(new Term(
                    FieldNames.UUID, "invalid-node-id")), // never matches
                    SHOULD);
        }

        return query;
    }

In the above example this generates over 2800 Lucene queries, which is the culprit. I wonder
if it wouldn't be faster to retrieve the IDs by using the JCR to retrieve the list of child
IDs ?

This was probably also missed because I didn't seem to find any performance tests on this
constraint.

    Fix Version/s:     (was: 2.2.0)
       Issue Type: Improvement  (was: Bug)

This probably won't make it in the 2.2 timeframe, so targetting just 2.3 for now.

When fixing this, we should start by adding a test case to the peformance test suite (possibly
with an option to compare SQL2 against XPath). That way we'll have solid numbers to back any
improvement ideas.

In general subtree queries are troublesome even for XPath, but there's still a lot we can
do to bring SQL2 at least close to that level. One idea I had (but didn't yet have time to
implement) was to collect the parent UUIDs of the entire subtree not by traversing each node
separately, but by per-level queries like 1) parent-id=<uuid-of-ancestor>, 2) parent-id=<uuid-of-child1>
parent-id=<uuid-of-child2> ..., 3) same for grandchildren, etc. That way we'd only need
to execute a handful of Lucene queries per one subtree constraint.

> Poor performance of ISDESCENDANTNODE on SQL 2 queries
> -----------------------------------------------------
>
>                 Key: JCR-2835
>                 URL: https://issues.apache.org/jira/browse/JCR-2835
>             Project: Jackrabbit Content Repository
>          Issue Type: Improvement
>          Components: jackrabbit-core, query
>    Affects Versions: 2.2.0
>            Reporter: Serge Huber
>             Fix For: 2.3.0
>
>         Attachments: JCR-2835_Poor_performance_on_ISDESCENDANTNODE_constraint_v1.patch
>
>
> Using the latest source code, I have noticed very bad performance on SQL-2 queries that
use the ISDESCENDANTNODE constraint on a large sub-tree. For example, the query : 
> select * from [jnt:news] as news where ISDESCENDANTNODE(news,'/root/site') order by news.[date]
desc 
> executes in 600ms 
> select * from [jnt:news] as news order by news.[date] desc
> executes in 4ms
> From looking at the problem in the Yourkit profiler, it seems that the culprit is the
constraint building, that uses recursive Lucene searches to build the list of descendant node
IDs : 
>     private Query getDescendantNodeQuery(
>             DescendantNode dn, JackrabbitIndexSearcher searcher)
>             throws RepositoryException, IOException {
>         BooleanQuery query = new BooleanQuery();
>         try {
>             LinkedList<NodeId> ids = new LinkedList<NodeId>();
>             NodeImpl ancestor = (NodeImpl) session.getNode(dn.getAncestorPath());
>             ids.add(ancestor.getNodeId());
>             while (!ids.isEmpty()) {
>                 String id = ids.removeFirst().toString();
>                 Query q = new JackrabbitTermQuery(new Term(FieldNames.PARENT, id));
>                 QueryHits hits = searcher.evaluate(q);
>                 ScoreNode sn = hits.nextScoreNode();
>                 if (sn != null) {
>                     query.add(q, SHOULD);
>                     do {
>                         ids.add(sn.getNodeId());
>                         sn = hits.nextScoreNode();
>                     } while (sn != null);
>                 }
>             }
>         } catch (PathNotFoundException e) {
>             query.add(new JackrabbitTermQuery(new Term(
>                     FieldNames.UUID, "invalid-node-id")), // never matches
>                     SHOULD);
>         }
>         return query;
>     }
> In the above example this generates over 2800 Lucene queries, which is the culprit. I
wonder if it wouldn't be faster to retrieve the IDs by using the JCR to retrieve the list
of child IDs ?
> This was probably also missed because I didn't seem to find any performance tests on
this constraint.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message