Return-Path: Delivered-To: apmail-jackrabbit-commits-archive@www.apache.org Received: (qmail 57318 invoked from network); 28 Nov 2006 17:45:39 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 28 Nov 2006 17:45:39 -0000 Received: (qmail 36342 invoked by uid 500); 28 Nov 2006 17:45:48 -0000 Delivered-To: apmail-jackrabbit-commits-archive@jackrabbit.apache.org Received: (qmail 36318 invoked by uid 500); 28 Nov 2006 17:45:48 -0000 Mailing-List: contact commits-help@jackrabbit.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@jackrabbit.apache.org Delivered-To: mailing list commits@jackrabbit.apache.org Received: (qmail 36309 invoked by uid 99); 28 Nov 2006 17:45:47 -0000 Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Nov 2006 09:45:47 -0800 X-ASF-Spam-Status: No, hits=-9.4 required=10.0 tests=ALL_TRUSTED,NO_REAL_NAME X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO eris.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Nov 2006 09:45:37 -0800 Received: by eris.apache.org (Postfix, from userid 65534) id 485941A9846; Tue, 28 Nov 2006 09:44:59 -0800 (PST) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r480138 - in /jackrabbit/trunk/jackrabbit/src: main/config/ main/java/org/apache/jackrabbit/core/query/lucene/ test/java/org/apache/jackrabbit/core/query/ Date: Tue, 28 Nov 2006 17:44:57 -0000 To: commits@jackrabbit.apache.org From: mreutegg@apache.org X-Mailer: svnmailer-1.1.0 Message-Id: <20061128174459.485941A9846@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: mreutegg Date: Tue Nov 28 09:44:54 2006 New Revision: 480138 URL: http://svn.apache.org/viewvc?view=rev&rev=480138 Log: JCR-651: Improve performance for queries with large result sets Added: jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/ScoreNode.java (with props) jackrabbit/trunk/jackrabbit/src/test/java/org/apache/jackrabbit/core/query/QueryResultTest.java (with props) Modified: jackrabbit/trunk/jackrabbit/src/main/config/repository.xml jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/DocOrderNodeIteratorImpl.java jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/NodeIteratorImpl.java jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/QueryImpl.java jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/QueryResultImpl.java jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/SearchIndex.java jackrabbit/trunk/jackrabbit/src/test/java/org/apache/jackrabbit/core/query/TestAll.java Modified: jackrabbit/trunk/jackrabbit/src/main/config/repository.xml URL: http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit/src/main/config/repository.xml?view=diff&rev=480138&r1=480137&r2=480138 ============================================================================== --- jackrabbit/trunk/jackrabbit/src/main/config/repository.xml (original) +++ jackrabbit/trunk/jackrabbit/src/main/config/repository.xml Tue Nov 28 09:44:54 2006 @@ -267,6 +267,8 @@ - respectDocumentOrder: If true and the query does not contain an 'order by' clause, result nodes will be in document order. For better performance when queries return a lot of nodes set to 'false'. + - resultFetchSize: The number of results the query handler should + initially fetch when a query is executed. Note: all parameters (except path) in this SearchIndex config are default values and can be omitted. @@ -287,6 +289,7 @@ + Modified: jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/DocOrderNodeIteratorImpl.java URL: http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/DocOrderNodeIteratorImpl.java?view=diff&rev=480138&r1=480137&r2=480138 ============================================================================== --- jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/DocOrderNodeIteratorImpl.java (original) +++ jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/DocOrderNodeIteratorImpl.java Tue Nov 28 09:44:54 2006 @@ -17,7 +17,6 @@ package org.apache.jackrabbit.core.query.lucene; import org.apache.jackrabbit.core.ItemManager; -import org.apache.jackrabbit.core.NodeId; import org.apache.jackrabbit.core.NodeImpl; import org.apache.jackrabbit.name.Path; import org.slf4j.Logger; @@ -42,26 +41,22 @@ /** A node iterator with ordered nodes */ private NodeIteratorImpl orderedNodes; - /** The DIs of the nodes in the result set */ - protected NodeId[] ids; - - /** The score values for the nodes in the result set */ - protected Float[] scores; + /** Unordered list of {@link ScoreNode}s. */ + private final List scoreNodes; /** ItemManager to turn UUIDs into Node instances */ protected final ItemManager itemMgr; /** - * Creates a DocOrderNodeIteratorImpl that orders the nodes - * with uuids in document order. - * @param itemMgr the item manager of the session executing the query. - * @param ids the uuids of the nodes. - * @param scores the score values of the nodes. + * Creates a DocOrderNodeIteratorImpl that orders the nodes in + * scoreNodes in document order. + * + * @param itemMgr the item manager of the session executing the query. + * @param scoreNodes the ids of the matching nodes with their score value. */ - DocOrderNodeIteratorImpl(final ItemManager itemMgr, NodeId[] ids, Float[] scores) { + DocOrderNodeIteratorImpl(final ItemManager itemMgr, List scoreNodes) { this.itemMgr = itemMgr; - this.ids = ids; - this.scores = scores; + this.scoreNodes = scoreNodes; } /** @@ -117,7 +112,7 @@ if (orderedNodes != null) { return orderedNodes.getSize(); } else { - return ids.length; + return scoreNodes.size(); } } @@ -155,10 +150,7 @@ return; } long time = System.currentTimeMillis(); - ScoreNode[] nodes = new ScoreNode[ids.length]; - for (int i = 0; i < ids.length; i++) { - nodes[i] = new ScoreNode(ids[i], scores[i]); - } + ScoreNode[] nodes = (ScoreNode[]) scoreNodes.toArray(new ScoreNode[scoreNodes.size()]); final List invalidIDs = new ArrayList(2); @@ -167,7 +159,7 @@ // previous sort run was not successful -> remove failed uuids List tmp = new ArrayList(); for (int i = 0; i < nodes.length; i++) { - if (!invalidIDs.contains(nodes[i].id)) { + if (!invalidIDs.contains(nodes[i].getNodeId())) { tmp.add(nodes[i]); } } @@ -184,20 +176,20 @@ try { NodeImpl node1; try { - node1 = (NodeImpl) itemMgr.getItem(n1.id); + node1 = (NodeImpl) itemMgr.getItem(n1.getNodeId()); } catch (RepositoryException e) { - log.warn("Node " + n1.id + " does not exist anymore: " + e); + log.warn("Node " + n1.getNodeId() + " does not exist anymore: " + e); // node does not exist anymore - invalidIDs.add(n1.id); + invalidIDs.add(n1.getNodeId()); throw new SortFailedException(); } NodeImpl node2; try { - node2 = (NodeImpl) itemMgr.getItem(n2.id); + node2 = (NodeImpl) itemMgr.getItem(n2.getNodeId()); } catch (RepositoryException e) { - log.warn("Node " + n2.id + " does not exist anymore: " + e); + log.warn("Node " + n2.getNodeId() + " does not exist anymore: " + e); // node does not exist anymore - invalidIDs.add(n2.id); + invalidIDs.add(n2.getNodeId()); throw new SortFailedException(); } Path.PathElement[] path1 = node1.getPrimaryPath().getElements(); @@ -246,8 +238,8 @@ } // if we get here something went wrong // remove both uuids from array - invalidIDs.add(n1.id); - invalidIDs.add(n2.id); + invalidIDs.add(n1.getNodeId()); + invalidIDs.add(n2.getNodeId()); // terminate sorting throw new SortFailedException(); } @@ -258,35 +250,10 @@ } while (invalidIDs.size() > 0); - // resize uuids and scores array if we had to remove some uuids - if (ids.length != nodes.length) { - ids = new NodeId[nodes.length]; - scores = new Float[nodes.length]; - } - - for (int i = 0; i < nodes.length; i++) { - ids[i] = nodes[i].id; - scores[i] = nodes[i].score; - } if (log.isDebugEnabled()) { - log.debug("" + ids.length + " node(s) ordered in " + (System.currentTimeMillis() - time) + " ms"); - } - orderedNodes = new NodeIteratorImpl(itemMgr, ids, scores); - } - - /** - * Simple helper class that associates a score with each node uuid. - */ - private static final class ScoreNode { - - final NodeId id; - - final Float score; - - ScoreNode(NodeId id, Float score) { - this.id = id; - this.score = score; + log.debug("" + nodes.length + " node(s) ordered in " + (System.currentTimeMillis() - time) + " ms"); } + orderedNodes = new NodeIteratorImpl(itemMgr, nodes); } /** Modified: jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/NodeIteratorImpl.java URL: http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/NodeIteratorImpl.java?view=diff&rev=480138&r1=480137&r2=480138 ============================================================================== --- jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/NodeIteratorImpl.java (original) +++ jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/NodeIteratorImpl.java Tue Nov 28 09:44:54 2006 @@ -17,7 +17,6 @@ package org.apache.jackrabbit.core.query.lucene; import org.apache.jackrabbit.core.ItemManager; -import org.apache.jackrabbit.core.NodeId; import org.apache.jackrabbit.core.NodeImpl; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -35,11 +34,8 @@ /** Logger instance for this class */ private static final Logger log = LoggerFactory.getLogger(NodeIteratorImpl.class); - /** The node ids of the nodes in the result set */ - protected final NodeId[] ids; - - /** The score values for the nodes in the result set */ - protected final Float[] scores; + /** The node ids of the nodes in the result set with their score value */ + protected final ScoreNode[] scoreNodes; /** ItemManager to turn UUIDs into Node instances */ protected final ItemManager itemMgr; @@ -57,15 +53,12 @@ * Creates a new NodeIteratorImpl instance. * @param itemMgr the ItemManager to turn UUIDs into * Node instances. - * @param ids the IDs of the result nodes. - * @param scores the corresponding score values for each result node. + * @param scoreNodes the node ids of the matching nodes. */ NodeIteratorImpl(ItemManager itemMgr, - NodeId[] ids, - Float[] scores) { + ScoreNode[] scoreNodes) { this.itemMgr = itemMgr; - this.ids = ids; - this.scores = scores; + this.scoreNodes = scoreNodes; fetchNext(); } @@ -114,7 +107,7 @@ if (skipNum < 0) { throw new IllegalArgumentException("skipNum must not be negative"); } - if ((pos + skipNum) > ids.length) { + if ((pos + skipNum) > scoreNodes.length) { throw new NoSuchElementException(); } if (skipNum == 0) { @@ -138,7 +131,7 @@ * @return the number of node in this iterator. */ public long getSize() { - return ids.length - invalid; + return scoreNodes.length - invalid; } /** @@ -176,7 +169,7 @@ if (!hasNext()) { throw new NoSuchElementException(); } - return scores[pos].floatValue(); + return scoreNodes[pos].getScore(); } /** @@ -188,12 +181,12 @@ protected void fetchNext() { // reset next = null; - while (next == null && (pos + 1) < ids.length) { + while (next == null && (pos + 1) < scoreNodes.length) { try { - next = (NodeImpl) itemMgr.getItem(ids[pos + 1]); + next = (NodeImpl) itemMgr.getItem(scoreNodes[pos + 1].getNodeId()); } catch (RepositoryException e) { log.warn("Exception retrieving Node with UUID: " - + ids[pos + 1] + ": " + e.toString()); + + scoreNodes[pos + 1].getNodeId() + ": " + e.toString()); // try next invalid++; pos++; Modified: jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/QueryImpl.java URL: http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/QueryImpl.java?view=diff&rev=480138&r1=480137&r2=480138 ============================================================================== --- jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/QueryImpl.java (original) +++ jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/QueryImpl.java Tue Nov 28 09:44:54 2006 @@ -17,7 +17,6 @@ package org.apache.jackrabbit.core.query.lucene; import org.apache.jackrabbit.core.ItemManager; -import org.apache.jackrabbit.core.NodeId; import org.apache.jackrabbit.core.SessionImpl; import org.apache.jackrabbit.core.nodetype.NodeTypeImpl; import org.apache.jackrabbit.core.nodetype.PropertyDefinitionImpl; @@ -30,7 +29,6 @@ import org.apache.jackrabbit.core.query.PropertyTypeRegistry; import org.apache.jackrabbit.core.query.QueryParser; import org.apache.jackrabbit.core.query.QueryRootNode; -import org.apache.jackrabbit.core.security.AccessManager; import org.apache.jackrabbit.name.QName; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -40,10 +38,8 @@ import javax.jcr.nodetype.PropertyDefinition; import javax.jcr.query.InvalidQueryException; import javax.jcr.query.QueryResult; -import java.io.IOException; import java.util.ArrayList; import java.util.Arrays; -import java.util.Collections; import java.util.List; /** @@ -164,47 +160,9 @@ ascSpecs[i] = orderSpecs[i].isAscending(); } - - List ids; - List scores; - AccessManager accessMgr = session.getAccessManager(); - - // execute it - QueryHits result = null; - try { - result = index.executeQuery(this, query, orderProperties, ascSpecs); - ids = new ArrayList(result.length()); - scores = new ArrayList(result.length()); - - for (int i = 0; i < result.length(); i++) { - NodeId id = NodeId.valueOf(result.doc(i).get(FieldNames.UUID)); - // check access - if (accessMgr.isGranted(id, AccessManager.READ)) { - ids.add(id); - scores.add(new Float(result.score(i))); - } - } - } catch (IOException e) { - log.error("Exception while executing query: ", e); - ids = Collections.EMPTY_LIST; - scores = Collections.EMPTY_LIST; - } finally { - if (result != null) { - try { - result.close(); - } catch (IOException e) { - log.warn("Unable to close query result: " + e); - } - } - } - - // return QueryResult - return new QueryResultImpl(itemMgr, - (NodeId[]) ids.toArray(new NodeId[ids.size()]), - (Float[]) scores.toArray(new Float[scores.size()]), - getSelectProperties(), - session.getNamespaceResolver(), - orderNode == null && documentOrder); + return new QueryResultImpl(index, itemMgr, session.getNamespaceResolver(), + session.getAccessManager(), this, query, getSelectProperties(), + orderProperties, ascSpecs, documentOrder); } /** Modified: jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/QueryResultImpl.java URL: http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/QueryResultImpl.java?view=diff&rev=480138&r1=480137&r2=480138 ============================================================================== --- jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/QueryResultImpl.java (original) +++ jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/QueryResultImpl.java Tue Nov 28 09:44:54 2006 @@ -18,17 +18,26 @@ import org.apache.jackrabbit.core.ItemManager; import org.apache.jackrabbit.core.NodeId; +import org.apache.jackrabbit.core.NodeImpl; +import org.apache.jackrabbit.core.security.AccessManager; import org.apache.jackrabbit.name.NamespaceResolver; import org.apache.jackrabbit.name.NoPrefixDeclaredException; import org.apache.jackrabbit.name.QName; import org.apache.jackrabbit.name.NameFormat; +import org.apache.lucene.search.Query; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import javax.jcr.NodeIterator; import javax.jcr.RepositoryException; +import javax.jcr.ItemNotFoundException; +import javax.jcr.Node; import javax.jcr.query.QueryResult; import javax.jcr.query.RowIterator; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; +import java.util.NoSuchElementException; /** * Implements the javax.jcr.query.QueryResult interface. @@ -41,29 +50,69 @@ private static final Logger log = LoggerFactory.getLogger(QueryResultImpl.class); /** + * The search index to execute the query. + */ + private final SearchIndex index; + + /** * The item manager of the session executing the query */ private final ItemManager itemMgr; /** - * The UUIDs of the result nodes + * The namespace resolver of the session executing the query */ - private final NodeId[] ids; + protected final NamespaceResolver resolver; /** - * The scores of the result nodes + * The access manager of the session that executes the query. */ - private final Float[] scores; + private final AccessManager accessMgr; + + /** + * The query instance which created this query result. + */ + protected final QueryImpl queryImpl; + + /** + * The lucene query to execute. + */ + protected final Query query; /** * The select properties */ - private final QName[] selectProps; + protected final QName[] selectProps; /** - * The namespace resolver of the session executing the query + * The names of properties to use for ordering the result set. + */ + protected final QName[] orderProps; + + /** + * The order specifier for each of the order properties. + */ + protected final boolean[] orderSpecs; + + /** + * The result nodes including their score. This list is populated on a lazy + * basis while a client iterates through the results. + */ + private final List resultNodes = new ArrayList(); + + /** + * This is the raw number of results that matched the query. This number + * also includes matches which will not be returned due to access + * restrictions. This value is set when the query is executed the first + * time. + */ + private int numResults = -1; + + /** + * The number of results that are invalid, either because a node does not + * exist anymore or because the session does not have access to the node. */ - private final NamespaceResolver resolver; + private int invalid = 0; /** * If true nodes are returned in document order. @@ -73,26 +122,43 @@ /** * Creates a new query result. * - * @param itemMgr the item manager of the session executing the query. - * @param ids the Ids of the result nodes. - * @param scores the score values of the result nodes. - * @param selectProps the select properties of the query. - * @param resolver the namespace resolver of the session executing the query. - * @param docOrder if true the result is returned in document - * order. - */ - public QueryResultImpl(ItemManager itemMgr, - NodeId[] ids, - Float[] scores, - QName[] selectProps, + * @param index the search index where the query is executed. + * @param itemMgr the item manager of the session executing the + * query. + * @param resolver the namespace resolver of the session executing the + * query. + * @param accessMgr the access manager of the session executiong the + * query. + * @param queryImpl the query instance which created this query result. + * @param query the lucene query to execute on the index. + * @param selectProps the select properties of the query. + * @param orderProps the names of the order properties. + * @param orderSpecs the order specs, one for each order property name. + * @param documentOrder if true the result is returned in + * document order. + */ + public QueryResultImpl(SearchIndex index, + ItemManager itemMgr, NamespaceResolver resolver, - boolean docOrder) { - this.ids = ids; - this.scores = scores; + AccessManager accessMgr, + QueryImpl queryImpl, + Query query, + QName[] selectProps, + QName[] orderProps, + boolean[] orderSpecs, + boolean documentOrder) throws RepositoryException { + this.index = index; this.itemMgr = itemMgr; - this.selectProps = selectProps; this.resolver = resolver; - this.docOrder = docOrder; + this.accessMgr = accessMgr; + this.queryImpl = queryImpl; + this.query = query; + this.selectProps = selectProps; + this.orderProps = orderProps; + this.orderSpecs = orderSpecs; + this.docOrder = orderProps.length == 0 && documentOrder; + // if document order is requested get all results right away + getResults(docOrder ? Integer.MAX_VALUE : index.getResultFetchSize()); } /** @@ -128,14 +194,248 @@ } /** + * Executes the query for this result and returns hits. The caller must + * close the query hits when he is done using it. + * + * @return hits for this query result. + * @throws IOException if an error occurs while executing the query. + */ + protected QueryHits executeQuery() throws IOException { + return index.executeQuery(queryImpl, query, orderProps, orderSpecs); + } + + //--------------------------------< internal >------------------------------ + + /** * Creates a node iterator over the result nodes. + * * @return a node iterator over the result nodes. */ private ScoreNodeIterator getNodeIterator() { if (docOrder) { - return new DocOrderNodeIteratorImpl(itemMgr, ids, scores); + return new DocOrderNodeIteratorImpl(itemMgr, resultNodes); } else { - return new NodeIteratorImpl(itemMgr, ids, scores); + return new LazyScoreNodeIterator(); + } + } + + /** + * Attempts to get size results and puts them into {@link + * #resultNodes}. If the size of {@link #resultNodes} is less than + * size then there are no more than resultNodes.size() + * results for this query. + * + * @param size the number of results to fetch for the query. + * @throws RepositoryException if an error occurs while executing the + * query. + */ + private void getResults(int size) throws RepositoryException { + if (log.isDebugEnabled()) { + log.debug("getResults(" + size + ")"); + } + if (resultNodes.size() >= size) { + // we already have them all + return; + } + + // execute it + QueryHits result = null; + try { + result = executeQuery(); + + // set num results with the first query execution + if (numResults == -1) { + numResults = result.length(); + } + + int start = resultNodes.size() + invalid; + int max = Math.min(result.length(), numResults); + for (int i = start; i < max && resultNodes.size() < size; i++) { + NodeId id = NodeId.valueOf(result.doc(i).get(FieldNames.UUID)); + // check access + try { + if (accessMgr.isGranted(id, AccessManager.READ)) { + resultNodes.add(new ScoreNode(id, result.score(i))); + } + } catch (ItemNotFoundException e) { + // has been deleted meanwhile + invalid++; + } + } + } catch (IOException e) { + log.error("Exception while executing query: ", e); + // todo throw? + } finally { + if (result != null) { + try { + result.close(); + } catch (IOException e) { + log.warn("Unable to close query result: " + e); + } + } + } + } + + private final class LazyScoreNodeIterator implements ScoreNodeIterator { + + private int position = -1; + + private boolean initialized = false; + + private NodeImpl next; + + /** + * {@inheritDoc} + */ + public float getScore() { + initialize(); + if (!hasNext()) { + throw new NoSuchElementException(); + } + return ((ScoreNode) resultNodes.get(position)).getScore(); + } + + /** + * {@inheritDoc} + */ + public NodeImpl nextNodeImpl() { + initialize(); + if (next == null) { + throw new NoSuchElementException(); + } + NodeImpl n = next; + fetchNext(); + return n; + } + + /** + * {@inheritDoc} + */ + public Node nextNode() { + return nextNodeImpl(); + } + + /** + * {@inheritDoc} + */ + public void skip(long skipNum) { + initialize(); + if (skipNum < 0) { + throw new IllegalArgumentException("skipNum must not be negative"); + } + if ((position + invalid + skipNum) > numResults) { + throw new NoSuchElementException(); + } + if (skipNum == 0) { + // do nothing + } else { + // attempt to get enough results + try { + getResults(position + invalid + (int) skipNum); + if (resultNodes.size() >= position + skipNum) { + // skip within already fetched results + position += skipNum - 1; + fetchNext(); + } else { + // not enough results after getResults() + throw new NoSuchElementException(); + } + } catch (RepositoryException e) { + throw new NoSuchElementException(e.getMessage()); + } + } + } + + /** + * {@inheritDoc} + *

+ * This value may shrink when the query result encounters non-existing + * nodes or the session does not have access to a node. + */ + public long getSize() { + return numResults - invalid; + } + + /** + * {@inheritDoc} + */ + public long getPosition() { + initialize(); + return position; + } + + /** + * @throws UnsupportedOperationException always. + */ + public void remove() { + throw new UnsupportedOperationException("remove"); + } + + /** + * {@inheritDoc} + */ + public boolean hasNext() { + initialize(); + return next != null; + } + + /** + * {@inheritDoc} + */ + public Object next() { + return nextNodeImpl(); + } + + /** + * Initializes this iterator but only if it is not yet initialized. + */ + private void initialize() { + if (!initialized) { + fetchNext(); + initialized = true; + } + } + + /** + * Fetches the next node to return by this iterator. If this method + * returns and {@link #next} is null then there is no next + * node. + */ + private void fetchNext() { + next = null; + int nextPos = position + 1; + while (next == null && (nextPos + invalid) < numResults) { + if (nextPos >= resultNodes.size()) { + // fetch more results + try { + int num; + if (resultNodes.size() == 0) { + num = index.getResultFetchSize(); + } else { + num = resultNodes.size() * 2; + } + getResults(num); + } catch (RepositoryException e) { + log.warn("Exception getting more results: " + e); + } + // check again + if (nextPos >= resultNodes.size()) { + // no more valid results + return; + } + } + ScoreNode sn = (ScoreNode) resultNodes.get(nextPos); + try { + next = (NodeImpl) itemMgr.getItem(sn.getNodeId()); + } catch (RepositoryException e) { + log.warn("Exception retrieving Node with UUID: " + + sn.getNodeId() + ": " + e.toString()); + // remove score node and try next + resultNodes.remove(nextPos); + invalid++; + } + } + position++; } } } Added: jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/ScoreNode.java URL: http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/ScoreNode.java?view=auto&rev=480138 ============================================================================== --- jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/ScoreNode.java (added) +++ jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/ScoreNode.java Tue Nov 28 09:44:54 2006 @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.jackrabbit.core.query.lucene; + +import org.apache.jackrabbit.core.NodeId; + +/** + * ScoreNode implements a simple container which holds a mapping + * of {@link NodeId} to a score value. + */ +final class ScoreNode { + + /** + * The id of a node. + */ + private final NodeId id; + + /** + * The score of the node. + */ + private final float score; + + /** + * Creates a new ScoreNode. + * + * @param id the node id. + * @param score the score value. + */ + ScoreNode(NodeId id, float score) { + this.id = id; + this.score = score; + } + + /** + * @return the node id for this ScoreNode. + */ + public NodeId getNodeId() { + return id; + } + + /** + * @return the score for this ScoreNode. + */ + public float getScore() { + return score; + } +} Propchange: jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/ScoreNode.java ------------------------------------------------------------------------------ svn:eol-style = native Modified: jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/SearchIndex.java URL: http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/SearchIndex.java?view=diff&rev=480138&r1=480137&r2=480138 ============================================================================== --- jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/SearchIndex.java (original) +++ jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/SearchIndex.java Tue Nov 28 09:44:54 2006 @@ -182,6 +182,13 @@ private int cacheSize = 1000; /** + * The number of documents that are pre fetched when a query is executed. + *

+ * Default value is: 50. + */ + private int resultFetchSize = 50; + + /** * Indicates if this SearchIndex is closed and cannot be used * anymore. */ @@ -782,6 +789,24 @@ delim = ","; } return names.toString(); + } + + /** + * Tells the query handler how many result should be fetched initially when + * a query is executed. + * + * @param size the number of results to fetch initially. + */ + public void setResultFetchSize(int size) { + resultFetchSize = size; + } + + /** + * @return the number of results the query handler will fetch initially when + * a query is executed. + */ + public int getResultFetchSize() { + return resultFetchSize; } //----------------------------< internal >---------------------------------- Added: jackrabbit/trunk/jackrabbit/src/test/java/org/apache/jackrabbit/core/query/QueryResultTest.java URL: http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit/src/test/java/org/apache/jackrabbit/core/query/QueryResultTest.java?view=auto&rev=480138 ============================================================================== --- jackrabbit/trunk/jackrabbit/src/test/java/org/apache/jackrabbit/core/query/QueryResultTest.java (added) +++ jackrabbit/trunk/jackrabbit/src/test/java/org/apache/jackrabbit/core/query/QueryResultTest.java Tue Nov 28 09:44:54 2006 @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.jackrabbit.core.query; + +import javax.jcr.RepositoryException; +import javax.jcr.NodeIterator; +import javax.jcr.query.QueryManager; +import javax.jcr.query.Query; +import javax.jcr.query.QueryResult; +import java.util.NoSuchElementException; + +/** + * QueryResultTest tests various methods on the + * NodeIterator returned by a QueryResult. + */ +public class QueryResultTest extends AbstractQueryTest { + + private static int INITIAL_NODE_NUM = 55; + + protected void setUp() throws Exception { + super.setUp(); + for (int i = 0; i < INITIAL_NODE_NUM; i++) { + testRootNode.addNode("node" + i).setProperty(propertyName1, i); + } + testRootNode.save(); + } + + public void testGetSize() throws RepositoryException { + QueryManager qm = superuser.getWorkspace().getQueryManager(); + for (int i = 0; i < 10; i++) { + String stmt = testPath + "/*[@" + propertyName1 + " < 1000]"; + QueryResult result = qm.createQuery(stmt, Query.XPATH).execute(); + assertEquals("Wrong size of NodeIterator in result", + INITIAL_NODE_NUM - i, result.getNodes().getSize()); + // remove node for the next iteration + testRootNode.getNode("node" + i).remove(); + testRootNode.save(); + } + } + + public void testGetSizeOrderByScore() throws RepositoryException { + QueryManager qm = superuser.getWorkspace().getQueryManager(); + for (int i = 0; i < 10; i++) { + String stmt = testPath + "/*[@" + propertyName1 + " < 1000] order by jcr:score()"; + QueryResult result = qm.createQuery(stmt, Query.XPATH).execute(); + assertEquals("Wrong size of NodeIterator in result", + INITIAL_NODE_NUM - i, result.getNodes().getSize()); + // remove node for the next iteration + testRootNode.getNode("node" + i).remove(); + testRootNode.save(); + } + } + + public void testIteratorNext() throws RepositoryException { + QueryManager qm = superuser.getWorkspace().getQueryManager(); + for (int i = 0; i < 10; i++) { + String stmt = testPath + "/*[@" + propertyName1 + " < 1000]"; + QueryResult result = qm.createQuery(stmt, Query.XPATH).execute(); + int size = 0; + for (NodeIterator it = result.getNodes(); it.hasNext(); ) { + it.nextNode(); + size++; + } + assertEquals("Wrong size of NodeIterator in result", + INITIAL_NODE_NUM - i, size); + // remove node for the next iteration + testRootNode.getNode("node" + i).remove(); + testRootNode.save(); + } + } + + public void testIteratorNextOrderByScore() throws RepositoryException { + QueryManager qm = superuser.getWorkspace().getQueryManager(); + for (int i = 0; i < 10; i++) { + String stmt = testPath + "/*[@" + propertyName1 + " < 1000] order by jcr:score()"; + QueryResult result = qm.createQuery(stmt, Query.XPATH).execute(); + int size = 0; + for (NodeIterator it = result.getNodes(); it.hasNext(); ) { + it.nextNode(); + size++; + } + assertEquals("Wrong size of NodeIterator in result", + INITIAL_NODE_NUM - i, size); + // remove node for the next iteration + testRootNode.getNode("node" + i).remove(); + testRootNode.save(); + } + } + + public void testSkip() throws RepositoryException { + QueryManager qm = superuser.getWorkspace().getQueryManager(); + for (int i = 0; i < 10; i++) { + String stmt = testPath + "/*[@" + propertyName1 + " < 1000]"; + QueryResult result = qm.createQuery(stmt, Query.XPATH).execute(); + for (int j = 0; j < INITIAL_NODE_NUM - i; j++) { + // skip to each node in the result + NodeIterator it = result.getNodes(); + it.skip(j); + long propValue = it.nextNode().getProperty(propertyName1).getLong(); + // expected = number of skipped nodes + number of deleted nodes + long expected = j + i; + assertEquals("Wrong node after skip()", expected, propValue); + } + try { + NodeIterator it = result.getNodes(); + it.skip(it.getSize() + 1); + fail("must throw NoSuchElementException"); + } catch (NoSuchElementException e) { + // correct + } + // remove node for the next iteration + testRootNode.getNode("node" + i).remove(); + testRootNode.save(); + } + } + + public void testSkipOrderByProperty() throws RepositoryException { + QueryManager qm = superuser.getWorkspace().getQueryManager(); + for (int i = 0; i < 10; i++) { + String stmt = testPath + "/*[@" + propertyName1 + + " < 1000] order by @" + propertyName1; + QueryResult result = qm.createQuery(stmt, Query.XPATH).execute(); + for (int j = 0; j < INITIAL_NODE_NUM - i; j++) { + // skip to each node in the result + NodeIterator it = result.getNodes(); + it.skip(j); + long propValue = it.nextNode().getProperty(propertyName1).getLong(); + // expected = number of skipped nodes + number of deleted nodes + long expected = j + i; + assertEquals("Wrong node after skip()", expected, propValue); + } + try { + NodeIterator it = result.getNodes(); + it.skip(it.getSize() + 1); + fail("must throw NoSuchElementException"); + } catch (NoSuchElementException e) { + // correct + } + // remove node for the next iteration + testRootNode.getNode("node" + i).remove(); + testRootNode.save(); + } + } + + public void testGetPosition() throws RepositoryException { + QueryManager qm = superuser.getWorkspace().getQueryManager(); + for (int i = 0; i < 10; i++) { + String stmt = testPath + "/*[@" + propertyName1 + " < 1000]"; + QueryResult result = qm.createQuery(stmt, Query.XPATH).execute(); + NodeIterator it = result.getNodes(); + assertEquals("Wrong position", 0, it.getPosition()); + int count = 0; + while (it.hasNext()) { + long position = it.getPosition(); + it.nextNode(); + assertEquals("Wrong position", count++, position); + } + try { + it.next(); + fail("must throw NoSuchElementException"); + } catch (Exception e) { + // correct + } + // remove node for the next iteration + testRootNode.getNode("node" + i).remove(); + testRootNode.save(); + } + } + + public void testGetPositionOrderBy() throws RepositoryException { + QueryManager qm = superuser.getWorkspace().getQueryManager(); + for (int i = 0; i < 10; i++) { + String stmt = testPath + "/*[@" + propertyName1 + " < 1000] order by jcr:score()"; + QueryResult result = qm.createQuery(stmt, Query.XPATH).execute(); + NodeIterator it = result.getNodes(); + assertEquals("Wrong position", 0, it.getPosition()); + int count = 0; + while (it.hasNext()) { + long position = it.getPosition(); + it.nextNode(); + assertEquals("Wrong position", count++, position); + } + try { + it.next(); + fail("must throw NoSuchElementException"); + } catch (Exception e) { + // correct + } + // remove node for the next iteration + testRootNode.getNode("node" + i).remove(); + testRootNode.save(); + } + } + + public void testPositionEmptyResult() throws RepositoryException { + QueryManager qm = superuser.getWorkspace().getQueryManager(); + String stmt = testPath + "/*[@" + propertyName1 + " > 1000]"; + QueryResult result = qm.createQuery(stmt, Query.XPATH).execute(); + NodeIterator it = result.getNodes(); + assertEquals("Wrong position", 0, it.getPosition()); + stmt += " order by jcr:score()"; + result = qm.createQuery(stmt, Query.XPATH).execute(); + it = result.getNodes(); + assertEquals("Wrong position", 0, it.getPosition()); + } +} Propchange: jackrabbit/trunk/jackrabbit/src/test/java/org/apache/jackrabbit/core/query/QueryResultTest.java ------------------------------------------------------------------------------ svn:eol-style = native Modified: jackrabbit/trunk/jackrabbit/src/test/java/org/apache/jackrabbit/core/query/TestAll.java URL: http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit/src/test/java/org/apache/jackrabbit/core/query/TestAll.java?view=diff&rev=480138&r1=480137&r2=480138 ============================================================================== --- jackrabbit/trunk/jackrabbit/src/test/java/org/apache/jackrabbit/core/query/TestAll.java (original) +++ jackrabbit/trunk/jackrabbit/src/test/java/org/apache/jackrabbit/core/query/TestAll.java Tue Nov 28 09:44:54 2006 @@ -47,6 +47,7 @@ suite.addTestSuite(VersionStoreQueryTest.class); suite.addTestSuite(UpperLowerCaseQueryTest.class); suite.addTestSuite(ChildAxisQueryTest.class); + suite.addTestSuite(QueryResultTest.class); // exclude long running tests per default //suite.addTestSuite(MassiveRangeTest.class);