lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martijn van Groningen (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-3109) group=true requests result in numerous redundant shard requests
Date Thu, 09 Feb 2012 18:56:01 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204737#comment-13204737
] 

Martijn van Groningen commented on SOLR-3109:
---------------------------------------------

No worries :-) I didn't want to move this work to anyone. Just wanted to say that the test
needs to be updated.

I put that comment b/c in the following three lines group.ngroups and group.truncate features
are tested. These features *only* work properly if documents belonging to a group reside in
the same shard. If documents belonging to a group do occur in more than one shard then the
results are very likely incorrect.

Tomorrow I will update the test case and get this patch committed. If you want to update the
test case and have time for that that would be great!
                
> group=true requests result in numerous redundant shard requests
> ---------------------------------------------------------------
>
>                 Key: SOLR-3109
>                 URL: https://issues.apache.org/jira/browse/SOLR-3109
>             Project: Solr
>          Issue Type: Bug
>          Components: search
>    Affects Versions: 3.5, 4.0
>         Environment: 64-bit Linux, sharded environment
>            Reporter: Russell Black
>            Assignee: Martijn van Groningen
>            Priority: Critical
>              Labels: patch, performance
>         Attachments: SOLR-3109.patch, SOLR-3109.patch, SOLR-3109.patch
>
>
> During the second phase of a group query, the collator sends a query to each of the shards.
 The purpose of this query is for shards to respond with the doc ids that match the set of
group ids returned from the first phase.  The problem is that it sends this second query to
each shard multiple times.  Specifically, in an environment with n shards, each shard will
be hit with an identical query n times during the second phase of query processing, resulting
in O(_n_ ^2^) performance where _n_ is the number of shards.
> I have traced this bug down to a single line in {{TopGroupsShardRequestFactory.java}},
and I am attaching a patch. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message