lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Russell Black (Issue Comment Edited) (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (SOLR-3109) group=true requests result in numerous redundant shard requests
Date Thu, 09 Feb 2012 16:07:59 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204533#comment-13204533
] 

Russell Black edited comment on SOLR-3109 at 2/9/12 4:06 PM:
-------------------------------------------------------------

The current {{TestDistributedGrouping}} test case is constructed in such a way that each record
has a unique value for it's search group field ({{i1}}), so that there is never more than
one record in any given search group.  This style of indexing conforms to the restriction
discussed earlier.  This is likely the reason there were no test failures.  
                
      was (Author: rblack):
    The test case is constructed in such a way that each record has a unique value for it's
search group field ({{i1}}), so that there is never more than one record in any given search
group.  This style of indexing conforms to the restriction discussed earlier.  This is likely
the reason there were no test failures.  
                  
> group=true requests result in numerous redundant shard requests
> ---------------------------------------------------------------
>
>                 Key: SOLR-3109
>                 URL: https://issues.apache.org/jira/browse/SOLR-3109
>             Project: Solr
>          Issue Type: Bug
>          Components: search
>    Affects Versions: 3.5, 4.0
>         Environment: 64-bit Linux, sharded environment
>            Reporter: Russell Black
>            Assignee: Martijn van Groningen
>            Priority: Critical
>              Labels: patch, performance
>         Attachments: SOLR-3109.patch, SOLR-3109.patch, SOLR-3109.patch
>
>
> During the second phase of a group query, the collator sends a query to each of the shards.
 The purpose of this query is for shards to respond with the doc ids that match the set of
group ids returned from the first phase.  The problem is that it sends this second query to
each shard multiple times.  Specifically, in an environment with n shards, each shard will
be hit with an identical query n times during the second phase of query processing, resulting
in O(_n_ ^2^) performance where _n_ is the number of shards.
> I have traced this bug down to a single line in {{TopGroupsShardRequestFactory.java}},
and I am attaching a patch. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message