lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martijn van Groningen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-2205) Grouping performance improvements
Date Fri, 29 Oct 2010 18:49:19 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12926405#action_12926405
] 

Martijn van Groningen commented on SOLR-2205:
---------------------------------------------

I wrote that in a hurry, but group.sort should be used. But you've to keep track of the lowest
first document of the top x groups in a separate field. If it less competitive then that you
can neglect the incoming doc, right? Obviously then you need to update the lowest first document
if the incoming doc is more competitive. So the code is different then what is in the TopGroupCollector.

bq.  we still really need to develop some good random tests to verify any optimizations +
corner case
I agree with that, we need more tests that show that the grouping functionality is working
properly. Should we put that in this issue or create an separate issue for this?

> Grouping performance improvements
> ---------------------------------
>
>                 Key: SOLR-2205
>                 URL: https://issues.apache.org/jira/browse/SOLR-2205
>             Project: Solr
>          Issue Type: Sub-task
>          Components: search
>    Affects Versions: 4.0
>            Reporter: Martijn van Groningen
>             Fix For: 4.0
>
>         Attachments: SOLR-2205.patch, SOLR-2205.patch
>
>
> This issue is dedicated to the performance of the grouping functionality.
> I've noticed that the code is not really performing on large indexes. Doing a search
(q=*:*) with grouping on an index from around 5M documents took around one second on my local
development machine. We had to support grouping on an index that holds around 50M documents
per machine, so we made some changes and were able to happily serve that amount of documents.
Patch will follow soon.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message