lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Smiley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-6581) Efficient DocValues support and numeric collapse field implementations for Collapse and Expand
Date Mon, 25 Jul 2016 16:08:20 GMT

    [ https://issues.apache.org/jira/browse/SOLR-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392175#comment-15392175
] 

David Smiley commented on SOLR-6581:
------------------------------------

Dallan:
bq. Any ideas how much slower numeric collapse/expand implementation is than string collapse/expand
with the top_fc hint?

I would guess a numeric impl would beat a string impl every time.

Hey FYI everyone... this is weird but while I just finished doing a major Solr 4.10 ->
Solr 6.1.0 upgrade, I found that the top_fc for the collapse had quite the opposite effect
in a test environment _with no indexing/commits_.  top_fc took twice as long as without. 
Shrug; no clue.  So folks don't go setting this blindly without actually measuirng before
& after in your own environment.

> Efficient DocValues support and numeric collapse field implementations for Collapse and
Expand
> ----------------------------------------------------------------------------------------------
>
>                 Key: SOLR-6581
>                 URL: https://issues.apache.org/jira/browse/SOLR-6581
>             Project: Solr
>          Issue Type: Bug
>            Reporter: Joel Bernstein
>            Assignee: Joel Bernstein
>            Priority: Minor
>             Fix For: 5.0, 6.0
>
>         Attachments: SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch,
SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch,
SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, renames.diff
>
>
> The 4x implementation of the CollapsingQParserPlugin and the ExpandComponent are optimized
to work with a top level FieldCache. Top level FieldCaches have a very fast docID to top-level
ordinal lookup. Fast access to the top-level ordinals allows for very high performance field
collapsing on high cardinality fields. 
> LUCENE-5666 unified the DocValues and FieldCache api's so that the top level FieldCache
is no longer in regular use. Instead all top level caches are accessed through MultiDocValues.

> This ticket does the following:
> 1) Optimizes Collapse and Expand to use MultiDocValues and makes this the default approach
when collapsing on String fields
> 2) Provides an option to use a top level FieldCache if the performance of MultiDocValues
is a blocker. The mechanism for switching to the FieldCache is a new "hint" parameter. If
the hint parameter is set to "top_fc" then the top-level FieldCache would be used for both
Collapse and Expand.
> Example syntax:
> {code}
> fq={!collapse field=x hint=TOP_FC}
> {code}
> 3)  Adds numeric collapse field implementations.
> 4) Resolves issue SOLR-6066
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message