lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Steigerwald (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-236) Field collapsing
Date Thu, 24 Jun 2010 19:41:36 GMT

    [ https://issues.apache.org/jira/browse/SOLR-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882299#action_12882299
] 

Doug Steigerwald commented on SOLR-236:
---------------------------------------

I keep running into an ArrayIndexOutOfBoundsException when sorting with field collapsing.
 I'm running Solr 1.4.1 with the field-collapse-5.patch along with the 3 files from Peter
for OOM issues.

We've got a basic query that returns all event type records in the index (object_class:events),
and one fq to make sure we're grabbing data for the correct site (site_id:86).  I'm sorting
on a category_id (TrieIntField).  Collapsing on a string (collapse.type=normal).  Here's a
basic query that doesn't work for us.

q=object_class:events&fq=site_id:86&sort=category_id+desc&collapse.field=rollup&collapse.type=normal

Jun 24, 2010 3:20:12 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.ArrayIndexOutOfBoundsException: -4294
	at org.apache.lucene.search.FieldComparator$IntComparator.copy(FieldComparator.java:328)
	at org.apache.lucene.search.TopFieldCollector$OutOfOrderOneComparatorNonScoringCollector.collect(TopFieldCollector.java:133)
	at org.apache.solr.search.SolrIndexSearcher.sortDocSet(SolrIndexSearcher.java:1487)
	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:931)
	at org.apache.solr.search.SolrIndexSearcher.getDocListAndSet(SolrIndexSearcher.java:1289)
	at org.apache.solr.handler.component.CollapseComponent.doProcess(CollapseComponent.java:176)
	at org.apache.solr.handler.component.CollapseComponent.process(CollapseComponent.java:127)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)


This is happening to one of our sites in production (the only site left using our events calendar)
and I can't seem to make it happen in development with some fake data.  We wiped all data
from our production indexes and reindexed recently (upgraded to Solr 1.4.0 a few weeks ago).
 Does anyone have any ideas what might be causing this?  I'm going to try and pull the database
to our development servers and see if I can reindex and reproduce the issue, but that will
take some time.  The copied index from production to development does show this issue.

Any hints?  This is happening when sorting on any TrieIntField or string field.  Normal collapsing
or adjacent.

> Field collapsing
> ----------------
>
>                 Key: SOLR-236
>                 URL: https://issues.apache.org/jira/browse/SOLR-236
>             Project: Solr
>          Issue Type: New Feature
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Emmanuel Keller
>            Assignee: Shalin Shekhar Mangar
>             Fix For: Next
>
>         Attachments: collapsing-patch-to-1.3.0-dieter.patch, collapsing-patch-to-1.3.0-ivan.patch,
collapsing-patch-to-1.3.0-ivan_2.patch, collapsing-patch-to-1.3.0-ivan_3.patch, DocSetScoreCollector.java,
field-collapse-3.patch, field-collapse-4-with-solrj.patch, field-collapse-5.patch, field-collapse-5.patch,
field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch,
field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch,
field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch, field-collapse-5.patch,
field-collapse-5.patch, field-collapse-solr-236-2.patch, field-collapse-solr-236.patch, field-collapsing-extended-592129.patch,
field_collapsing_1.1.0.patch, field_collapsing_1.3.patch, field_collapsing_dsteigerwald.diff,
field_collapsing_dsteigerwald.diff, field_collapsing_dsteigerwald.diff, NonAdjacentDocumentCollapser.java,
NonAdjacentDocumentCollapserTest.java, quasidistributed.additional.patch, SOLR-236-FieldCollapsing.patch,
SOLR-236-FieldCollapsing.patch, SOLR-236-FieldCollapsing.patch, SOLR-236-trunk.patch, SOLR-236-trunk.patch,
SOLR-236-trunk.patch, SOLR-236-trunk.patch, SOLR-236-trunk.patch, SOLR-236.patch, SOLR-236.patch,
SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, SOLR-236.patch, solr-236.patch,
SOLR-236_collapsing.patch, SOLR-236_collapsing.patch
>
>
> This patch include a new feature called "Field collapsing".
> "Used in order to collapse a group of results with similar value for a given field to
a single entry in the result set. Site collapsing is a special case of this, where all results
for a given web site is collapsed into one or two entries in the result set, typically with
an associated "more documents from this site" link. See also Duplicate detection."
> http://www.fastsearch.com/glossary.aspx?m=48&amid=299
> The implementation add 3 new query parameters (SolrParams):
> "collapse.field" to choose the field used to group results
> "collapse.type" normal (default value) or adjacent
> "collapse.max" to select how many continuous results are allowed before collapsing
> TODO (in progress):
> - More documentation (on source code)
> - Test cases
> Two patches:
> - "field_collapsing.patch" for current development version
> - "field_collapsing_1.1.0.patch" for Solr-1.1.0
> P.S.: Feedback and misspelling correction are welcome ;-)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message