lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam H." <jimmoe...@gmail.com>
Subject Re: FieldCache usage for custom field collapse in solr 1.4
Date Mon, 06 Dec 2010 20:24:53 GMT
Hey Yonik.
Thanks for clarifying.
The reason I went rolling my own way - I asked previously is there's any
plan to back-port the field collapse to solr 1.4 and
I understood that its not at all straight forward.

If you think it'll be fairly easy to look at the new code in Solr 4.0 trunk
and use that as basis for example I'd go ahead and do that.

Q - does the field collapse componet expect the field to collapse on to be
stored? or does it also try to use field cache trickery?

Thanks,
Adam

On Mon, Dec 6, 2010 at 9:42 AM, Yonik Seeley <yonik@lucidimagination.com>wrote:

> On Sun, Dec 5, 2010 at 6:12 PM, Adam H. <jimmoefoe@gmail.com> wrote:
> > StringIndex fieldCacheVals = FieldCache.DEFAULT.getStringIndex(reader,
> > collapseField);
> >
> > where 'reader' is the instance of the SolrIndexReader passed along to the
> > component with the ResponseBuilder.SolrQueryRequest object.
> >
> > As I understand, this can double memory usage due to (re)loading this
> > fieldcache on a reader-wide basis rather than on a per segment basis?
>
> Yep.  Sorting and function queries use per-segment FieldCache entries.
> So If you also request a FieldCache from the top level reader, it
> won't reuse the per-segment caches and hence will take up 2x memory
> over just using per-segment.
>
> Solr's field collapsing already works on a per-segment basis... if
> your needs are at all general, it could make sense to try and get it
> rolled into solr rather than implementing custom code.
>
> -Yonik
> http://www.lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Mime
View raw message