lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam H." <jimmoe...@gmail.com>
Subject Re: FieldCache usage for custom field collapse in solr 1.4
Date Tue, 07 Dec 2010 00:29:49 GMT
One more comment/question -
Having looked at the Solr stats panel, I do not see detailed memory usage
for the field i'm collapsing on in the lucene FieldCache entries listings.

As I understand ( after having looked through this ticket:
https://issues.apache.org/jira/browse/SOLR-1292 ), this means that its not
an 'insanity' instance,
and so actually I am not using double the memory, but rather only have this
field in the FieldCache on the whole index level.

This got me thinking - If i'm not using any segment-level fieldcaching for
this field, there's no reason not to use an index-wide one,
as long as I can guarantee thats the only use case for this field in the
fieldcache.. is this correct?

Thanks again for helping me out with this delicate subject :)

Adam

On Mon, Dec 6, 2010 at 3:21 PM, Adam H. <jimmoefoe@gmail.com> wrote:

> ah! so just so I can get cracking on this - Can you be alittle more
> specific? e.g
>
> in my component implementation that runs in the request handling after the
> normal QueryComponent,
> How would I access the specific field value for the documents that were
> retrieved?
>
> i.e how would it fit in a code like this if at all:
>
> // docList is the matching documents for given offset/rows/query
> DocIterator it = docList.iterator();
>
>         while (it.hasNext()) {
>             docId = it.next();
>             score = it.score();
>
>
>             // this would've worked if this was stored field:
>             // reader.document(docId).get(fieldName)
>             ??
>
>         }
>
>
>
> On Mon, Dec 6, 2010 at 2:57 PM, Yonik Seeley <yonik@lucidimagination.com>wrote:
>
>> On Mon, Dec 6, 2010 at 5:48 PM, Adam H. <jimmoefoe@gmail.com> wrote:
>> > In other words, using a per-segment fieldcache collection as a
>> > post-processing step (e.g after QueryComponent did its collection) does
>> not
>> > seem at all trivial, if at all possible ( is it possible? )
>>
>> Sure, it's possible, and not too hard (as long as no sort field involves
>> score).
>> Just instruct the QueryComponent to retrieve the set of all matching
>> documents, then you can use that to run then through whatever
>> collectors you want again.  I've been meaning to implement this
>> optimization to field collapsing...
>>
>> Depending on the details, either replacing the QueryComponent with
>> your custom one, or inserting an additional component after the query
>> component could make sense.
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>

Mime
View raw message