lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eran Sevi" <erans...@gmail.com>
Subject Re: Sorting consumes hundreds of MBytes RAM
Date Sun, 27 Apr 2008 06:57:39 GMT
If you read the payloads in sequence they're not arranged by their original
position whereas when you use a stored field you get the terms in the
correct order.
If you need to sort the values it doesn't matter of course.
On Fri, Apr 25, 2008 at 5:42 PM, Nadav Har'El <nyh@math.technion.ac.il>
wrote:

> On Mon, Apr 14, 2008, Chris Hostetter wrote about "Re: Sorting consumes
> hundreds of MBytes RAM":
> > : And question #2: what am I going to do against it? Index  sharding?
> >
> > The only suggestion i can offer is to take a look at LUCENE-769 ... it
> > takes a completley differnet appraoch of using a FieldSelector to access
> > the *stored* field and sort on it ... the memory usage of FieldCache is
> > eliminatedand the expense of longer search times ... in cases where you
> > expect queries to match on a very small subset of the total index, it
> > could be worth using.
>
> Instead of using a stored field, I would recommend using *payloads*.
> If you store the field's valye as payload on a custom term, you basically
> get a posting-list of the field value, which can be (theoretically, at
> least)
> efficiently skipped on one hand - and read in sequence on the other hand.
>
> --
> Nadav Har'El                        |       Friday, Apr 25 2008, 20 Nisan
> 5768
> IBM Haifa Research Lab
>  |-----------------------------------------
>                                    |Business jargon is the art of saying
> http://nadav.harel.org.il           |nothing while appearing to say a lot.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message