lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: Sort-field for ALL docs in FieldCache for sort queries -> OOM on lots of docs
Date Thu, 21 Mar 2013 21:50:14 GMT
On 3/21/2013 4:05 AM, Per Steffensen wrote:
> Can anyone else elaborate? How to "activate" it? How to make sure, for
> sorting, that sort-field-value for all docs are not read into memory for
> sorting - leading to OOM when you have a lot of docs? Can this feature
> be activated on top of an existing 4.0 index, or do you have to re-index
> everything?

There is one requirement that may not be obvious - every document must 
have a value in the field, so you must either make the field either 
required or give it a default value in the schema.  Solr 4.2 will refuse 
to start the core if this requirement is not met.  The example schema 
hints that the value might need to be single-valued.  I have not tested 
this.  Sorting is already problematic on multi-valued fields, so I 
assume that this won't be the case for you.

To use docValues, add docValues="true" and then either set 
required="true" or default="<somevalue>" on the field definition in 
schema.xml, restart Solr or reload the core, and reindex.  Your index 
will get bigger.

If the touted behavior of handling the sort mechanism in OS disk cache 
memory (or just reading the disk if there's not enough memory) rather 
than heap is correct, then it should solve your issues.  I hope it does!

Thanks,
Shawn


Mime
View raw message