lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: java.lang.OutOfMemoryError: Java heap space when sorting the fields
Date Wed, 19 Mar 2008 23:53:01 GMT

: You might think, if I only ask for the top 10 docs, don't i only read 10 field
: values? But of course you don't know what docs will be returned as each search
: comes in...so you have to cache them all.

Arguements have been made in the past that when you have an index 
large enough that the FieldCache is prohibitively expensive, and you 
expect the number of *matching* documents to be significantly smaller then 
the total number of documents, it might make sense to Sort using 
LazyLoaded "stored" field values ... it's a pretty radical shift from the 
current sorting approach, and would require some explicit work on the part 
of the indexing/searching code, but a patch along that lines was created a 
while back...

https://issues.apache.org/jira/browse/LUCENE-769

...based on the last comment i posted in that issue, i clearly 
thought it showed a lot of promise, but needed cleaned up.  I admittedly 
lost track of it as other things took priority in life.  if people want 
to dust it off, try it out, and make comments about it's effectiveness I'm 
sure it could be revived.




-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message