lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bohnsack, Sven" <Sven.Bohns...@shopping24.de>
Subject How to deal with many files using solr external file field
Date Tue, 07 Jun 2011 20:25:46 GMT
Hi all,

we're using solr 1.4 and external file field ([1]) for sorting our searchresults. We have
about 40.000 Terms, for which we use this sorting option.
Currently we're running into massive OutOfMemory-Problems and were not pretty sure, what's
the matter. It seems that the garbage collector stops working or some processes are going
wild. However, solr starts to allocate more and more RAM until we experience this OutOfMemory-Exception.


We noticed the following:

For some terms one could see in the solr log that there appear some java.io.FileNotFoundExceptions,
when solr tries to load an external file for a term for which there is not such a file, e.g.
solr tries to load the external score file for "trousers" but there ist none in the /solr/data-Folder.

Question: is it possible, that those exceptions are responsible for the OutOfMemory-Problem
or could it be due to the large(?) number of 40k terms for which we want to sort the result
via external file field?

I'm looking forward for your answers, suggestions and ideas :)


Regards
Sven


[1]: http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message