lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Britske <gbr...@gmail.com>
Subject fieldcache gives OOM. Deos a LRU-style fieldcache exist?
Date Mon, 12 Nov 2007 15:46:29 GMT

First my question: 

Is there an (experimental / patch-version) lucene-fieldcache available which
uses some kind of eviction-strategy (LRU or whatever) so that OOM's would
never happen in my case, but instead some sorts would simply get evicted?

Now the background for those who are interested: 

I'm having a usecase in which products can have a lot of variants. (Up to
10.000) 
These variants can each have their own price (Which I need to sort and
filter by ) and a bunch of stored values (which I just need to display to
the client). At all times only 1 variant per product needs to be shown in a
search-result. I've modeled this by letting each possible variant have 2
fields: 1 pricefield and 1 storagefield. This means that every entry in the
index is a product with possible more than 20.000 fields. Potentially:
because the actual collection of available variants is sparse per product.
This all sounds a bit exotic I know, but uptil I was pointed to this problem
everything went fine. (index-time is a 'bit' slow but that's no real
problem.)

The problem is that potentially all pricefields of all variants can be used
for sorting, meaning that the fieldcache would have to contain up to 10.000
cached lists. This will give OutOfMemory-errors sooner or later, (products
will grow to about 400.000, so the fieldcache will grow to about 16GB) which
is not what my evironment will be able to handle any time soon.

Thanks in advance, 

Geert-Jan
-- 
View this message in context: http://www.nabble.com/fieldcache-gives-OOM.-Deos-a-LRU-style-fieldcache-exist--tf4791826.html#a13707745
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message