lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: FieldCache Question
Date Wed, 04 Feb 2009 17:01:17 GMT
Todd Benge wrote:
> Hi,
>
> I've been looking into the FieldCache API because of memory problems
> we've been seeing in our production environment.
>
> We use various different sorts so over time the cache builds up and
> servers stop responding.
>
> I decided to apply the patch for JIRA 831:
>
> https://issues.apache.org/jira/browse/LUCENE-831
>   
This patch is likely to change quite a bit soon. A lot of what it was 
doing has been taken care of by a separate issue. Whats likely to remain 
is just the API change, but even that is likely to change (an iterator 
api was discussed). So thats a volatile patch at the moment.
> so that I can switch out the caching to an LRU based cache.
>
> One thing I noticed in the code is that there is an array cache built
> per IndexReader.  I'm wondering what the consequences would be if the
> arrays were merged for all IndexReaders.  That would result in single
> cached array for Strings, Floats, Ints, etc.
>   
The original intent of this patch was almost to do the opposite if I am 
understanding you right. In any case, I am not sure why you think this 
will help?
> It looks like the arrays are used by the various comparators during
> the sort but wasn't exactly sure so thought I'd ask what would happen
> if I combined the arrays.
>   
I wouldn't combine them. Wouldn't this give less granularity to your LRU 
cache anyway? That code is likely to change drastically, and I'm not 
sure what advantage you would see. Do you have an index so large that 
you can't supply the necessary RAM for the fieldcaches?

How is your custom cache implementation working? Since you are trying to 
tweak things, it sounds like you are still having issues? I am not sure 
how combining arrays would help you, but I don't understand what you are 
trying to solve yet.

- Mark

> Thanks,
>
> Todd
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>   


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message