lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: DIH - Out of Memory error when using CachedsqlEntityProcessor
Date Sat, 13 Mar 2010 23:36:03 GMT
On 03/13/2010 06:26 PM, Mark Miller wrote:
> I don't really follow DataImportHandler, but it looks like its using 
> an unbounded cache (simple HashMap).
>
> Perhaps we should make the cache size configurable?
>
> The impl seems a little odd - the caching occurs in the base class - 
> so caching impls that extends it don't really have full control - they 
> just kind of "turn on" the caching in the base class? Kind of an odd 
> approach - to cache you have to turn on the cache support in the base 
> class and impl a couple custom methods as well?
>
Looking a little closer, really it seems like all of the caching support 
should be lifted out of EntityProcessorBase and into something like 
CachedEntityProcessorBase. Not a huge deal, but a cleaner design I 
think. There is no real need for anyone looking at EntityProcessorBase 
to think about caching.

Then caching impls can either extend that for some base support, or just 
cache in a completely different way - without the "default caching" kind 
of always being in the chain (even though it's technically "off").

-- 
- Mark

http://www.lucidimagination.com




Mime
View raw message