lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bbarani <bbar...@gmail.com>
Subject Need info on CachedSQLentity processor
Date Mon, 05 Apr 2010 18:28:03 GMT

Hi,

I am using cachedSqlEntityprocessor in DIH to index the data. Please find
below my dataconfig structure,

<entity x query="select * from x"> ---> object 
<entity y query="select * from y" processor="cachedSqlEntityprocessor"
cachekey=y.id cachevalue=x.id> --> object properties 

For each and every object I would be retrieveing corresponding object
properties (in my subqueries).

I get in to OOM very often and I think thats a trade off if I use
cachedSqlEntityprocessor. 

My assumption is that when I use cachedSqlEntityprocessor the indexing
happens as follows,

First entity x will get executed and the entire table gets stored in cache

next entity y gets executed and entire table gets stored in cache 

Finally the compasion heppens through hash map .

So always I need to have the memory allocated to SOLR JVM more than or equal
to the data present in tables?


Now my final question is that even after SOLR complexes indexing the memory
used previously is not getting released. I could still see the JVM consuming
1.5 GB after the indexing completes. I tried to use Java hotspot options but
didnt see any differences..

Any thoughts / confirmation on my assumptions above would be of great help
to me to get in to  a decision of choosing cachedSqlEntityprocessor or not.

Thanks,
BB



-- 
View this message in context: http://n3.nabble.com/Need-info-on-CachedSQLentity-processor-tp698418p698418.html
Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message