lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abhijit bashetti <abhijitbashe...@gmail.com>
Subject Re: OOM issue
Date Thu, 15 Sep 2011 13:40:48 GMT
Hi Eric,

Thanks for the reply. It is very useful for me.

For point 1. : I do need 10 core and it will go on increasing in future.

I have document that belongs to different workspaces , so the

1 workspace = 1 core ; I cant go with one core. Currrently having 10 core

but in future the count may go 40+.


For point 2.: Currently I have not given any thought on it , but yes I
think in future I

may have to go for the master/slave setup


For point 3: the current cache size for document cache , filter cache
and query cache is 512 for each

the ramBufferSizeMB size is 512M. Shall I reduce the same to 128M?

For point 4: I didnot get you why should I use SolrJ with Tika? Do you mean
sending the new/updated documents to Tika for reindexing?
Then I am already doing it using data-config. I have written the query in
data-config in way that it takes the path of updated/new documents.

Thanks in advance!

Regards,
Abhijit


Multiple webapps will not help you, they're still on the underlying
memory. In fact, it'll make matters worse since they won't share
resources.

So questions become:
1> Why do you have 10 cores? Putting 10 cores on the same machine
doesn't really do much. It can make lots of sense to put 10 cores on the
same machine for *indexing*, then replicate them out. But putting
10 cores on one machine in hopes of making better use of memory
isn't useful. It may be useful to just go to one core.

2> Indexing, reindexing and searching on a single machine is requiring a
lot from that machine. Really you should consider having a master/slave
setup.

3> But assuming more hardware of any sort isn't in the cards, sure. reduce
your cache sizes. Look at <ramBufferSizeMB> and make it small.

4> Consider indexing with Tika via SolrJ and only sending the finished
document to Solr.

Best
Erick

On Mon, Sep 12, 2011 at 5:42 AM, Manish Bafna <manish.bafna.82@gmail.com> wrote:
> Number of cache is definitely going to reduce heap usage.
>
> Can you run those xlsx file separately with Tika and see if you are getting
> OOM issue.


> On Mon, Sep 12, 2011 at 3:09 PM, abhijit bashetti <abhijitbashetti@gmail.com
>> wrote:
>
>> I am facing the OOM issue.
>>
>> OTHER than increasing the RAM , Can we chnage some other parameters to
>> avoid the OOM issue.
>>
>>
>> such as minimizing the filter cache size , document cache size etc.
>>
>> Can you suggest me some other option to avoid the OOM issue?
>>
>>
>> Thanks in advance!
>>
>>
>> Regards,
>>
>> Abhijit

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message