lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vishnu perumal <vis...@knackforge.com>
Subject Re: SOLR server - Memory issue
Date Tue, 28 Jul 2015 04:24:21 GMT
Thanks for the reply..

On Wed, Jul 22, 2015 at 9:55 PM, Erick Erickson <erickerickson@gmail.com>
wrote:

> Frankly, I'm surprised it runs at all. 650M dos in 2G of memory is very,
> very, very aggressive.
> To get it to run at all I'm guessing that you have turned off things like
> term vectors, position info
> and the like.
>
> Anyway...
> Solr/Lucene cache various data as queries come in, so the scenario is this
> I think
> > Solr starts up
> > you fire a query or two at it and it tries to load  caches then blows up
> with OOM errors.
>
> The fact that all your physical memory is used up is due to MMapDirectory
> and isn't a problem, see:
>
> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
>
> But you're really putting more documents in these machines than I _ever_
> expect to see
> in that little memory. I think you need to back up and do a sizing
> exercise to set expectations, see:
>
>
> https://lucidworks.com/blog/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/
>
> Best,
> Erick
>
>
> On Wed, Jul 22, 2015 at 4:26 AM, Vishnu perumal <vishnu@knackforge.com>
> wrote:
>
>> HI,
>>
>> I am running SOLR server ( solr version 4.10.3 & java version 1.7.0_67).
>>
>> Server Details : Amazon aws c3.xlarge server (7.5 GB RAM, 4vCPU). Linux
>> Ubuntu 14.04.1 LTS and installed  Bitnami Apache Solr 4.10.3-0. Allocated
>> JVM memory minimum 1GB and maximun 2GB. The server has around 650 million
>> dcoucment indexed (Indexed field count is 37 & stored field count is 10).
>>
>> [image: Inline image 1]
>>
>> When the solr server starts, it works fine for the few queries (filter
>> query). But after running a few more select queries (SpatialSearch), it
>> takes longer to respond the queries and physical memory usage goes to full
>> (99.5%).
>>
>> [image: Inline image 2]
>>
>> and getting the java heap space error.
>>
>>
>> [image: Inline image 3]
>>
>>
>> Does it cache all the result? but I have disabled documents caching,
>> field caching. I am expecting that after query gets completed, the memory
>> should be released in a few seconds. But that is not happening. Do I miss
>> anything here?
>>
>>
>> I have atached the config file here for your reference. I must admit, I
>> am not that expert with Solr.
>>
>>
>>
>>
>>
>> Thankyou,
>> Vishnu Perumal,
>> Vishnu@knackforge.com
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message