lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From KaktuChakarabati <jimmoe...@gmail.com>
Subject Re: Solr RAM Requirements
Date Wed, 17 Mar 2010 00:29:36 GMT

Hey Peter,
Thanks for your reply.
My question was mainly about the fact there seems to be two different
aspects to the solr RAM usage: in-process and out-process. 
By that I mean, yes i know the many different parameters/caches to do with
solr in-process memory usage and related culprits, however I also understand
that as for actual index access (posting list, positional index etc), solr
mostly delegates the access/caching of this to the OS/disk cache. 
So I guess my question is more about that: namely, what would be a good way
to calculate an overall ram requirement profile for a server running solr? 
Also, I was under the impression benefits from RAMDirectory would be minimal
when disk caches are effective no?
And does RAMDirectory work with replication? if so, doesnt it slow it down?
( on each replication, load up entire index to RAM at once? )



Peter Sturge wrote:
> 
> On Tue, Mar 16, 2010 at 9:08 PM, KaktuChakarabati
> <jimmoefoe@gmail.com>wrote:
> 
>>
>> Hey,
>> I am trying to understand what kind of calculation I should do in order
>> to
>> come up with reasonable RAM size for a given solr machine.
>>
>> Suppose the index size is at 16GB.
>> The Max heap allocated to JVM is about 12GB.
>>
>> The machine I'm trying now has 24GB.
>> When the machine is running for a while serving production, I can see in
>> top
>> that the resident memory taken by the jvm is indeed at 12gb.
>> Now, on top of this i should assume that if i want the whole index to fit
>> in
>> disk cache i need about 12gb+16gb = 28GB of RAM just for that. Is this
>> kind
>> of calculation correct or am i off here?
>>
>>
> Hmmm..not quite. The idea of the ram usage isn't to simply hold the index
> in
> memory - if you want this use a RAMDirectory.
> The memory being used will be a combination of various caches (Lucene and
> Solr), index buffers et al., and of course the server itself. The
> specifics
> depend very
> much on what your server is doing at any given time - e.g. lots of
> concurrent searches, lots of indexing, both etc., and how things are setup
> in your solrconfig.xml.
> 
> A really excellent resource that's worth looking at regarding all this can
> be found here:
> 
> http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Scaling-Lucene-and-Solr
> 
> 
> 
>> Any other recommendations Anyone could make w.r.t these numbers ?
>>
>> Thanks,
>> -Chak
>> --
>> View this message in context:
>> http://old.nabble.com/Solr-RAM-Requirements-tp27924551p27924551.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
> 
> 

-- 
View this message in context: http://old.nabble.com/Solr-RAM-Requirements-tp27924551p27926536.html
Sent from the Solr - User mailing list archive at Nabble.com.


Mime
View raw message