Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of anithian@gmail.com designates
 209.85.214.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type;
        b=N3dkK268yvJFDGClxA2IkB/y6TPD1kyPXeC1IFWXG4Ig2qP0SL5WjSiGsPPZlYI7dO
         3ZR1pb9jgxYen0RbkI6dfiB/To8etW9fJfOgQqWXP/ihc5MNlk0nROWt4nx8ZGBOUaeJ
         2YBfD03fEZ+1/tmZfJks77FwFoesEB/D5OZmU=
MIME-Version: 1.0
In-Reply-To: <1957FC4418E54F29BF7545C64017F545@udc70634P002>
References: <AANLkTikkP4QTaPa17W6bsR2tn1uba1X=P78EYiJRFfMb@mail.gmail.com>
 <AANLkTinvWr7L1FKrhKYPQDvtQJTN08J1Czhf1+O64=Mn@mail.gmail.com>
 <AANLkTi=fwHCfYPNweQUxzOG5T08oWNZSVYO5Vdm7d8Tr@mail.gmail.com>
 <AANLkTikE6geR7+XfB_FTc6XcJDgp8uFOYzbgGjBzf7U8@mail.gmail.com>
 <1957FC4418E54F29BF7545C64017F545@udc70634P002>
From: Amit Nithian <anithian@gmail.com>
Date: Mon, 30 Aug 2010 20:34:13 -0700
Message-ID: <AANLkTim7SAszQYNmXUiqQo7+-4PFqSOZrPoCh2n+qHRc@mail.gmail.com>
Subject: Re: Hardware Specs Question
To: solr-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=001485f872ca6a1868048f164243

--001485f872ca6a1868048f164243
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Lance,

makes sense and I have heard about the long GC times on large heaps but I
personally haven't experienced a slowdown but that doesn't mean anything
either :-). Agreed that tuning the SOLR caching is the way to go.

I haven't followed all the solr/lucene changes but from what I remember
there are synchronization points that could be a bottleneck where adding
more cores won't help this problem? Or am I completely missing something.

Thanks again
Amit

On Mon, Aug 30, 2010 at 8:28 PM, scott chu (=E6=9C=B1=E7=82=8E=E8=A9=B9) <s=
cott.chu@udngroup.com>wrote:

> I am also curious as Amit does. Can you make an example about the garbage
> collection problem you mentioned?
>
> ----- Original Message ----- From: "Lance Norskog" <goksron@gmail.com>
> To: <solr-user@lucene.apache.org>
> Sent: Tuesday, August 31, 2010 9:14 AM
> Subject: Re: Hardware Specs Question
>
>
>
>  It generally works best to tune the Solr caches and allocate enough
>> RAM to run comfortably. Linux & Windows et. al. have their own cache
>> of disk blocks. They use very good algorithms for managing this cache.
>> Also, they do not make long garbage collection passes.
>>
>> On Mon, Aug 30, 2010 at 5:48 PM, Amit Nithian <anithian@gmail.com> wrote=
:
>>
>>> Lance,
>>>
>>> Thanks for your help. What do you mean by that the OS can keep the inde=
x
>>> in
>>> memory better than Solr? Do you mean that you should use another means =
to
>>> keep the index in memory (i.e. ramdisk)? Is there a generally accepted
>>> heap
>>> size/index size that you follow?
>>>
>>> Thanks
>>> Amit
>>>
>>> On Mon, Aug 30, 2010 at 5:00 PM, Lance Norskog <goksron@gmail.com>
>>> wrote:
>>>
>>>  The price-performance knee for small servers is 32G ram, 2-6 SATA
>>>> disks on a raid, 8/16 cores. You can buy these servers and half-fill
>>>> them, leaving room for expansion.
>>>>
>>>> I have not done benchmarks about the max # of processors that can be
>>>> kept busy during indexing or querying, and the total numbers: QPS,
>>>> response time averages & variability, etc.
>>>>
>>>> If your index file size is 8G, and your Java heap is 8G, you will do
>>>> long garbage collection cycles. The operating system is very good at
>>>> keeping your index in memory- better than Solr can.
>>>>
>>>> Lance
>>>>
>>>> On Mon, Aug 30, 2010 at 4:52 PM, Amit Nithian <anithian@gmail.com>
>>>> wrote:
>>>> > Hi all,
>>>> >
>>>> > I am curious to know get some opinions on at what point having more =
>
>>>> CPU
>>>> > cores shows diminishing returns in terms of QPS. Our index size is >
>>>> about
>>>> 8GB
>>>> > and we have 16GB of RAM on a quad core 4 x 2.4 GHz AMD Opteron 2216.
>>>> > Currently I have the heap to 8GB.
>>>> >
>>>> > We are looking to get more servers to increase capacity and because =
>
>>>> the
>>>> > warranty is set to expire on our old servers and so I was curious >
>>>> before
>>>> > asking for a certain spec what others run and at what point does >
>>>> having
>>>> more
>>>> > cores cease to matter? Mainly looking at somewhere between 4-12 core=
s
>>>> > per
>>>> > server.
>>>> >
>>>> > Thanks!
>>>> > Amit
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Lance Norskog
>>>> goksron@gmail.com
>>>>
>>>>
>>>
>>
>>
>> --
>> Lance Norskog
>> goksron@gmail.com
>>
>>
>
>
> -------------------------------------------------------------------------=
-------
>
>
>
> ___b___J_T_________f_r_C
> Checked by AVG - www.avg.com
> Version: 9.0.851 / Virus Database: 271.1.1/3102 - Release Date: 08/30/10
> 14:35:00
>
>

--001485f872ca6a1868048f164243--