Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of magnito@gmail.com designates
 209.85.214.169 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=TTG+3GV5dwIaun2DNhTq24pEab4RdESxSwPanMR7ho9lmYZzTApEjTO+t7SC71m8dp
         HcdkhSVO8Ug6lo1vCiodl2DZpzmHmEneopkOPX2uPXLfNUhHeIjhkFs4F4JjjLP0npgk
         DB9dD/K0PuDwtVWgmdSkndLHH5uod0Q8rbeH0=
MIME-Version: 1.0
In-Reply-To: <AANLkTik3ROB4uxoeOe=0iEoAx51AOga7DP=tVmL-U+Yi@mail.gmail.com>
References: <AANLkTin_Pk3cPKvZg6YT5X_bBvckSr8Hn71ov+foF5YK@mail.gmail.com>
	<5A76F6CE309AD049AAF9A039A39242820F189C89@sc-mbx04.TheFacebook.com>
	<AANLkTikBDYm3J-ETtw75GuOZk+fMHWStb9bQpthdb+ur@mail.gmail.com>
	<5A76F6CE309AD049AAF9A039A39242820F18EE23@sc-mbx04.TheFacebook.com>
	<AANLkTincdTWT2QTqBdY_JWVTmMAZ2CDcxfN4i1uq8OyB@mail.gmail.com>
	<AANLkTimpQZ3arDGYSrbjJ=hu+svAKcC7zWwCGo+UZJNa@mail.gmail.com>
	<AANLkTikzqo=e+1grqmk+XdJzCkMwQmGjn-G=1Rx+bjer@mail.gmail.com>
	<AANLkTik3ROB4uxoeOe=0iEoAx51AOga7DP=tVmL-U+Yi@mail.gmail.com>
Date: Tue, 23 Nov 2010 16:23:19 -0800
Message-ID: <AANLkTi=57+oq-foxAwzAWivzpCm-JXvEttmN3Hevur81@mail.gmail.com>
Subject: Re: question about meta data query intensity
From: Jack Levin <magnito@gmail.com>
To: user@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

its taking some queries, but at 3% rate of what we expect to give it later.

-Jack

On Tue, Nov 23, 2010 at 4:19 PM, Stack <stack@duboce.net> wrote:
> To be clear, the cluster is not taking queries and .META. is still
> being hit at rate of 6k/second?
> St.Ack
>
> On Tue, Nov 23, 2010 at 4:15 PM, Jack Levin <magnito@gmail.com> wrote:
>> its requests=3D6204 ... but we have not been loading cluster with
>> queries at all. =A0I see that CPU is about 35% used vs other boxes at
>> user cpu of 10% or so... So its really CPU load that worries me than
>> the IO.
>>
>> -Jack
>>
>> On Tue, Nov 23, 2010 at 1:55 PM, Stack <stack@duboce.net> wrote:
>>> On Tue, Nov 23, 2010 at 11:06 AM, Jack Levin <magnito@gmail.com> wrote:
>>>> its REST, and generally no long lived clients, yes, caching of regions
>>>> helps however, we expect long tail hits that will be uncached, which
>>>> may stress out meta region, that being said, is it possible create
>>>> affinity and nail meta region into a beefy server or set of beefy
>>>> servers?
>>>>
>>>
>>> The REST server should be caching region locations for you.
>>>
>>> On the .META. side, since its accessed so frequently, it should be
>>> nailed into the block cache but if 1000 regions sitting beside that
>>> .META. there could be contention.
>>>
>>> There is also hbase.client.prefetch.limit, the number of region
>>> locations to fetch every time we do a lookup into .META. Currently its
>>> set to 10. =A0You could try setting this down to 1?
>>>
>>> What are you seeing for request rates and load on the .META. hosting
>>> regionserver?
>>>
>>> St.Ack
>>>
>>
>