hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jack Levin <magn...@gmail.com>
Subject Re: question about meta data query intensity
Date Wed, 24 Nov 2010 00:23:19 GMT
its taking some queries, but at 3% rate of what we expect to give it later.

-Jack

On Tue, Nov 23, 2010 at 4:19 PM, Stack <stack@duboce.net> wrote:
> To be clear, the cluster is not taking queries and .META. is still
> being hit at rate of 6k/second?
> St.Ack
>
> On Tue, Nov 23, 2010 at 4:15 PM, Jack Levin <magnito@gmail.com> wrote:
>> its requests=6204 ... but we have not been loading cluster with
>> queries at all.  I see that CPU is about 35% used vs other boxes at
>> user cpu of 10% or so... So its really CPU load that worries me than
>> the IO.
>>
>> -Jack
>>
>> On Tue, Nov 23, 2010 at 1:55 PM, Stack <stack@duboce.net> wrote:
>>> On Tue, Nov 23, 2010 at 11:06 AM, Jack Levin <magnito@gmail.com> wrote:
>>>> its REST, and generally no long lived clients, yes, caching of regions
>>>> helps however, we expect long tail hits that will be uncached, which
>>>> may stress out meta region, that being said, is it possible create
>>>> affinity and nail meta region into a beefy server or set of beefy
>>>> servers?
>>>>
>>>
>>> The REST server should be caching region locations for you.
>>>
>>> On the .META. side, since its accessed so frequently, it should be
>>> nailed into the block cache but if 1000 regions sitting beside that
>>> .META. there could be contention.
>>>
>>> There is also hbase.client.prefetch.limit, the number of region
>>> locations to fetch every time we do a lookup into .META. Currently its
>>> set to 10.  You could try setting this down to 1?
>>>
>>> What are you seeing for request rates and load on the .META. hosting
>>> regionserver?
>>>
>>> St.Ack
>>>
>>
>

Mime
View raw message