hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Viral Bajaria <viral.baja...@gmail.com>
Subject Re: max regionserver handler count
Date Tue, 30 Apr 2013 06:10:56 GMT
Thanks for getting back, Ted. I totally understand other priorities and
will wait for some feedback. I am adding some more info to this post to
allow better diagnosing of performance.

I hit my region servers with a lot of GET requests (~20K per second per
regionserver) using asynchbase in my test environment, the storage pattern
is very similar to OpenTSDB though with a lot more columns. Each row is
around 45-50 bytes long. The regionservers have a lot of RAM available to
them (48 out of 60 GB) and they are not sharing resources with anyone else,
so memory is not under pressure. The total # of rows in the table is around
100M and growing (there is a put process too)

GETs take over 15s for 16K rows, and I don't see any operationTooSlow logs
in the regionserver logs either. PUTs take around 1s for 16K rows (deferred
log flush is enabled though).

I looked at the RPC stats and it seems the RPC threads were always doing
something and I assumed my requests were waiting on handlers and so thought
of experimenting by increasing number of handlers. But as mentioned in my
thread, going above 10K kills my regionserver.

Thanks,
Viral

On Mon, Apr 29, 2013 at 9:43 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Viral:
> I am currently dealing with some high priority bugs so I didn't have time
> to look deeper into your case.
>
> My feeling is that raising max regionserver handler count shouldn't be the
> key to boosting performance.
>
> Cheers
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message