hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: A general approach to finding issues/bottlenecks?
Date Mon, 26 Mar 2012 16:23:41 GMT
On Sun, Mar 25, 2012 at 8:08 PM, Juhani Connolly
<juhani_connolly@cyberagent.co.jp> wrote:
> Still stuck with a severely under-performing cluster where with every
> version change the problem seems to change.
>

Understood.  Going from 0.90 to 0.92, there were nearly 1k changes made.

> I'm curious to know if anyone has devised a systematic approach to finding
> issues in their configurations.
>

Do these help: http://hbase.apache.org/book.html#trouble.casestudy and
http://gbif.blogspot.com/2012/03/hbase-performance-evaluation-continued.html


> What tools do people look to, and what are they most suspicious of
> initially?

Ganglia or opentsdb w/ a big of log splunking and some thread dumping
is usually how a few of us proceed.  At extreme, added debug logging.

>  I'm concerned I may be overlooking something major as after going
> through hardware saturation checks and service logs, returning everything to
> the simplest configuration possible and trying various other configurations,
> I'm now mainly diving around the source code and stack dumps from the
> master/region servers/datanodes/namenode and making no progress, and I don't
> feel this is the right approach, but can't see any other alternative.
>

Let us try and give you feedback on the other thread you started.

St.Ack

Mime
View raw message