Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of saint.ack@gmail.com designates
 74.125.82.169 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=wS/SewkOHhuX2Bmq7sQz/FKHRcP+YB7p2HBmt0nitzL7zbfF8SN9V/f6M9d5U6Pqjq
         xPfmdd4HviL6x9yVkP7cuFi0OZIDPcRZXmjMemTuGKtgQYpgWI9CZaQmYCSQ0uqnolMz
         /Zo9vY0R5WXFr/soHleO157xqUsN5MDPcB1co=
MIME-Version: 1.0
Sender: saint.ack@gmail.com
In-Reply-To: 
 <DE357E04199C8A4782DA2C8130FD64CA01132D723C@RHV-MEXMS-002.corp.ebay.com>
References: 
 <DE357E04199C8A4782DA2C8130FD64CA01132D6DB0@RHV-MEXMS-002.corp.ebay.com>
	<AANLkTimoHRWHTsxhyPKA0O85XNNcy8+UCzaZoNKpSDyn@mail.gmail.com>
	<DE357E04199C8A4782DA2C8130FD64CA01132D723C@RHV-MEXMS-002.corp.ebay.com>
Date: Fri, 20 Aug 2010 17:28:40 -0700
Message-ID: <AANLkTinc=CfeWX2=0jEZxpx5_=xoXvbsrWdqaPLKN-j+@mail.gmail.com>
Subject: Re: improving random read performance
From: Stack <stack@duboce.net>
To: user@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Yeah.  Thats downside to using HADOOP_CLASSPATH.  If its contents
change, processes that depend on it need relaunch.

If you don't want to change per hbase version, bundle hbase into you
job jar? From http://hbase.apache.org/docs/r0.20.6/api/org/apache/hadoop/hb=
ase/mapreduce/package-summary.html#classpath

"Another possibility, if for example you do not have access to
hadoop-env.sh or are unable to restart the hadoop cluster, is bundling
the hbase jars into a mapreduce job jar adding it and its dependencies
under the job jar lib/ directory and the hbase conf into the job jars
top-level directory."

You do not need to restart hdfs.  You'd just need to restart
tasktrackers since these are the doohickeys running the MR jobs that
depend on hbase.

St.Ack

On Fri, Aug 20, 2010 at 5:14 PM, Sharma, Avani <agsharma@ebay.com> wrote:
> Thanks. After I upgrade to 0.20.6, I have to go and change the HADOOP_CLA=
SSPATH in hadoop-env.sh which points to hbase jars, since the names of the =
jars have changed : /usr/local/hbase-0.20.6/hbase-0.20.6.jar.
>
> Will I have to restart hdfs after the change - this would mean upgrade re=
quired hbase restart and hdfs restart ?
> I am guessing there might be a simpler way to do this.
>
> -Avani
>
> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
> Sent: Thursday, August 19, 2010 8:53 PM
> To: user@hbase.apache.org
> Subject: Re: improving random read performance
>
> On Thu, Aug 19, 2010 at 5:53 PM, Sharma, Avani <agsharma@ebay.com> wrote:
>>
>> I am currently running under Hbasev0.20.3. I increased the block cache t=
o .4 from .2. The heap size is 2GB.
>> The default regionserver handler count is 25 in hbase-default.xml. Will =
try LZO compression.
>>
>
> Up your handlers to 100. =A0Up your block cache even more... .5 or .6
> even if you mostly random reading. =A0Can you give it more RAM than 4G?
>
>
>> What are other performance tunings I can do ?
>> In particular, will applying HBASE-2180 or upgrading to a newer version =
help ?
>>
>
> Yes, hbase-2180 will make a difference. =A0You should upgrade to 0.20.6
> to catch 2180 and other fixes.
>
>> Another thing I notice is that the performance via stargate vs Java api =
is comparable for random reads.I thought stargate would have some latency. =
Is this expected?
>
> I'd expect come tax going via REST.
>
> St.Ack
>