hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: improving random read performance
Date Sat, 21 Aug 2010 00:28:40 GMT
Yeah.  Thats downside to using HADOOP_CLASSPATH.  If its contents
change, processes that depend on it need relaunch.

If you don't want to change per hbase version, bundle hbase into you
job jar? From http://hbase.apache.org/docs/r0.20.6/api/org/apache/hadoop/hbase/mapreduce/package-summary.html#classpath

"Another possibility, if for example you do not have access to
hadoop-env.sh or are unable to restart the hadoop cluster, is bundling
the hbase jars into a mapreduce job jar adding it and its dependencies
under the job jar lib/ directory and the hbase conf into the job jars
top-level directory."

You do not need to restart hdfs.  You'd just need to restart
tasktrackers since these are the doohickeys running the MR jobs that
depend on hbase.


On Fri, Aug 20, 2010 at 5:14 PM, Sharma, Avani <agsharma@ebay.com> wrote:
> Thanks. After I upgrade to 0.20.6, I have to go and change the HADOOP_CLASSPATH in hadoop-env.sh
which points to hbase jars, since the names of the jars have changed : /usr/local/hbase-0.20.6/hbase-0.20.6.jar.
> Will I have to restart hdfs after the change - this would mean upgrade required hbase
restart and hdfs restart ?
> I am guessing there might be a simpler way to do this.
> -Avani
> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
> Sent: Thursday, August 19, 2010 8:53 PM
> To: user@hbase.apache.org
> Subject: Re: improving random read performance
> On Thu, Aug 19, 2010 at 5:53 PM, Sharma, Avani <agsharma@ebay.com> wrote:
>> I am currently running under Hbasev0.20.3. I increased the block cache to .4 from
.2. The heap size is 2GB.
>> The default regionserver handler count is 25 in hbase-default.xml. Will try LZO compression.
> Up your handlers to 100.  Up your block cache even more... .5 or .6
> even if you mostly random reading.  Can you give it more RAM than 4G?
>> What are other performance tunings I can do ?
>> In particular, will applying HBASE-2180 or upgrading to a newer version help ?
> Yes, hbase-2180 will make a difference.  You should upgrade to 0.20.6
> to catch 2180 and other fixes.
>> Another thing I notice is that the performance via stargate vs Java api is comparable
for random reads.I thought stargate would have some latency. Is this expected?
> I'd expect come tax going via REST.
> St.Ack

View raw message