hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "LEI Xiaofeng" <le...@ihep.ac.cn>
Subject Re: Re: Re: hbase read performance tuning failed
Date Wed, 08 Jan 2014 01:54:48 GMT
In addition, I am using c++ to access HBase. Will it make any difference compared to using
Java? 


> -----原始邮件-----
> 发件人: "LEI Xiaofeng" <leixf@ihep.ac.cn>
> 发送时间: 2014年1月8日 星期三
> 收件人: user@hbase.apache.org, doug.meil@explorysmedical.com
> 抄送: 
> 主题: Re: Re: hbase read performance tuning failed
> 
> I tried to set "BLOCKSIZE" to 0 when created table to tune the scan performance, but
it did not work. Another thing is what do you mean by "turing HBase checksumming on"? I do
not know how to do it. Could you give mw some detailed information?
> 
> Thanks
> 
> 
> > -----原始邮件-----
> > 发件人: "Doug Meil" <doug.meil@explorysmedical.com>
> > 发送时间: 2014年1月8日 星期三
> > 收件人: "user@hbase.apache.org" <user@hbase.apache.org>, "lars hofhansl"
<larsh@apache.org>
> > 抄送: 
> > 主题: Re: hbase read performance tuning failed
> > 
> > 
> > In addition to what Lars just said about the blocksize, this is a similar
> > question to another one that somebody asked, and it's always good to make
> > sure that you understand where your data is. As a sanity check, make sure
> > it's not all on one or two RSs (look at the hbase web pages or with tools
> > like Hannibal).
> > 
> > 
> > Also, you definitely want to to turn HBase checksumming on - and when you
> > do so you'll need to re-create the HFiles (e.g., you can't just change the
> > config and bounce the HBase cluster).  That's a significant reduction in
> > I/O.
> > 
> > Likewise, if you are doing a full-scan, make sure that you select only the
> > attributes you need...
> > 
> > See this for more:  http://hbase.apache.org/book.html#perf.reading
> > 
> > 
> > 
> > 
> > 
> > On 1/7/14 1:24 PM, "lars hofhansl" <larsh@apache.org> wrote:
> > 
> > >If increasing hbase.client.scanner.caching makes no difference you have
> > >another issue.
> > >How many rows do you expect your to return?
> > >
> > >On contemporary hardware I manage to scan a few million KeyValues (i.e.
> > >columns) per second and per CPU core.
> > >Note that for scan performance you want to increase the BLOCKSIZE.
> > >
> > >
> > >-- Lars
> > >
> > >
> > >
> > >________________________________
> > > From: LEI Xiaofeng <leixf@ihep.ac.cn>
> > >To: user@hbase.apache.org
> > >Sent: Monday, January 6, 2014 11:06 PM
> > >Subject: hbase read performance tuning failed
> > > 
> > >
> > >Hi,
> > >I am running hbase-0.94.6-cdh4.5.0 and set up a cluster of 5 nodes. The
> > >random read performance is ok, but the scan performance is poor.
> > >I tried to increase "hbase.client.scanner.caching" to 100 to promote the
> > >scan performance but it made  no difference. And when I tried to make
> > >smaller blocks by setting "BLOCKSIZE" when created tables to get better
> > >random read performance it made no difference too.
> > >So, I am wondering if anyone could give some advice to solve this problem.
> > >
> > >
> > >
> > >Thanks
> > 
> 


Mime
View raw message