hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Meil <doug.m...@explorysmedical.com>
Subject Re: So Bad Random Read Performance
Date Sat, 30 Jul 2011 13:31:59 GMT

For background information on HDFS-Hbase performance, reviewing the
umbrella ticket is a good place to start...

https://issues.apache.org/jira/browse/HDFS-1599


But if I am reading this correctly...

>  I write 100M data in Hbase,only 1M key/value,so all keys are in only
*one*
> Store,and only 2 HFiles in that Store


...  all your data wound up in one Store, thus it's all on one DataNode.



See the Hbase book on pre-splitting tables if you haven't done that
already...

http://hbase.apache.org/book.html#performance



On 7/30/11 7:29 AM, "李宏展" <lihongzhan135@gmail.com> wrote:

>which property in configuration affects the concerrency?
>
>
>在 2011年7月30日 下午6:21,Ryan Rawson <ryanobjc@gmail.com>写道:
>
>> What is the level of concurrency?  I find that HDFS gets worse
>> performing with more concurrent read threads.
>>
>> -ryan
>>
>> 2011/7/30 seven garfee <garfee.seven@gmail.com>:
>> > hi,all
>> >  I set up a cluster on 4 machine.(1 HMaster,4RegionServer).
>> >  Each Machine has a 16G mem, one 2T Sata disk,CentOS 5.3,XFS .
>> >  I write 100M data in Hbase,only 1M key/value,so all keys are in only
>> *one*
>> > Store,and only 2 HFiles in that Store.(I used Large Data Set,but
>>random
>> read
>> > performance is bad,so i try 100M)
>> >  Then I test random read with existing keys with only 1 client. But I
>>got
>> a
>> > Bad Performance,About >100ms latency per read.
>> >
>> > Why did I get so long latency?
>> >
>> >  I dig into the metric log,and found that,
>> >  in metrics_rpc.log:
>> >      get_avg_time=146, get_num_ops=22
>> >      get_avg_time=94, get_num_ops=127
>> >      get_avg_time=113, get_num_ops=182
>> >      .......
>> >
>> > and in metrics_hbase.log:
>> >      fsReadLatency_avg_time=140, fsReadLatency_num_ops=31
>> >      fsReadLatency_avg_time=93, fsReadLatency_num_ops=137
>> >      fsReadLatency_avg_time=109, fsReadLatency_num_ops=188
>> >      .......
>> >
>> > Is that say the fsReadLatency causes the random get latency so long?
>> >
>> > But i didn't tuning hdfs almost,except some ips/ports and
>>max.xcievers,
>> > Should I tuning for hdfs?And How?
>> >
>>


Mime
View raw message