hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5699) Run with > 1 WAL in HRegionServer
Date Tue, 12 Jun 2012 19:09:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13293853#comment-13293853

Todd Lipcon commented on HBASE-5699:

bq. I think we should wait for test result with HBASE-6116 before we invest more time in this.

HBASE-6116 seems like it would improve latency but hurt throughput -- on a typical gbit link,
the parallel writes would limit us to 50M/sec for 3 replicas, whereas pipelined writes could
give us 100M+.

The other main advantage of this JIRA is that the speed of the WAL is currently limited to
the minimum speed of the 3 disks chosen in the pipeline. Given that disks can be heavily loaded,
the probability of getting even a full disk's worth of throughput is low -- the likelihood
is that at least one of those disks is also being written to or read from at least another
client. So typically any single HDFS stream is limited to 35-40MB/sec in my experience.

Given that gbit is much faster than this, we can get better throughput by adding parallel
WALs, so as to stripe across disks and dynamically push writes to less-loaded disks.
> Run with > 1 WAL in HRegionServer
> ---------------------------------
>                 Key: HBASE-5699
>                 URL: https://issues.apache.org/jira/browse/HBASE-5699
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: binlijin
>            Assignee: Li Pi
>         Attachments: PerfHbase.txt

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message