hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-6116) Allow parallel HDFS writes for HLogs.
Date Sat, 23 Jun 2012 23:34:43 GMT

     [ https://issues.apache.org/jira/browse/HBASE-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Purtell updated HBASE-6116:
----------------------------------

    Attachment: pipelined-vs-parallel-comparison.zip

Attached is a comparison of pipelined vs. parallel sync differences on two identical (but
different) 5 slave EC2 clusters. I modified HBase to use a new histogram metric for recording
HLog sync latency and then ran a write dominant workload on each cluster using LoadTestTool
for 60 minutes and captured RegionServer metrics at one second intervals.

The first tab of the spreadsheet describes the experiment parameters. The second shows mean,
99th percentile, and standard deviation for pipelined syncs as reported. The third shows mean,
99th percentile, and standard deviation for parallel syncs as reported. The fourth has some
simple graphs I threw together for illustration. The remaining tabs contain the detail of
the captured metrics for each host.
                
> Allow parallel HDFS writes for HLogs.
> -------------------------------------
>
>                 Key: HBASE-6116
>                 URL: https://issues.apache.org/jira/browse/HBASE-6116
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>         Attachments: 6116-v1.txt, pipelined-vs-parallel-comparison.zip
>
>
> In HDFS-1783 I adapted Dhrubas changes to be used in Hadoop trunk.
> This issue will include the necessary reflection changes to optionally enable this for
the WALs in HBase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message