hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1623) High Availability Framework for HDFS NN
Date Fri, 17 Feb 2012 08:22:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210132#comment-13210132

Konstantin Shvachko commented on HDFS-1623:

I'd recommend 2 series of DFSIO consisting of -write -read and -append in each series and
-fileSize = 1 to 10GB. Pick one value for all runs. We want files with multiple blocks.
Series 1. -nrFiles = 95
Series 2. -nrFiles = 95*4
I chose 95, which is a bit less than # of nodes (100).
And 95*4 - intended to spin 4 drives on most of the nodes if you have 4 drives or more.
Don't forget to turn off speculation.
And please watch std deviation in the results.
In my experience Throughput values don't make sense if std deviation is high.
> High Availability Framework for HDFS NN
> ---------------------------------------
>                 Key: HDFS-1623
>                 URL: https://issues.apache.org/jira/browse/HDFS-1623
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Sanjay Radia
>            Assignee: Sanjay Radia
>         Attachments: HA-tests.pdf, HDFS-High-Availability.pdf, NameNode HA_v2.pdf, NameNode
HA_v2_1.pdf, Namenode HA Framework.pdf, ha-testplan.pdf, ha-testplan.tex

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message