hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Foley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2126) Improve Namenode startup time [umbrella task]
Date Tue, 05 Jul 2011 20:18:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13060108#comment-13060108
] 

Matt Foley commented on HDFS-2126:
----------------------------------

HDFS-1295 decreased IBR processing time by more than 1/3.  On the server-class machines in
our test cluster, it enabled processing speeds of about 450 Block Reports per minute.  Testing
also established that even in large clusters it is still best to start all the Datanodes together,
and not try to stagger them manually; the Namenode does not have difficulty enqueuing the
flow.  

HDFS-1541 and HDFS-1767 help prevent stochastic errors in IBR processing.  It is hard to measure
the exact improvement, but they eliminate the two most frequently seen causes of re-processing
of nodes.

> Improve Namenode startup time [umbrella task]
> ---------------------------------------------
>
>                 Key: HDFS-2126
>                 URL: https://issues.apache.org/jira/browse/HDFS-2126
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 0.20.2
>            Reporter: Matt Foley
>
> This is an umbrella task to group the improvements in Namenode startup latency made over
the last few months, and track remaining ideas.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message