hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Created: (HDFS-1729) Improve metrics for measuring NN startup costs.
Date Mon, 07 Mar 2011 21:49:00 GMT
Improve metrics for measuring NN startup costs.

                 Key: HDFS-1729
                 URL: https://issues.apache.org/jira/browse/HDFS-1729
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: name-node
            Reporter: Owen O'Malley
            Assignee: Matt Foley
             Fix For: 0.20.100

Current logging and metrics are insufficient to diagnose latency problems in cluster startup.
1. better logs in both Datanode and Namenode for Initial Block Report processing, to help
distinguish between block
report processing problems and RPC/queuing problems;
2. new logs to measure cost of scanning all blocks for over/under/invalid replicas, which
occurs in Namenode just
before exiting safe mode;
3. new logs to measure cost of processing the under/invalid replica queues (created by the
above mentioned scan), which
occurs just after exiting safe mode, and is said to take 100% of CPU.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message