hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-11313) Segmented Block Reports
Date Wed, 11 Jan 2017 01:49:58 GMT
Konstantin Shvachko created HDFS-11313:

             Summary: Segmented Block Reports
                 Key: HDFS-11313
                 URL: https://issues.apache.org/jira/browse/HDFS-11313
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: datanode, namenode
    Affects Versions: 2.6.2
            Reporter: Konstantin Shvachko

Block reports from a single DataNode can be currently split into multiple RPCs each reporting
a single DataNode storage (disk). The reports are still large since disks are getting bigger.
Splitting blockReport RPCs into multiple smaller calls would improve NameNode performance
and overall HDFS stability.
This was discussed in multiple jiras. Here the approach is to let NameNode divide blockID
space into segments and then ask DataNodes to report replicas in a particular range of IDs.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message