hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajith S (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8574) When block count exceeds dfs.blockreport.split.threshold, the block report are sent in one per message
Date Thu, 11 Jun 2015 03:07:00 GMT
Ajith S created HDFS-8574:
-----------------------------

             Summary: When block count exceeds dfs.blockreport.split.threshold, the block
report are sent in one per message
                 Key: HDFS-8574
                 URL: https://issues.apache.org/jira/browse/HDFS-8574
             Project: Hadoop HDFS
          Issue Type: Bug
    Affects Versions: 2.7.0
            Reporter: Ajith S
            Assignee: Ajith S


This piece of code in {{org.apache.hadoop.hdfs.server.datanode.BPServiceActor.blockReport()}}

{code}
// Send one block report per message.
        for (int r = 0; r < reports.length; r++) {
          StorageBlockReport singleReport[] = { reports[r] };
          DatanodeCommand cmd = bpNamenode.blockReport(
              bpRegistration, bpos.getBlockPoolId(), singleReport,
              new BlockReportContext(reports.length, r, reportId));
          numReportsSent++;
          numRPCs++;
          if (cmd != null) {
            cmds.add(cmd);
          }
{code}

is creating many cmds in case the block count exceeds the {{dfs.blockreport.split.threshold}}
limit. A better way for this will be spliting the block reports in equal number of buckets
of size {{dfs.blockreport.split.threshold}} therefore reducing the number of RPCs in block
reporting



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message