hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lin Yiqun (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-10227) BlockManager should decrease blocksScheduled count for timeout replication
Date Tue, 29 Mar 2016 06:28:25 GMT
Lin Yiqun created HDFS-10227:

             Summary: BlockManager should decrease blocksScheduled count for timeout replication
                 Key: HDFS-10227
                 URL: https://issues.apache.org/jira/browse/HDFS-10227
             Project: Hadoop HDFS
          Issue Type: Bug
            Reporter: Lin Yiqun
            Assignee: Lin Yiqun

In {{BlockManager#processPendingReplications}}, it suggests that we could invoke decBlocksScheduled()
for timeout replication.
   * If there were any replication requests that timed out, reap them
   * and put them back into the neededReplication queue
  private void processPendingReplications() {
    BlockInfo[] timedOutItems = pendingReplications.getTimedOutBlocks();
    if (timedOutItems != null) {
      try {
        for (int i = 0; i < timedOutItems.length; i++) {
           * Use the blockinfo from the blocksmap to be certain we're working
           * with the most up-to-date block information (e.g. genstamp).
          BlockInfo bi = blocksMap.getStoredBlock(timedOutItems[i]);
          if (bi == null) {
          NumberReplicas num = countNodes(timedOutItems[i]);
          if (isNeededReconstruction(bi, num.liveReplicas())) {
            neededReconstruction.add(bi, num.liveReplicas(),
                num.readOnlyReplicas(), num.decommissionedAndDecommissioning(),
      } finally {
      /* If we know the target datanodes where the replication timedout,
       * we could invoke decBlocksScheduled() on it. Its ok for now.
The comment seems right. After the timeout items are added to {{neededReplications}}, the
blocksScheduled count will be repeated increased when these timeout replications removed from
{{neededReplications}} to {{pendingReplications}}.

This message was sent by Atlassian JIRA

View raw message