hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lin Yiqun (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-10227) BlockManager should decrease blocksScheduled count for timeout replication
Date Tue, 29 Mar 2016 08:22:25 GMT

     [ https://issues.apache.org/jira/browse/HDFS-10227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lin Yiqun updated HDFS-10227:
-----------------------------
    Status: Patch Available  (was: Open)

Attach a initial patch, kindly review.

> BlockManager should decrease blocksScheduled count for timeout replication
> --------------------------------------------------------------------------
>
>                 Key: HDFS-10227
>                 URL: https://issues.apache.org/jira/browse/HDFS-10227
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Lin Yiqun
>            Assignee: Lin Yiqun
>
> In {{BlockManager#processPendingReplications}}, it suggests that we could invoke decBlocksScheduled()
for timeout replication.
> {code}
>   /**
>    * If there were any replication requests that timed out, reap them
>    * and put them back into the neededReplication queue
>    */
>   private void processPendingReplications() {
>     BlockInfo[] timedOutItems = pendingReplications.getTimedOutBlocks();
>     if (timedOutItems != null) {
>       namesystem.writeLock();
>       try {
>         for (int i = 0; i < timedOutItems.length; i++) {
>           /*
>            * Use the blockinfo from the blocksmap to be certain we're working
>            * with the most up-to-date block information (e.g. genstamp).
>            */
>           BlockInfo bi = blocksMap.getStoredBlock(timedOutItems[i]);
>           if (bi == null) {
>             continue;
>           }
>           NumberReplicas num = countNodes(timedOutItems[i]);
>           if (isNeededReconstruction(bi, num.liveReplicas())) {
>             neededReconstruction.add(bi, num.liveReplicas(),
>                 num.readOnlyReplicas(), num.decommissionedAndDecommissioning(),
>                 getReplication(bi));
>           }
>         }
>       } finally {
>         namesystem.writeUnlock();
>       }
>       /* If we know the target datanodes where the replication timedout,
>        * we could invoke decBlocksScheduled() on it. Its ok for now.
>        */
>     }
>   }
> {code}
> The comment seems right. After the timeout items are added to {{neededReplications}},
the blocksScheduled count will be repeated increased when these timeout replications removed
from {{neededReplications}} to {{pendingReplications}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message