Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Tue, 13 Dec 2011 01:31:31 +0000 (UTC)
From: "Uma Maheswara Rao G (Commented) (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <1933316121.3901.1323739891974.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1697525269.9764.1300388549618.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HDFS-1765) Block Replication should respect
 under-replication block priority
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168042#comment-13168042 ] 

Uma Maheswara Rao G commented on HDFS-1765:
-------------------------------------------

Thanks a lot Eli, for the reviews!

Updated the patch. Functionally same as last patch with addressing the comments from Eli.

TestReplicationPolicy:

    * rename chooseUnderReplicatedBlocks to chosenBlocks
    Done
    * The new testChooseUnderReplicatedBlocks looks great
    Thanks Eli. :-)

UnderReplicatedBlocks:

    * Can remove the " // replicate blocks" comment since we're not actually replicating blocks here.
    
     Actually this is in BlockManager and was a old comment. Just removed it.

    * The comment in chooseUnderReplicatedBlocks references priorityVsReplIndex instead of priorityToReplIdx
      couldn't figure out in last patch. Done. Thanks for catching it.
    * "is no blocks" -> "are no blocks"
      " "full fill" -> "fulfill" - done.
      

Thanks
Uma

                
> Block Replication should respect under-replication block priority
> -----------------------------------------------------------------
>
>                 Key: HDFS-1765
>                 URL: https://issues.apache.org/jira/browse/HDFS-1765
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 0.23.0
>            Reporter: Hairong Kuang
>            Assignee: Uma Maheswara Rao G
>             Fix For: 0.24.0
>
>         Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.pdf, underReplicatedQueue.pdf
>
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Currently under-replicated blocks are assigned different priorities depending on how many replicas a block has. However the replication monitor works on blocks in a round-robin fashion. So the newly added high priority blocks won't get replicated until all low-priority blocks are done. One example is that on decommissioning datanode WebUI we often observe that "blocks with only decommissioning replicas" do not get scheduled to replicate before other blocks, so risking data availability if the node is shutdown for repair before decommission completes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira