hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinay (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3734) TestFSEditLogLoader.testReplicationAdjusted() will hang if number of blocks are more than one
Date Wed, 01 Aug 2012 10:55:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426514#comment-13426514
] 

Vinay commented on HDFS-3734:
-----------------------------

Hi [~yanbo], thanks for taking a look at this.
{quote}According to the BlockPlacementPolicy, if the total DataNode numbers of the cluster
is less than the min replication, the real replication will equal to the smaller one(in this
case is the total DataNode numbers).{quote}

Here, Cluster have 2 DataNodes, and file is written with replication factor as 1. 
Later min replication is changed to 2, same as number of datanodes available and cluster is
restarted.

So problem is not with the replication. actual problem is namenode entering to safemode on
restart, if more than one blocks are there in the cluster and min replication is increased.
And in safemode, no replication will happen to satisfy the min replication.

Here no need of modifying the test scenario,
instead I would suggest to allow to replicate blocks in case of at least one replica is available.
After replication, min replication expectation will be satisfied automatically.
                
> TestFSEditLogLoader.testReplicationAdjusted() will hang if number of blocks are more
than one
> ---------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3734
>                 URL: https://issues.apache.org/jira/browse/HDFS-3734
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.1.0-alpha, 3.0.0
>            Reporter: Vinay
>
> TestFSEditLogLoader.testReplicationAdjusted() which was added in HDFS-2003 will fail
if number of blocks before cluster restart are more than one.
> Test Scenario:
> --------------
> 1. Write a file with min replication as 1 and replication factor as 1.
> 2. Change the min replication to 2 and restart the cluster.
> Expected: Min replication should be automatically reset on cluster restart by replicating
more blocks.
> Currently, if the number of blocks before restart is only one, then on restart NN will
not enter safemode, hence replication will happen and satisfies min replication factor.
> If initial blocks count is more than 1 which are having replication factor as 1, then
on restart NN will enter safemode and will never come out.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message