hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Phulari (JIRA)" <j...@apache.org>
Subject [jira] Created: (HDFS-811) Additional tests(Unit tests & Functional tests) for HDFS-457.
Date Fri, 04 Dec 2009 23:11:20 GMT
Additional tests(Unit tests & Functional tests)  for HDFS-457.

                 Key: HDFS-811
                 URL: https://issues.apache.org/jira/browse/HDFS-811
             Project: Hadoop HDFS
          Issue Type: Test
          Components: test
    Affects Versions: 0.22.0
            Reporter: Ravi Phulari
            Priority: Minor
             Fix For: 0.22.0

 HDFS-457 introduced a improvement which allows  datanode to continue if a volume for replica
storage fails. Previously a datanode resigned if any volume failed. 

Description of HDFS-457
Current implementation shuts DataNode down completely when one of the configured volumes of
the storage fails.
This is rather wasteful behavior because it decreases utilization (good storage becomes unavailable)
and imposes extra load on the system (replication of the blocks from the good volumes). These
problems will become even more prominent when we move to mixed (heterogeneous) clusters with
many more volumes per Data Node.

I suggest following additional tests for this improvement. 

#1 Test successive  volume failures ( Minimum 4 volumes )
#2 Test if each volume failure reports reduction in available DFS space and remaining space.
#3 Test if failure of all volumes on a data nodes leads to the data node failure.
#4 Test if correcting failed storage disk brings updates and increments available DFS space.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message