hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Phulari (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HDFS-555) A few improvements to DataNodeCluster - HADOOP-5556
Date Fri, 21 Aug 2009 00:50:14 GMT

     [ https://issues.apache.org/jira/browse/HDFS-555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Ravi Phulari updated HDFS-555:

    Attachment: HDFS-555-v1.patch

     [exec] There appear to be 148 release audit warnings before the patch and 148 release
audit warnings after applying the patch.
     [exec] +1 overall.  
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec]     +1 tests included.  The patch appears to include 12 new or modified tests.
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec]     +1 release audit.  The applied patch does not increase the total number of
release audit warnings.
     [exec] ======================================================================
     [exec] ======================================================================
     [exec]     Finished build.
     [exec] ======================================================================
     [exec] ======================================================================

Total time: 16 minutes 22 seconds

> A few improvements to DataNodeCluster - HADOOP-5556 
> ----------------------------------------------------
>                 Key: HDFS-555
>                 URL: https://issues.apache.org/jira/browse/HDFS-555
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 0.21.0
>            Reporter: Ravi Phulari
>            Assignee: Hairong Kuang
>             Fix For: 0.21.0
>         Attachments: HDFS-555-v1.patch, HDFS-555.patch
> Opening jira to address HDFS code changes made in HADOOP-5556.
> DataNodeCluster is a great tool to simulate a large scale DFS cluster using a small set
of machines. A few suggestions to improve this tool:
>    1. DataNodeCluster uses MiniDFSCluster#startDataNode to start multiple instances of
DataNode on one machine. MiniDFSCluster sets DataNode's address to be We should
allow to set its address to so DataNodes in different machines could communicate.
>    2. Currently the size of the blocks injected to DataNode and created in CreatedEditsLog
is hardcoded as 10. It would be more convenient if this could be configurable. Also we need
to make sure that both use the same block size.
>    3. If the replication factor of blocks is larger than 1, currently a DataNode in DataNodeCluster
will be injected blocks multiple times and therefore it sends block reports to NameNode multiple
times. Initial block reports contain only a portion of its blocks and therefore may cause
unnecessary block replications. It would be cleaner if only one block report with all its
blocks is sent.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message