Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Sat, 24 Sep 2011 11:37:26 +0000 (UTC)
From: "Steve Loughran (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <1419670056.9023.1316864246372.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <489103151.12345.1311782109715.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Work logged] (HDFS-2209) MiniDFS cluster improvements
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HDFS-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel\#worklog-{worklog.getId()} ]

Steve Loughran logged work on HDFS-2209:
----------------------------------------

                Author: Steve Loughran
            Created on: 24/Sep/11 11:36
            Start Date: 24/Sep/11 11:35
    Worklog Time Spent: 0.5h 
      Work Description: coding; test runs take the time

Issue Time Tracking
-------------------

            Worklog Id:     (was: 12034)
            Time Spent: 0.5h
    Remaining Estimate: 3h  (was: 1h)

> MiniDFS cluster improvements
> ----------------------------
>
>                 Key: HDFS-2209
>                 URL: https://issues.apache.org/jira/browse/HDFS-2209
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 0.20.203.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Minor
>   Original Estimate: 1h
>          Time Spent: 0.5h
>  Remaining Estimate: 3h
>
> I've been deploying MiniDFSCluster for some testing, and while using it/looking through the code I made some notes of where there are issues and improvement opportunities. This is mostly minor as its a test tool, but a risk of synchronization problems is there and does need addressing; the rest are all feature creep. 
> Field {{nameNode}} should be marked as volatile as the shutdown operation can be in a different thread than startup. Best of all, 
> add synchronized methods to set and get the field, as well as shutdown.
> The data dir is set from from System Properties.
> {code}
>     base_dir = new File(System.getProperty("test.build.data", "build/test/data"), "dfs/");
>     data_dir = new File(base_dir, "data");
> {code}
> This is done in {{formatDataNodeDirs()}} {{corruptBlockOnDataNode()}} and  the constructor.
> Improvement: have a test property in the conf file, and only read the system property if this is unset. This will enable
>  multiple MiniDFSClusters to come up in the same JVM, and handle shutdown/startup race conditions better, and avoid the
>  "java.io.IOException: Cannot lock storage build/test/data/dfs/name1. The directory is already locked." messages
> Messages should log to the commons logging and not {{System.err}} and {{System.out}}. This enables containers to catch and stream better, 
> and include more diagnostics such as timestamp and thread Id
> Class could benefit from a method to return the FS URI, rather than just the FS. This currently has to be worked around with some tricks involving a cached configuration
> {{waitActive()}} could get confused if "localhost" maps to an IPv6 address. Better to ask for 127.0.0.1 as the hostname; Junit
> test runs may need to be set up to force in IPv4 too.
> {{injectBlocks}} has a spelling error in the IOException, "SumulatedFSDataset" is the correct spelling

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira