Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 6 Apr 2011 18:13:06 +0000 (UTC)
From: "Hadoop QA (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <989941559.38395.1302113586407.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1826418763.13691.1299240756959.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HDFS-1725) Set storage directories only at
 FSImage construction (was Cleanup FSImage construction)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016470#comment-13016470 ] 

Hadoop QA commented on HDFS-1725:
---------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12475605/HDFS-1725.patch
  against trunk revision 1087900.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 10 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    -1 core tests.  The patch failed these core unit tests:
                  org.apache.hadoop.hdfs.TestFileConcurrentReader

    -1 contrib tests.  The patch failed contrib unit tests.

    +1 system test framework.  The patch passed system test framework compile.

Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/320//testReport/
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/320//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/320//console

This message is automatically generated.

> Set storage directories only at FSImage construction (was Cleanup FSImage construction)
> ---------------------------------------------------------------------------------------
>
>                 Key: HDFS-1725
>                 URL: https://issues.apache.org/jira/browse/HDFS-1725
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: Edit log branch (HDFS-1073)
>
>         Attachments: HDFS-1725-review-guide.pdf, HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.patch
>
>
> HDFS-1580 proposes extending FSEditLog to allow it to use editlog streams which are not backed by StorageDirectory. Currently, to set the the directories used for edits, NNStorage#setStorageDirectory is called with a list of URIs as the second argument. NNStorage takes this list or URIs, takes all file:/// URIs and adds them to its StorageDirectory list. Then, when opened, FSEditLog will request a list of StorageDirectories from NNStorage and create a list of EditLogOutputStreams based on these.
> This approach cannot work with HDFS-1580. NNStorage exists solely to deal with filesystem based storage. As such, only StorageDirectories can be retrieved from NNStorage by FSEditLog. So, FSEditLog should get the URI from some place other than NNStorage. This presents a further problem, in that, NNStorage#setStorageDirectories is the current way of setting the URIs for images and edits. This call can happen at any time, so the directories in NNStorage can change at any time. If FSEditLog is to get its URIs from elsewhere, this opens up the risk of the filesystem directories in NNStorage and filesystem URIs being out of sync.
> A solution to this is to stipulate that the URIs for NNStorage are set only once, on construction. All proper uses of NNStorage#setStorageDirectories are being called just after construction of the image in any case. All other cases are using NNStorage#setStorageDirectories not to set the storage directories, but for the side effects of this call. This guide explains these other cases.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira