Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <2101865180.1245198487441.JavaMail.jira@brutus>
Date: Tue, 16 Jun 2009 17:28:07 -0700 (PDT)
From: "Michele (aka pirroh) Catasta (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Issue Comment Edited: (HADOOP-2366) Space in the value for
 dfs.data.dir can cause great problems
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720438#action_12720438 ] 

Michele (aka pirroh) Catasta edited comment on HADOOP-2366 at 6/16/09 5:27 PM:
-------------------------------------------------------------------------------

I attached a separate patch which exposes a getTrimmedStrings() method and modifies only the behavior of the Datanode.
When the property value is empty, the behavior of getTrimmedStrings is by purpose different from getStrings(). I'd rather pass an empty array which is handled correctly by Datanode.instance(), than a null which causes NPE. 
Just let me know if you want me to fix this behavior and make it consistent with the other methods, mine was just a proposal.

Todd: I run a whole ant clean test cycle with the other patch, and it fails anyway. I gave a shallow look with greps on getStrings() when I was attaching the first version, and I had your same impression. Actually, after that fix you suggested, I was expecting the tests to run smoothly, but probably there's something missing still. I'll leave the patch attached, in case it could be useful.

Tsz Wo: thanks for the hints. run-test-core was successful, hope will be the same with the Hudson build.

      was (Author: pirroh):
    I attached a separate patch which exposes a getTrimmedStrings() method and modifies only the behavior of the Datanode.
When the property value is empty, the behavior of getTrimmedStrings is by purpose different from getStrings(). I'd rather pass an empty array which is handled correctly by Datanode.instance(), than a null which causes NPE. 
Just let me know if you want me to fix this behavior and make it consistent with the other methods, mine was just a purpose.

Todd: I run a whole ant clean test cycle with the other patch, and it fails anyway. I gave a shallow look with greps on getStrings() when I was attaching the first version, and I had your same impression. Actually, after that fix you suggested, I was expecting smooth tests, but probably there's something missing still. I'll leave the patch attached, in case it could be useful.

Tsz Wo: thanks for the hints. run-test-core was successful, hope will be the same with the Hudson build.
  
> Space in the value for dfs.data.dir can cause great problems
> ------------------------------------------------------------
>
>                 Key: HADOOP-2366
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2366
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: conf
>            Reporter: Ted Dunning
>            Assignee: Todd Lipcon
>         Attachments: HADOOP-2366-trimmed.patch, HADOOP-2366.patch
>
>
> The following configuration causes problems:
> <property>
>   <name>dfs.data.dir</name>
>   <value>/mnt/hstore2/hdfs, /home/foo/dfs</value>  
>   <description>
>   Determines where on the local filesystem an DFS data node  should store its bl
> ocks.  If this is a comma-delimited  list of directories, then data will be stor
> ed in all named  directories, typically on different devices.  Directories that 
> do not exist are ignored.  
>   </description>
> </property>
> The problem is that the space after the comma causes the second directory for storage to be " /home/foo/dfs" which is in a directory named <SPACE> which contains a sub-dir named "home" in the hadoop datanodes default directory.  This will typically cause the user's home partition to fill, but will be very hard for the user to understand since a directory with a whitespace name is hard to understand.
> My proposed solution would be to trimLeft all path names from this and similar property after splitting on comma.  This still allows spaces in file and directory names but avoids this problem. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.