hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michele (aka pirroh) Catasta (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (HADOOP-2366) Space in the value for dfs.data.dir can cause great problems
Date Sun, 14 Jun 2009 15:10:09 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719276#action_12719276
] 

Michele (aka pirroh) Catasta edited comment on HADOOP-2366 at 6/14/09 8:09 AM:
-------------------------------------------------------------------------------

Patch updated, now it's using {code:java}split("\\s*,\\s*"){code}

@tlipcon: Thanks for the comment! Gotta be honest, I wasn't using the regex because I thought
mine was the only way to let getStrings() behave as it was doing before regarding trailing
empty tokens. 
Actually, I took a look at the code which is using getStrings(), and throwing away the trailing
empty token should not break anything (while helps the users who leave a final comma without
any following path). Anyway, to make it behave as it was before, just add a -1 as the second
argument of split(). Hope it's OK now :-)

      was (Author: pirroh):
    Patch updated, now it's using split("\\s*,\\s*"). 

@tlipcon: Thanks for the comment! Gotta be honest, I wasn't using the regex because I thought
mine was the only way to let getStrings() behave as it was doing before regarding trailing
empty tokens. 
Actually, I took a look at the code which is using getStrings(), and throwing away the trailing
empty token should not break anything (while helps the users who leave a final comma without
any following path). Anyway, to make it behave as it was before, just add a -1 as the second
argument of split(). Hope it's OK now :-)
  
> Space in the value for dfs.data.dir can cause great problems
> ------------------------------------------------------------
>
>                 Key: HADOOP-2366
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2366
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: conf
>            Reporter: Ted Dunning
>            Assignee: Todd Lipcon
>         Attachments: HADOOP-2366.patch
>
>
> The following configuration causes problems:
> <property>
>   <name>dfs.data.dir</name>
>   <value>/mnt/hstore2/hdfs, /home/foo/dfs</value>  
>   <description>
>   Determines where on the local filesystem an DFS data node  should store its bl
> ocks.  If this is a comma-delimited  list of directories, then data will be stor
> ed in all named  directories, typically on different devices.  Directories that 
> do not exist are ignored.  
>   </description>
> </property>
> The problem is that the space after the comma causes the second directory for storage
to be " /home/foo/dfs" which is in a directory named <SPACE> which contains a sub-dir
named "home" in the hadoop datanodes default directory.  This will typically cause the user's
home partition to fill, but will be very hard for the user to understand since a directory
with a whitespace name is hard to understand.
> My proposed solution would be to trimLeft all path names from this and similar property
after splitting on comma.  This still allows spaces in file and directory names but avoids
this problem. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message