hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8807) dfs.datanode.data.dir does not handle spaces between storageType and URI correctly
Date Wed, 22 Jul 2015 21:41:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637687#comment-14637687
] 

Allen Wittenauer commented on HDFS-8807:
----------------------------------------

Fundamentally, this is the problem:

bq. The [StorageType][URI] notation 

If we had a DTD for the XML, what would it look like? We've basically built an untestable
monster.  If my app is not Hadoop and want to parse or build this file, how is it supposed
to be coded?

> dfs.datanode.data.dir does not handle spaces between storageType and URI correctly
> ----------------------------------------------------------------------------------
>
>                 Key: HDFS-8807
>                 URL: https://issues.apache.org/jira/browse/HDFS-8807
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: HDFS
>    Affects Versions: 2.8.0
>            Reporter: Anu Engineer
>            Assignee: Anu Engineer
>             Fix For: 2.8.0
>
>
> if you add a space between the storage type and file URI then datanodes fail during startup.
> Here is an example of "mis-configration" that leads to datanode failure.
> {code}
> <property>
>     <name>dfs.datanode.data.dir</name>
>     <value>
>       [DISK] file://tmp/hadoop-aengineer/disk1/dfs/data
>     </value>
>   </property>
> {code}
> Here is the "fixed" version. Please *note* the lack of space between \[DISK\] and file
URI
> {code}
> <property>
>     <name>dfs.datanode.data.dir</name>
>     <value>
>       [DISK]file://tmp/hadoop-aengineer/disk1/dfs/data
>     </value>
>   </property>
> {code}
> we fail with a parsing error, here is the info from the datanode logs.
> {code}
> 2015-07-22 13:34:36,812 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Exception
in secureMain
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Illegal character in
scheme name at index 0:  file://tmp/hadoop-aengineer/disk1/dfs/data
>         at org.apache.hadoop.fs.Path.initialize(Path.java:204)
>         at org.apache.hadoop.fs.Path.<init>(Path.java:170)
>         at org.apache.hadoop.hdfs.server.datanode.StorageLocation.parse(StorageLocation.java:97)
>         at org.apache.hadoop.hdfs.server.datanode.DataNode.getStorageLocations(DataNode.java:2314)
>         at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2298)
>         at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2349)
>         at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2529)
>         at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2553)
> Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 0:
 file://tmp/hadoop-aengineer/disk1/dfs/data
>         at java.net.URI$Parser.fail(URI.java:2829)
>         at java.net.URI$Parser.checkChars(URI.java:3002)
>         at java.net.URI$Parser.checkChar(URI.java:3012)
>         at java.net.URI$Parser.parse(URI.java:3028)
>         at java.net.URI.<init>(URI.java:753)
>         at org.apache.hadoop.fs.Path.initialize(Path.java:201)
>         ... 7 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message