hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5496) "du" fails on Cygwin
Date Mon, 16 Mar 2009 19:47:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682427#action_12682427

Doug Cutting commented on HADOOP-5496:

Does the TestDU unit test fail on Windows without this?  If not, can we easily modify it so
that it does?

> "du" fails on Cygwin
> --------------------
>                 Key: HADOOP-5496
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5496
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.19.1
>         Environment: Windows + Cygwin
>            Reporter: Kohsuke Kawaguchi
>            Assignee: Kohsuke Kawaguchi
>         Attachments: HADOOP-5486
> When I try to run a datanode on Windows, I get the following exception:
> {noformat}
> java.io.IOException: Expecting a line not the end of stream
> 	at org.apache.hadoop.fs.DU.parseExecResult(DU.java:181)
> 	at org.apache.hadoop.util.Shell.runCommand(Shell.java:179)
> 	at org.apache.hadoop.util.Shell.run(Shell.java:134)
> 	at org.apache.hadoop.fs.DU.<init>(DU.java:53)
> 	at org.apache.hadoop.fs.DU.<init>(DU.java:63)
> 	at org.apache.hadoop.hdfs.server.datanode.FSDataset$FSVolume.<init>(FSDataset.java:325)
> 	at org.apache.hadoop.hdfs.server.datanode.FSDataset.<init>(FSDataset.java:681)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:291)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:205)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1238)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1193)
> {noformat}
> This is because Hadoop execs "du -sk C:\tmp\hadoop-SYSTEM\dfs\data" with a Windows path
representation, which cygwin du doesn't understand.
> {noformat}
> C:\hudson>du -sk C:\tmp\hadoop-SYSTEM\dfs\data
> du -sk C:\tmp\hadoop-SYSTEM\dfs\data
> du: cannot access `C:\\tmp\\hadoop-SYSTEM\\dfs\\data': No such file or directory
> {noformat}
> For this to work correctly, Hadoop would have to run cygpath first to get a Unix path
representation, then to call DU.
> Also, I had to use the debugger to get this information. Shell.runCommand should catch
IOException from parseExecResult and add the buffered stderr to simplify the error diagnostics.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message