hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10775) Shell operations could fail with meaningful errors on windows if winutils.exe not found
Date Wed, 02 Jul 2014 13:23:24 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049892#comment-14049892
] 

Steve Loughran commented on HADOOP-10775:
-----------------------------------------

Example log with stack trace; the minicluster is trying to create a symlink by executing the
command returned by {{Shell.getSymlinkCommand(targetPath, linkPath))}}
{code}
java.lang.NullPointerException: null
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1010)
	at org.apache.hadoop.util.Shell.runCommand(Shell.java:445)
	at org.apache.hadoop.util.Shell.run(Shell.java:418)
	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
	at org.apache.hadoop.yarn.server.MiniYARNCluster.<init>(MiniYARNCluster.java:171)
	at org.apache.hadoop.yarn.server.MiniYARNCluster.<init>(MiniYARNCluster.java:197)
	at org.apache.hadoop.yarn.server.MiniYARNCluster.<init>(MiniYARNCluster.java:209)
	at org.apache.slider.test.YarnMiniClusterTestBase.createMiniCluster(YarnMiniClusterTestBase.groovy:241)
	at org.apache.slider.test.YarnZKMiniClusterTestBase.createMiniCluster(YarnZKMiniClusterTestBase.groovy:105)
	at org.apache.slider.test.YarnZKMiniClusterTestBase.createMiniCluster(YarnZKMiniClusterTestBase.groovy:124)
	at org.apache.slider.agent.standalone.TestBadAMHeap.testBadAMHeap(TestBadAMHeap.groovy:43)Standard
Output2014-07-02 13:44:56,336 [main] ERROR util.Shell (Shell.java:getWinUtilsPath(336)) -
Failed to locate the winutils binary in the hadoop binary pathjava.io.IOException: Could not
locate executable null\bin\winutils.exe in the Hadoop binaries.
	at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
	at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
	at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
	at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)
	at org.apache.hadoop.yarn.conf.YarnConfiguration.<clinit>(YarnConfiguration.java:517)
	at org.apache.slider.common.tools.SliderUtils.createConfiguration(SliderUtils.java:413)
	at org.apache.slider.test.YarnMiniClusterTestBase.<clinit>(YarnMiniClusterTestBase.groovy:87)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
{code}

> Shell operations could fail with meaningful errors on windows if winutils.exe not found
> ---------------------------------------------------------------------------------------
>
>                 Key: HADOOP-10775
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10775
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: util
>    Affects Versions: trunk-win
>         Environment: Apache jenkins windows1 server
>            Reporter: Steve Loughran
>            Priority: Minor
>
> If {{winutils.exe}} can't be found {{HADOOP_HOME}} wrong/unset or other causes, then
an error is logged -but when any of the {{Shell}} operations are used, an NPE is raised rather
than something meaningful.
> The error message at setup time should be preserved and then raised before any attempt
to invoke a winutils-driven process made



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message