reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Weller <swel...@microsoft.com.INVALID>
Subject FW: Output error from attempting to run HelloREEF.exe
Date Fri, 28 Jul 2017 19:41:18 GMT
I am having a problem where I am unable to run the 'HelloREEF.exe' example using either the
'yarn' or 'hdi' setting on my Windows HDI cluster headnode.

Does anyone have any suggestions/ideas what I might be doing wrong or what settings I should
check?


Thanks a lot!

Stephen Weller




From: Stephen Weller
Sent: Friday, July 28, 2017 10:41 AM
To: Rogan Carr <rocarr@microsoft.com>; Doug Service <dougse@microsoft.com>
Cc: Scott Inglis <singlis@microsoft.com>
Subject: RE: Output error from attempting to run HelloREEF.exe

Hi Rogan,

Running in local mode works fine, but with setting 'yarn' it fails and it fails on my local
dev machine as well...

Here is the output from that run:

C:\gitrepos\reef\lang\cs\bin\x64\Debug\Org.Apache.REEF.Examples.HelloREEF>.\Org.
Apache.REEF.Examples.HelloREEF.exe yarn
Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl Error: 0 : 2017-
07-28T17:33:15.8987875+00:00 0001
ERROR: ExceptionCaught TargetInvocationException
Encountered error [System.Reflection.TargetInvocationException: Exception has be
en thrown by the target of an invocation. ---> System.ArgumentException:  is not
configured or does not exist.
   at Org.Apache.REEF.Utilities.Runtime.Yarn.YarnConfiguration..ctor(String hado
opConfDir, String yarnConfigFileName, Nullable`1 useHttps) in C:\gitrepos\reef\l
ang\cs\Org.Apache.REEF.Utilities\Runtime\Yarn\YarnConfiguration.cs:line 103
   at Org.Apache.REEF.Utilities.Runtime.Yarn.YarnConfiguration.GetConfiguration(
String hadoopConfDir, String yarnConfigFileName, Nullable`1 useHttps) in C:\gitr
epos\reef\lang\cs\Org.Apache.REEF.Utilities\Runtime\Yarn\YarnConfiguration.cs:li
ne 82
   at Org.Apache.REEF.Client.Yarn.RestClient.YarnConfigurationUrlProvider..ctor(
String hadoopConfigDir, Boolean useHttps) in C:\gitrepos\reef\lang\cs\Org.Apache
.REEF.Client\YARN\RestClient\YarnConfigurationUrlProvider.cs:line 70
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments,
Signature sig, Boolean constructor)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, B
inder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
   at Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFrom
Plan(InjectionPlan plan) in C:\gitrepos\reef\lang\cs\Org.Apache.REEF.Tang\Implem
entations\InjectionPlan\InjectorImpl.cs:line 164]
Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl Error: 0 : 2017-
07-28T17:33:15.9144145+00:00 0001
ERROR: ExceptionThrowing InjectionException
Encountered error [Org.Apache.REEF.Tang.Exceptions.InjectionException: Could not
invoke constructor: new Org.Apache.REEF.Client.Yarn.RestClient.YarnConfiguratio
nUrlProvider(System.String Org.Apache.REEF.Client.Yarn.RestClient.YarnConfigurat
ionUrlProvider+HadoopConfigurationDirectory =  , System.Boolean Org.Apache.REEF.
Client.Yarn.RestClient.YarnConfigurationUrlProvider+UseHttpsForYarnCommunication
= False) ---> System.Reflection.TargetInvocationException: Exception has been t
hrown by the target of an invocation. ---> System.ArgumentException:  is not con
figured or does not exist.
   at Org.Apache.REEF.Utilities.Runtime.Yarn.YarnConfiguration..ctor(String hado
opConfDir, String yarnConfigFileName, Nullable`1 useHttps) in C:\gitrepos\reef\l
ang\cs\Org.Apache.REEF.Utilities\Runtime\Yarn\YarnConfiguration.cs:line 103
   at Org.Apache.REEF.Utilities.Runtime.Yarn.YarnConfiguration.GetConfiguration(
String hadoopConfDir, String yarnConfigFileName, Nullable`1 useHttps) in C:\gitr
epos\reef\lang\cs\Org.Apache.REEF.Utilities\Runtime\Yarn\YarnConfiguration.cs:li
ne 82
   at Org.Apache.REEF.Client.Yarn.RestClient.YarnConfigurationUrlProvider..ctor(
String hadoopConfigDir, Boolean useHttps) in C:\gitrepos\reef\lang\cs\Org.Apache
.REEF.Client\YARN\RestClient\YarnConfigurationUrlProvider.cs:line 70
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments,
Signature sig, Boolean constructor)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, B
inder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
   at Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFrom
Plan(InjectionPlan plan) in C:\gitrepos\reef\lang\cs\Org.Apache.REEF.Tang\Implem
entations\InjectionPlan\InjectorImpl.cs:line 164
   --- End of inner exception stack trace ---]


Steve Weller




From: Rogan Carr
Sent: Friday, July 28, 2017 10:04 AM
To: Stephen Weller <sweller@microsoft.com<mailto:sweller@microsoft.com>>; Doug
Service <dougse@microsoft.com<mailto:dougse@microsoft.com>>
Cc: Scott Inglis <singlis@microsoft.com<mailto:singlis@microsoft.com>>
Subject: RE: Output error from attempting to run HelloREEF.exe

Hi Steve,

Try running HelloREEF from the same directory as the binary. The REEF client has a shortcoming
where it only looks in the current directory for supporting files.

Best,
Rogan

From: Stephen Weller
Sent: Friday, July 28, 2017 9:51 AM
To: Doug Service <dougse@microsoft.com<mailto:dougse@microsoft.com>>
Cc: Scott Inglis <singlis@microsoft.com<mailto:singlis@microsoft.com>>; Rogan
Carr <rocarr@microsoft.com<mailto:rocarr@microsoft.com>>
Subject: Output error from attempting to run HelloREEF.exe

Here is the output from the run - it is failing with a severe error - no doubt I'm still missing
a setting I need:

PS J:\gitrepos\reef\lang\cs\bin\x64\Debug\Org.Apache.REEF.Examples.HelloREEF> .\Org.Apache.REEF.Examples.HelloREEF.exe
yarn
Org.Apache.REEF.Client.Common.DriverFolderPreparationHelper Information: 0 : 2017-07-28T16:35:14.6496227+00:00
0001
INFO: Done preparing Driver filesystem layout in D:\Users\rdpuser\AppData\Local\Temp\3\reef-HelloREEF-20170728163331130
Org.Apache.REEF.Client.Common.JavaClientLauncher Information: 0 : 2017-07-28T16:35:20.5201394+00:00
0001
INFO: Launch Java with command: C:\apps\dist\java\bin\java.exe -cp C:\apps\dist\hadoop-2.7.1.2.3.3.1-25\etc\hadoop;C:\a
ps\dist\hadoop-2.7.1.2.3.3.1-25\share\hadoop\common\lib\*;C:\apps\dist\hadoop-2.7.1.2.3.3.1-25\share\hadoop\common\*;C:
apps\dist\hadoop-2.7.1.2.3.3.1-25\share\hadoop\hdfs;C:\apps\dist\hadoop-2.7.1.2.3.3.1-25\share\hadoop\hdfs\lib\*;C:\app
\dist\hadoop-2.7.1.2.3.3.1-25\share\hadoop\hdfs\*;C:\apps\dist\hadoop-2.7.1.2.3.3.1-25\share\hadoop\yarn\lib\*;C:\apps\
ist\hadoop-2.7.1.2.3.3.1-25\share\hadoop\yarn\*;C:\apps\dist\hadoop-2.7.1.2.3.3.1-25\share\hadoop\mapreduce\lib\*;C:\ap
s\dist\hadoop-2.7.1.2.3.3.1-25\share\hadoop\mapreduce\*;C:\apps\dist\azureLogging\*;C:\apps\dist\tez-0.7.0.2.3.3.1-25\c
nf\;C:\apps\dist\tez-0.7.0.2.3.3.1-25\*;C:\apps\dist\tez-0.7.0.2.3.3.1-25\lib\*;;./reef-bridge-client-0.16.0-SNAPSHOT-s
aded.jar org.apache.reef.bridge.client.YarnJobSubmissionClient D:\Users\rdpuser\AppData\Local\Temp\3\reef-HelloREEF-201
0728163331130\job-submission-params.json D:\Users\rdpuser\AppData\Local\Temp\3\reef-HelloREEF-20170728163331130\app-sub
ission-params.json
Org.Apache.REEF.Client.Common.JavaClientLauncher Error: 0 : 2017-07-28T16:35:31.2249354+00:00
0003
ERROR: Jul 28, 2017 4:35:31 PM org.apache.reef.bridge.client.YarnJobSubmissionClient main
Org.Apache.REEF.Client.Common.JavaClientLauncher Error: 0 : 2017-07-28T16:35:31.2259347+00:00
0003
ERROR: INFO: YARN job submission received from C#: YarnClusterSubmissionFromCS{driverFolder=D:\Users\rdpuser\AppData\Lo
al\Temp\3\reef-HelloREEF-20170728163331130, jobId='HelloREEF', driverMemory=512, tcpBeginPort=8900,
tcpRangeCount=1000,
tcpTryCount=1000, maxApplicationSubmissions=1, driverRecoveryTimeout=-1, priority=1, queue='default',
tokenKind='NULL',
tokenService='NULL', fileSystemUrl='NULL', jobSubmissionDirectoryPrefix='/vol1/tmp'}
Org.Apache.REEF.Client.Common.JavaClientLauncher Error: 0 : 2017-07-28T16:35:31.2289223+00:00
0006
ERROR: Jul 28, 2017 4:35:31 PM org.apache.reef.bridge.client.YarnJobSubmissionClient main
Org.Apache.REEF.Client.Common.JavaClientLauncher Error: 0 : 2017-07-28T16:35:31.2289223+00:00
0006
ERROR: INFO: FileSystemUrl is not set, use default from the environment.
Org.Apache.REEF.Client.Common.JavaClientLauncher Error: 0 : 2017-07-28T16:36:01.3339099+00:00
0006
ERROR: Jul 28, 2017 4:36:01 PM org.apache.reef.runtime.yarn.YarnClasspathProvider <init>
Org.Apache.REEF.Client.Common.JavaClientLauncher Error: 0 : 2017-07-28T16:36:01.3349076+00:00
0006
ERROR: SEVERE: YarnConfiguration.YARN_APPLICATION_CLASSPATH is empty. This indicates a broken
cluster configuration.


Rogan do you have any ideas what other settings I need to pass in here?



Steve Weller

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message