reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julia Wang (QIUHE)" <Qiuhe.W...@microsoft.com.INVALID>
Subject RE: YARN test failure
Date Fri, 30 Jun 2017 22:03:14 GMT
Check if the path for HADOOP_CONF_DIR is set correctly in the yarn machine where you run the
test. 

Julia

-----Original Message-----
From: 안석찬 [mailto:ahnsc93@gmail.com] 
Sent: Friday, June 30, 2017 3:32 AM
To: dev@reef.apache.org
Subject: YARN test failure

Hi,

I implemented YARN node labeling on REEF and passed the local tests.(`mvn test` on reef home)
And now I’m trying to test my code on pseudo-cluster environment. 

I set up a single-node in a pseudo-distributed mode and ran an example code on it. It works
well up to here.
But it fails on reef yarn test(reef/bin/runyarntest.sh). It says repeatedly like below.

SEVERE: YarnConfiguration.YARN_APPLICATION_CLASSPATH is empty. This indicates a broken cluster
configuration.

I have a hard time finding the reason why reef can’t find the application classpath.
Could you help me figure out why?

more error messages:
$scan ./runyarntests.sh
java -cp :/Users/scan/Workspace/reef/lang/java/reef-tests/target/reef-tests-0.16.0-SNAPSHOT-test-jar-with-dependencies.jar:/Users/scan/hadoop/etc/hadoop:/Users/scan/hadoop/etc/hadoop:/Users/scan/hadoop/etc/hadoop:/Users/scan/hadoop/share/hadoop/common/lib/*:/Users/scan/hadoop/share/hadoop/common/*:/Users/scan/hadoop/share/hadoop/hdfs:/Users/scan/hadoop/share/hadoop/hdfs/lib/*:/Users/scan/hadoop/share/hadoop/hdfs/*:/Users/scan/hadoop/share/hadoop/yarn/lib/*:/Users/scan/hadoop/share/hadoop/yarn/*:/Users/scan/hadoop/share/hadoop/mapreduce/lib/*:/Users/scan/hadoop/share/hadoop/mapreduce/*:/Users/scan/hadoop/contrib/capacity-scheduler/*.jar:/Users/scan/hadoop/share/hadoop/yarn/*:/Users/scan/hadoop/share/hadoop/yarn/lib/*
org.junit.runner.JUnitCore org.apache.reef.tests.AllTestsSuite
JUnit version 4.11
Jun 30, 2017 7:28:02 PM org.apache.reef.tests.TestEnvironmentFactory getNewTestEnvironment
INFO: Running tests on YARN
.Jun 30, 2017 7:28:03 PM org.apache.reef.runtime.yarn.YarnClasspathProvider <init>
SEVERE: YarnConfiguration.YARN_APPLICATION_CLASSPATH is empty. This indicates a broken cluster
configuration.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/scan/Workspace/reef/lang/java/reef-tests/target/reef-tests-0.16.0-SNAPSHOT-test-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/scan/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.slf4j.org%2Fcodes.html%23multiple_bindings&data=02%7C01%7CQiuhe.Wang%40microsoft.com%7C9192745026db4d2872b808d4bfa355c6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636344155623769306&sdata=zHWeZxV4tBTkcO0XqBLn380xbh9SP8lp26YdW7QajgE%3D&reserved=0
for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
Jun 30, 2017 7:28:03 PM org.apache.hadoop.util.NativeCodeLoader <clinit>
WARNING: Unable to load native-hadoop library for your platform... using builtin-java classes
where applicable EJun 30, 2017 7:28:03 PM org.apache.reef.tests.TestEnvironmentFactory getNewTestEnvironment
INFO: Running tests on YARN
.Jun 30, 2017 7:28:03 PM org.apache.reef.runtime.yarn.YarnClasspathProvider <init>
SEVERE: YarnConfiguration.YARN_APPLICATION_CLASSPATH is empty. This indicates a broken cluster
configuration.
EJun 30, 2017 7:28:03 PM org.apache.reef.tests.TestEnvironmentFactory getNewTestEnvironment
INFO: Running tests on YARN
.Jun 30, 2017 7:28:03 PM org.apache.reef.runtime.yarn.YarnClasspathProvider <init>
SEVERE: YarnConfiguration.YARN_APPLICATION_CLASSPATH is empty. This indicates a broken cluster
configuration.
EJun 30, 2017 7:28:03 PM org.apache.reef.tests.TestEnvironmentFactory getNewTestEnvironment
INFO: Running tests on YARN
.Jun 30, 2017 7:28:03 PM org.apache.reef.runtime.yarn.YarnClasspathProvider <init>
SEVERE: YarnConfiguration.YARN_APPLICATION_CLASSPATH is empty. This indicates a broken cluster
configuration.
EJun 30, 2017 7:28:03 PM org.apache.reef.tests.TestEnvironmentFactory getNewTestEnvironment
INFO: Running tests on YARN
.Jun 30, 2017 7:28:03 PM org.apache.reef.runtime.yarn.YarnClasspathProvider <init>
SEVERE: YarnConfiguration.YARN_APPLICATION_CLASSPATH is empty. This indicates a broken cluster
configuration.
EJun 30, 2017 7:28:03 PM org.apache.reef.tests.TestEnvironmentFactory getNewTestEnvironment
INFO: Running tests on YARN
.Jun 30, 2017 7:28:03 PM org.apache.reef.runtime.yarn.YarnClasspathProvider <init>
SEVERE: YarnConfiguration.YARN_APPLICATION_CLASSPATH is empty. This indicates a broken cluster
configuration.
EJun 30, 2017 7:28:03 PM org.apache.reef.tests.TestEnvironmentFactory getNewTestEnvironment
…

hadoop related part of my .zshrc:

export HADOOP_HOME=/Users/scan/hadoop
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
export HADOOP_INSTALL=$HADOOP_HOME
export REEF_HOME=/Users/scan/Workspace/dummy/reef
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
Mime
View raw message