giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rahman, Muntasir Raihan" <mrahm...@illinois.edu>
Subject RE: container failure while running Giraph Shortest Path on Yarn
Date Sun, 22 May 2016 16:47:46 GMT
Hi,

Below are my yarn configs.  I have tried changing/increasing the yarn memory limits, and also
map/reduce memory limits in mapred-site.xml, but it doesn’t help.

Thank you!

Muntasir.

Yarn-site.xml:

“<configuration>

<!-- Site specific YARN configuration properties -->
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
    </property>
    <property>
        <name>yarn.resourcemanager.resource-tracker.address</name>
        <value>node-00.pcap-cluster.iss.emulab.net:8025</value>
    </property>
    <property>
        <name>yarn.resourcemanager.scheduler.address</name>
        <value>node-00.pcap-cluster.iss.emulab.net:8030</value>
    </property>
   <!-- Use CapacityScheduler -->
    <property>
        <name>yarn.resourcemanager.scheduler.class</name>
        <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
    </property>
    <property>
        <name>yarn.resourcemanager.address</name>
        <value>node-00.pcap-cluster.iss.emulab.net:8040</value>
    </property>
   <property>
        <name>yarn.nodemanager.local-dirs</name>
        <value>/tmp/mrahman2/hadoop-2.7.0/nm-local-dirs</value>
    </property>
    <property>
        <name>yarn.nodemanager.log-dirs</name>
        <value>/tmp/mrahman2/hadoop-2.7.0/nm-log-dirs</value>
    </property>
    <property>
                <name>yarn.nodemanager.vmem-check-enabled</name>
                <value>false</value>
    </property>

        <property>
                <name>yarn.nodemanager.pmem-check-enabled</name>
                <value>false</value>
        </property>

        <property>
                <name>yarn.scheduler.minimum-allocation-mb</name>
                <value>4096</value>
        </property>
<property>
                <name>yarn.scheduler.maximum-allocation-mb</name>
                <value>8192</value>
        </property>

    <property>
        <name>yarn.resourcemanager.resume-local-only</name>
        <value>false</value>
    </property>

<!-- medium containers -->
    <property>
        <name>yarn.nodemanager.resource.memory-mb</name>
        <value>4096</value>
    </property>

</configuration>
”

From yarn-env.sh:

“export HADOOP_HOME=/mnt/hadoop/hadoop-2.7.0
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME

# User for YARN daemons
export HADOOP_YARN_USER=${HADOOP_YARN_USER:-yarn}

# resolve links - $0 may be a softlink
export YARN_CONF_DIR="${YARN_CONF_DIR:-$YARN_HOME/conf}"
export HADOOP_CONF_DIR=${YARN_CONF_DIR}

# some Java parameters
# export JAVA_HOME=/home/y/libexec/jdk1.6.0/

export JAVA_HOME=/usr
echo $JAVA_HOME
if [ "$JAVA_HOME" != "" ]; then
  #echo "run java in $JAVA_HOME"
  JAVA_HOME=$JAVA_HOME
fi

if [ "$JAVA_HOME" = "" ]; then
  echo "Error: JAVA_HOME is not set."
  exit 1
fi

JAVA=$JAVA_HOME/bin/java
# small containers
JAVA_HEAP_MAX=-Xmx500m

if [ "$YARN_HEAPSIZE" != "" ]; then
  #echo "run with heapsize $YARN_HEAPSIZE"
  JAVA_HEAP_MAX="-Xmx""$YARN_HEAPSIZE""m"
  #echo $JAVA_HEAP_MAX
Fi
IFS=


# default log directory & file
if [ "$YARN_LOG_DIR" = "" ]; then
  YARN_LOG_DIR="/tmp/mrahman2/hadoop-2.7.0/yarn-logs"
fi
if [ "$YARN_LOGFILE" = "" ]; then
  YARN_LOGFILE='yarn.log'
fi

# default policy file for service-level authorization
if [ "$YARN_POLICYFILE" = "" ]; then
  YARN_POLICYFILE="hadoop-policy.xml"
fi

# restore ordinary behaviour
unset IFS

YARN_OPTS="$YARN_OPTS -Dhadoop.log.dir=$YARN_LOG_DIR"
YARN_OPTS="$YARN_OPTS -Dyarn.log.dir=$YARN_LOG_DIR"
YARN_OPTS="$YARN_OPTS -Dhadoop.log.file=$YARN_LOGFILE"
YARN_OPTS="$YARN_OPTS -Dyarn.log.file=$YARN_LOGFILE"
YARN_OPTS="$YARN_OPTS -Dyarn.home.dir=$YARN_COMMON_HOME"
YARN_OPTS="$YARN_OPTS -Dyarn.id.str=$YARN_IDENT_STRING"
YARN_OPTS="$YARN_OPTS -Dhadoop.root.logger=${YARN_ROOT_LOGGER:-INFO,console}"
YARN_OPTS="$YARN_OPTS -Dyarn.root.logger=${YARN_ROOT_LOGGER:-INFO,console}"
if [ "x$JAVA_LIBRARY_PATH" != "x" ]; then
  YARN_OPTS="$YARN_OPTS -Djava.library.path=$JAVA_LIBRARY_PATH"
fi
YARN_OPTS="$YARN_OPTS -Dyarn.policy.file=$YARN_POLICYFILE"


”

From: Ramesh Krishnan [mailto:ramesh.154089@gmail.com]
Sent: Sunday, May 22, 2016 11:31 AM
To: user@giraph.apache.org
Subject: Re: container failure while running Giraph Shortest Path on Yarn

Hi,
This does not seem to be a dependecy issue . This is an exception in launch of the container.
Can you check the yarn configurations.
Thanks,
Ramesh


On Sun, May 22, 2016 at 9:52 PM, Rahman, Muntasir Raihan <mrahman2@illinois.edu<mailto:mrahman2@illinois.edu>>
wrote:
Thanks for the suggestion, but unfortunately it didn’t fix the issue.

Muntasir.

From: Ravikant Dindokar [mailto:ravikant.iisc@gmail.com<mailto:ravikant.iisc@gmail.com>]
Sent: Sunday, May 22, 2016 3:37 AM
To: user@giraph.apache.org<mailto:user@giraph.apache.org>
Subject: Re: container failure while running Giraph Shortest Path on Yarn

try adding

-libjars giraph-core/target/giraph-1.1.0-for-hadoop-2.7.0-jar-with-dependencies.jar after
org.apache.giraph.GiraphRunner


On Sat, May 21, 2016 at 4:21 AM, Rahman, Muntasir Raihan <mrahman2@illinois.edu<mailto:mrahman2@illinois.edu>>
wrote:
Hi,

I have been trying to run some Giraph programs on top of Yarn, but I keep getting the following
error:


“Container id: container_1463763127358_0002_01_000003
Exit code: 2
Stack trace: ExitCodeException exitCode=2:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
        at org.apache.hadoop.util.Shell.run(Shell.java:456)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
        at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 2
”

I ran the following command:

“$HADOOP_HOME/bin/hadoop jar giraph-examples/target/giraph-examples-1.1.0-for-hadoop-2.7.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip <inputPath>
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op <outputPath> -w 1
-yj giraph-examples-1.1.0-for-hadoop-2.7.0-jar-with-dependencies.jar”


Are there any suggestions or hints to fix this?

Regards
Muntasir.


Mime
View raw message