hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anil gupta <anilgupt...@gmail.com>
Subject Yarn job runs in Local Mode even though the cluster is running in Distributed Mode
Date Wed, 13 Jun 2012 19:16:05 GMT
Hi All

I am using cdh4 for running a HBase cluster on CentOs6.0. I have 5
nodes in my cluster(2 Admin Node and 3 DN).
My resourcemanager is up and running and showing that all three DN are
running the nodemanager. HDFS is also working fine and showing 3 DN's.

But when i fire the pi example job. It starts to run in Local mode.
Here is the console output:
sudo -u hdfs yarn jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-
examples.jar pi 10 1000000000
Number of Maps  = 10
Samples per Map = 1000000000
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
12/06/13 12:03:27 WARN conf.Configuration: session.id is deprecated.
Instead, use dfs.metrics.session-id
12/06/13 12:03:27 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
12/06/13 12:03:27 INFO util.NativeCodeLoader: Loaded the native-hadoop
12/06/13 12:03:27 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the
12/06/13 12:03:28 INFO mapred.FileInputFormat: Total input paths to
process : 10
12/06/13 12:03:29 INFO mapred.JobClient: Running job: job_local_0001
12/06/13 12:03:29 INFO mapred.LocalJobRunner: OutputCommitter set in
config null
12/06/13 12:03:29 INFO mapred.LocalJobRunner: OutputCommitter is
12/06/13 12:03:29 WARN mapreduce.Counters: Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
12/06/13 12:03:29 INFO util.ProcessTree: setsid exited with exit code
12/06/13 12:03:29 INFO mapred.Task:  Using ResourceCalculatorPlugin :
12/06/13 12:03:29 WARN mapreduce.Counters: Counter name
MAP_INPUT_BYTES is deprecated. Use FileInputFormatCounters as group
name and  BYTES_READ as counter name instead
12/06/13 12:03:29 INFO mapred.MapTask: numReduceTasks: 1
12/06/13 12:03:29 INFO mapred.MapTask: io.sort.mb = 100
12/06/13 12:03:30 INFO mapred.MapTask: data buffer = 79691776/99614720
12/06/13 12:03:30 INFO mapred.MapTask: record buffer = 262144/327680
12/06/13 12:03:30 INFO mapred.JobClient:  map 0% reduce 0%
12/06/13 12:03:35 INFO mapred.LocalJobRunner: Generated 95735000
12/06/13 12:03:36 INFO mapred.JobClient:  map 100% reduce 0%
12/06/13 12:03:38 INFO mapred.LocalJobRunner: Generated 151872000

Here is the content of yarn-site.xml:




    <description>List of directories to store localized files in.</

    <description>Where to store container logs.</description>

    <description>Where to aggregate logs to.</description>

    <description>Classpath for typical applications.</description>

Can anyone tell me what is the problem over here? Appreciate your
Anil Gupta

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message