hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen, Richard" <richard.c...@intel.com>
Subject hadoop-3.0.0-SNAPSHOT runtime exception when trying PI example
Date Thu, 20 Feb 2014 06:09:36 GMT
Dear Hadoop User group,

This is my virgin post. So please bear with me if I made any mistakes at my early time here.

Recently I successfully downloaded and compiled Hadoop trunk code (svn co http://svn.apache.org/repos/asf/hadoop/common/trunk/
hadoop-trunk)
I also deployed the hadoop-3.0.0-SNAPSHOT distribution to a physical cluster of 5 nodes each
running 64-bit Cent OS 6.

After careful configuration, when I tried running the 1st PI job example:
 $hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar pi 4 10

The result was successfully calculated but some exception message was also dumped during the
execution and I couldn’t figure out why.
The exception message goes like this:

Starting Job
14/02/20 13:34:12 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
14/02/20 13:34:12 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker,
sessionId=
14/02/20 13:34:12 INFO input.FileInputFormat: Total input paths to process : 4
14/02/20 13:34:12 INFO mapreduce.JobSubmitter: number of splits:4
14/02/20 13:34:12 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1598491292_0001
14/02/20 13:34:12 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
14/02/20 13:34:12 INFO mapreduce.Job: Running job: job_local1598491292_0001
14/02/20 13:34:12 INFO mapred.LocalJobRunner: OutputCommitter set in config null
14/02/20 13:34:12 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
14/02/20 13:34:13 INFO mapred.LocalJobRunner: Waiting for map tasks
14/02/20 13:34:13 INFO mapred.LocalJobRunner: Starting task: attempt_local1598491292_0001_m_000000_0
14/02/20 13:34:13 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
14/02/20 13:34:13 INFO mapred.MapTask: Processing split: hdfs://samdev06:9000/user/hadoop/QuasiMonteCarlo_1392874450186_1650772770/in/part0:0+118
14/02/20 13:34:13 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
14/02/20 13:34:13 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
14/02/20 13:34:13 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
14/02/20 13:34:13 INFO mapred.MapTask: soft limit at 83886080
14/02/20 13:34:13 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
14/02/20 13:34:13 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
14/02/20 13:34:13 INFO mapred.LocalJobRunner:
14/02/20 13:34:13 INFO mapred.MapTask: Starting flush of map output
14/02/20 13:34:13 INFO mapred.MapTask: Spilling map output
14/02/20 13:34:13 INFO mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600
14/02/20 13:34:13 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568);
length = 5/6553600
14/02/20 13:34:13 INFO mapred.MapTask: Finished spill 0
14/02/20 13:34:13 INFO mapred.Task: Task:attempt_local1598491292_0001_m_000000_0 is done.
And is in the process of committing
14/02/20 13:34:13 WARN mapred.Task: Could not find output size
java.io.FileNotFoundException: File does not exist: /tmp/hadoop/mapred/local/localRunner/hadoop/jobcache/job_local1598491292_0001/attempt_local1598491292_0001_m_000000_0/output/file.out
                at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072)
                at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064)
                at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
                at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064)
                at org.apache.hadoop.mapred.Task.calculateOutputSize(Task.java:1124)
                at org.apache.hadoop.mapred.Task.sendLastUpdate(Task.java:1102)
                at org.apache.hadoop.mapred.Task.done(Task.java:1048)
                at org.apache.hadoop.mapred.MapTask.run(MapTask.java:344)
                at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:244)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
                at java.util.concurrent.FutureTask.run(FutureTask.java:262)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
                at java.lang.Thread.run(Thread.java:744)


Please kindly help me on this.

Thanks in advance!

Richard

Mime
View raw message