hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@apache.org>
Subject Re: hadoop-3.0.0-SNAPSHOT runtime exception when trying PI example
Date Fri, 21 Feb 2014 05:36:43 GMT
Are you trying to run it on a cluster or locally? There is something called 'local mode' which
is getting activated here (see that job-id has a _local_ string).

You need to point your configuration to the real ResourceManager. Share your configs. Or better
share the resource that you are following as instructions.

+Viond

On Feb 19, 2014, at 10:09 PM, Chen, Richard <richard.chen@intel.com> wrote:

> Dear Hadoop User group,
>  
> This is my virgin post. So please bear with me if I made any mistakes at my early time
here.
>  
> Recently I successfully downloaded and compiled Hadoop trunk code (svn co http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-trunk)
> I also deployed the hadoop-3.0.0-SNAPSHOT distribution to a physical cluster of 5 nodes
each running 64-bit Cent OS 6.
>  
> After careful configuration, when I tried running the 1st PI job example:
>  $hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar pi 4
10
>  
> The result was successfully calculated but some exception message was also dumped during
the execution and I couldn’t figure out why.
> The exception message goes like this:
>  
> Starting Job
> 14/02/20 13:34:12 INFO Configuration.deprecation: session.id is deprecated. Instead,
use dfs.metrics.session-id
> 14/02/20 13:34:12 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker,
sessionId=
> 14/02/20 13:34:12 INFO input.FileInputFormat: Total input paths to process : 4
> 14/02/20 13:34:12 INFO mapreduce.JobSubmitter: number of splits:4
> 14/02/20 13:34:12 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1598491292_0001
> 14/02/20 13:34:12 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
> 14/02/20 13:34:12 INFO mapreduce.Job: Running job: job_local1598491292_0001
> 14/02/20 13:34:12 INFO mapred.LocalJobRunner: OutputCommitter set in config null
> 14/02/20 13:34:12 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
> 14/02/20 13:34:13 INFO mapred.LocalJobRunner: Waiting for map tasks
> 14/02/20 13:34:13 INFO mapred.LocalJobRunner: Starting task: attempt_local1598491292_0001_m_000000_0
> 14/02/20 13:34:13 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
> 14/02/20 13:34:13 INFO mapred.MapTask: Processing split:hdfs://samdev06:9000/user/hadoop/QuasiMonteCarlo_1392874450186_1650772770/in/part0:0+118
> 14/02/20 13:34:13 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> 14/02/20 13:34:13 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
> 14/02/20 13:34:13 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
> 14/02/20 13:34:13 INFO mapred.MapTask: soft limit at 83886080
> 14/02/20 13:34:13 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
> 14/02/20 13:34:13 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
> 14/02/20 13:34:13 INFO mapred.LocalJobRunner:
> 14/02/20 13:34:13 INFO mapred.MapTask: Starting flush of map output
> 14/02/20 13:34:13 INFO mapred.MapTask: Spilling map output
> 14/02/20 13:34:13 INFO mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600
> 14/02/20 13:34:13 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568);
length = 5/6553600
> 14/02/20 13:34:13 INFO mapred.MapTask: Finished spill 0
> 14/02/20 13:34:13 INFO mapred.Task: Task:attempt_local1598491292_0001_m_000000_0 is done.
And is in the process of committing
> 14/02/20 13:34:13 WARN mapred.Task: Could not find output size
> java.io.FileNotFoundException: File does not exist: /tmp/hadoop/mapred/local/localRunner/hadoop/jobcache/job_local1598491292_0001/attempt_local1598491292_0001_m_000000_0/output/file.out
>                 at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072)
>                 at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064)
>                 at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>                 at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064)
>                 at org.apache.hadoop.mapred.Task.calculateOutputSize(Task.java:1124)
>                 at org.apache.hadoop.mapred.Task.sendLastUpdate(Task.java:1102)
>                 at org.apache.hadoop.mapred.Task.done(Task.java:1048)
>                 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:344)
>                 at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:244)
>                 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>                 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>                 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>                 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>                 at java.lang.Thread.run(Thread.java:744)
>  
>  
> Please kindly help me on this.
>  
> Thanks in advance!
>  
> Richard


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message