hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-6179) Running any MapReduce jobs is throwing Heap Error
Date Wed, 03 Jun 2015 11:18:49 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Harsh J resolved MAPREDUCE-6179.
--------------------------------
    Resolution: Invalid

Please use user@ mailing lists for general questions. The JIRA is not a place to assist over
end-user problems, but for reporting actual identified issues.

> Running any MapReduce jobs is throwing Heap Error
> -------------------------------------------------
>
>                 Key: MAPREDUCE-6179
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6179
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 2.2.0
>         Environment: Linux 2.6.32-358.el6.x86_64
> RAM : 12 GB
>            Reporter: Ketan Deshmukh
>
> I have a hadoop distribution installed on a cluster with 1 namenode and 2 data nodes.
I have been trying to run the default mapreduce example included in the hadoop package and
its been throwing the following error :
> "Error occurred during initialization of VM
> Could not reserve enough space for object heap"
> The complete message test is as following :
> [192769@hawq /]$ hadoop jar /usr/lib/gphd/hadoop-mapreduce/hadoop-mapreduce-examples-2.2.0-gphd-3.0.1.0.jar
pi 10 100
> Number of Maps  = 10
> Samples per Map = 100
> Wrote input for Map #0
> Wrote input for Map #1
> Wrote input for Map #2
> Wrote input for Map #3
> Wrote input for Map #4
> Wrote input for Map #5
> Wrote input for Map #6
> Wrote input for Map #7
> Wrote input for Map #8
> Wrote input for Map #9
> Starting Job
> 14/12/04 17:28:44 INFO client.RMProxy: Connecting to ResourceManager at dn2.tcsgegdc.com/3.209.124.208:8032
> 14/12/04 17:28:44 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 50 for 192769
on 3.209.124.204:8020
> 14/12/04 17:28:44 INFO security.TokenCache: Got dt for hdfs://pcc.tcsgegdc.com:8020;
Kind: HDFS_DELEGATION_TOKEN, Service: 3.209.124.204:8020, Ident: (HDFS_DELEGATION_TOKEN token
50 for 192769)
> 14/12/04 17:28:44 INFO input.FileInputFormat: Total input paths to process : 10
> 14/12/04 17:28:44 INFO mapreduce.JobSubmitter: number of splits:10
> 14/12/04 17:28:44 INFO Configuration.deprecation: user.name is deprecated. Instead, use
mapreduce.job.user.name
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapred.jar is deprecated. Instead,
use mapreduce.job.jar
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution
is deprecated. Instead, use mapreduce.map.speculative
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated.
Instead, use mapreduce.job.reduces
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapred.output.value.class is deprecated.
Instead, use mapreduce.job.output.value.class
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution
is deprecated. Instead, use mapreduce.reduce.speculative
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapreduce.map.class is deprecated.
Instead, use mapreduce.job.map.class
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead,
use mapreduce.job.name
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapreduce.reduce.class is deprecated.
Instead, use mapreduce.job.reduce.class
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapreduce.inputformat.class is deprecated.
Instead, use mapreduce.job.inputformat.class
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead,
use mapreduce.input.fileinputformat.inputdir
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead,
use mapreduce.output.fileoutputformat.outputdir
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapreduce.outputformat.class is deprecated.
Instead, use mapreduce.job.outputformat.class
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead,
use mapreduce.job.maps
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapred.output.key.class is deprecated.
Instead, use mapreduce.job.output.key.class
> 14/12/04 17:28:44 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead,
use mapreduce.job.working.dir
> 14/12/04 17:28:44 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1413806590994_0039
> 14/12/04 17:28:44 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service:
3.209.124.204:8020, Ident: (HDFS_DELEGATION_TOKEN token 50 for 192769)
> 14/12/04 17:28:44 INFO impl.YarnClientImpl: Submitted application application_1413806590994_0039
to ResourceManager at dn2.tcsgegdc.com/3.209.124.208:8032
> 14/12/04 17:28:44 INFO mapreduce.Job: The url to track the job: http://dn2.tcsgegdc.com:8088/proxy/application_1413806590994_0039/
> 14/12/04 17:28:44 INFO mapreduce.Job: Running job: job_1413806590994_0039
> 14/12/04 17:28:48 INFO mapreduce.Job: Job job_1413806590994_0039 running in uber mode
: false
> 14/12/04 17:28:48 INFO mapreduce.Job:  map 0% reduce 0%
> 14/12/04 17:28:48 INFO mapreduce.Job: Job job_1413806590994_0039 failed with state FAILED
due to: Application application_1413806590994_0039 failed 2 times due to AM Container for
appattempt_1413806590994_0039_000002 exited with  exitCode: -1000 due to: Application application_1413806590994_0039
initialization failed (exitCode=1) with output: main : command provided 0
> main : user is 192769
> Error occurred during initialization of VM
> Could not reserve enough space for object heap
> .Failing this attempt.. Failing the application.
> 14/12/04 17:28:49 INFO mapreduce.Job: Counters: 0
> Job Finished in 4.863 seconds
> java.io.FileNotFoundException: File does not exist: hdfs://pcc.tcsgegdc.com:8020/user/192769/QuasiMonteCarlo_1417694322429_282484071/out/reduce-out
>         at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)
>         at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1112)
>         at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>         at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1112)
>         at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1749)
>         at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1773)
>         at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:314)
>         at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>         at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
>         at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
>         at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
> Please help understand what is causing this error as all installation steps were followed
to the word.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message