hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Error: java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.get
Date Sun, 18 Jul 2010 22:10:34 GMT
Have you looked at https://issues.apache.org/jira/browse/MAPREDUCE-969 ?
It points to https://issues.apache.org/jira/browse/HADOOP-4744

See if HADOOP-4744 is contained in the hadoop version you're running.

I am reading cdh3b2 source code. Please confirm that *ReduceTask.java:2683 *is
this line:
              List<MapOutputLocation> loc = mapLocations.get(host);
If so, you can print the value of host, or value of URI u which host is
derived.

Cheers

On Fri, Jul 16, 2010 at 9:42 AM, Chinni, Ravi <rchinni@syncsort.com> wrote:

>  Hadoop version: 0.20.2
>
>
>
> *From:* Ted Yu [mailto:yuzhihong@gmail.com]
> *Sent:* Friday, July 16, 2010 12:34 PM
> *To:* mapreduce-user@hadoop.apache.org
> *Subject:* Re: Error: java.lang.NullPointerException at
> java.util.concurrent.ConcurrentHashMap.get
>
>
>
> What version of hadoop are you using ?
>
> On Fri, Jul 16, 2010 at 8:56 AM, Chinni, Ravi <rchinni@syncsort.com>
> wrote:
>
> I am trying to run the terasort example with a small input on a 4 node
> cluster. I just did the minimal configuration (fs.default.name, master,
> slaves etc.), but did not do anything specific to terasort. I am getting the
> following java.lang.NullPointerException on running terasort example:
>
>
>
>
>
> 10/07/16 11:48:29 INFO terasort.TeraSort: starting
>
> 10/07/16 11:48:29 INFO mapred.FileInputFormat: Total input paths to process
> : 1
>
> 10/07/16 11:48:29 INFO util.NativeCodeLoader: Loaded the native-hadoop
> library
>
> 10/07/16 11:48:29 INFO zlib.ZlibFactory: Successfully loaded & initialized
> native-zlib library
>
> 10/07/16 11:48:29 INFO compress.CodecPool: Got brand-new compressor
>
> Making 1 from 10 records
>
> Step size is 10.0
>
> 10/07/16 11:48:30 INFO mapred.JobClient: Running job: job_201006251254_0010
>
> 10/07/16 11:48:31 INFO mapred.JobClient:  map 0% reduce 0%
>
> 10/07/16 11:48:38 INFO mapred.JobClient:  map 50% reduce 0%
>
> 10/07/16 11:48:39 INFO mapred.JobClient:  map 100% reduce 0%
>
> 10/07/16 11:48:43 INFO mapred.JobClient: Task Id :
> attempt_201006251254_0010_r_000000_0, Status : FAILED
>
> *Error: java.lang.NullPointerException*
>
> *        at
> java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)*
>
> *        at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
> *
>
> *        at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)
> *
>
>
>
> 10/07/16 11:48:49 INFO mapred.JobClient: Task Id :
> attempt_201006251254_0010_r_000000_1, Status : FAILED
>
> Error: java.lang.NullPointerException
>
>         at
> java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
>
>         at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
>
>         at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)
>
>
>
> 10/07/16 11:48:57 INFO mapred.JobClient: Task Id :
> attempt_201006251254_0010_r_000000_2, Status : FAILED
>
> Error: java.lang.NullPointerException
>
>         at
> java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
>
>         at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
>
>         at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)
>
>
>
>
>
>
>
> Also, I got similar error from reduce jobs on running a streaming
> map-reduce application. In this case also, the map tasks completed
> successfully.
>
>
>
>
>
> Any pointers are appreciated.
>
>
>
> Thanks,
>
> *Ravi Chinni*
>
>
>
>
>
>
> _____________________________________________________________________________
>
>
>
> ATTENTION:
>
>
>
> The information contained in this message (including any files transmitted
> with this message) may contain proprietary, trade secret or other
> confidential and/or legally privileged information. Any pricing information
> contained in this message or in any files transmitted with this message is
> always confidential and cannot be shared with any third parties without
> prior written approval from Syncsort. This message is intended to be read
> only by the individual or entity to whom it is addressed or by their
> designee. If the reader of this message is not the intended recipient, you
> are on notice that any use, disclosure, copying or distribution of this
> message, in any form, is strictly prohibited. If you have received this
> message in error, please immediately notify the sender and/or Syncsort and
> destroy all copies of this message in your possession, custody or control.
>
>
>

Mime
View raw message