hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chinni, Ravi" <rchi...@syncsort.com>
Subject RE: Error: java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.get
Date Mon, 19 Jul 2010 15:41:10 GMT
The problem got fixed by using ip-address in all hadoop configuration
files and also in /etc/hosts file.

 

Anyway, thanks Ted. To partially answer your question - line
ReduceTask.java:2683 is 'List<MapOutputLocation> loc =
mapLocations.get(host);'

 

 

 

From: Ted Yu [mailto:yuzhihong@gmail.com] 
Sent: Sunday, July 18, 2010 6:11 PM
To: mapreduce-user@hadoop.apache.org
Subject: Re: Error: java.lang.NullPointerException at
java.util.concurrent.ConcurrentHashMap.get

 

Have you looked at https://issues.apache.org/jira/browse/MAPREDUCE-969 ?
It points to https://issues.apache.org/jira/browse/HADOOP-4744

See if HADOOP-4744 is contained in the hadoop version you're running.

I am reading cdh3b2 source code. Please confirm that
ReduceTask.java:2683 is this line:
              List<MapOutputLocation> loc = mapLocations.get(host);
If so, you can print the value of host, or value of URI u which host is
derived.

Cheers

On Fri, Jul 16, 2010 at 9:42 AM, Chinni, Ravi <rchinni@syncsort.com>
wrote:

Hadoop version: 0.20.2

 

From: Ted Yu [mailto:yuzhihong@gmail.com] 
Sent: Friday, July 16, 2010 12:34 PM
To: mapreduce-user@hadoop.apache.org
Subject: Re: Error: java.lang.NullPointerException at
java.util.concurrent.ConcurrentHashMap.get

 

What version of hadoop are you using ?

On Fri, Jul 16, 2010 at 8:56 AM, Chinni, Ravi <rchinni@syncsort.com>
wrote:

I am trying to run the terasort example with a small input on a 4 node
cluster. I just did the minimal configuration (fs.default.name, master,
slaves etc.), but did not do anything specific to terasort. I am getting
the following java.lang.NullPointerException on running terasort
example:

 

 

10/07/16 11:48:29 INFO terasort.TeraSort: starting

10/07/16 11:48:29 INFO mapred.FileInputFormat: Total input paths to
process : 1

10/07/16 11:48:29 INFO util.NativeCodeLoader: Loaded the native-hadoop
library

10/07/16 11:48:29 INFO zlib.ZlibFactory: Successfully loaded &
initialized native-zlib library

10/07/16 11:48:29 INFO compress.CodecPool: Got brand-new compressor

Making 1 from 10 records

Step size is 10.0

10/07/16 11:48:30 INFO mapred.JobClient: Running job:
job_201006251254_0010

10/07/16 11:48:31 INFO mapred.JobClient:  map 0% reduce 0%

10/07/16 11:48:38 INFO mapred.JobClient:  map 50% reduce 0%

10/07/16 11:48:39 INFO mapred.JobClient:  map 100% reduce 0%

10/07/16 11:48:43 INFO mapred.JobClient: Task Id :
attempt_201006251254_0010_r_000000_0, Status : FAILED

Error: java.lang.NullPointerException

        at
java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)

        at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getM
apCompletionEvents(ReduceTask.java:2683)

        at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(
ReduceTask.java:2605)

 

10/07/16 11:48:49 INFO mapred.JobClient: Task Id :
attempt_201006251254_0010_r_000000_1, Status : FAILED

Error: java.lang.NullPointerException

        at
java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)

        at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getM
apCompletionEvents(ReduceTask.java:2683)

        at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(
ReduceTask.java:2605)

 

10/07/16 11:48:57 INFO mapred.JobClient: Task Id :
attempt_201006251254_0010_r_000000_2, Status : FAILED

Error: java.lang.NullPointerException

        at
java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)

        at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getM
apCompletionEvents(ReduceTask.java:2683)

        at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(
ReduceTask.java:2605)

 

 

 

Also, I got similar error from reduce jobs on running a streaming
map-reduce application. In this case also, the map tasks completed
successfully.

 

 

Any pointers are appreciated.

 

Thanks,

Ravi Chinni

 

 

________________________________________________________________________
_____

 

ATTENTION:

 

The information contained in this message (including any files
transmitted with this message) may contain proprietary, trade secret or
other  confidential and/or legally privileged information. Any pricing
information contained in this message or in any files transmitted with
this message is always confidential and cannot be shared with any third
parties without prior written approval from Syncsort. This message is
intended to be read only by the individual or entity to whom it is
addressed or by their designee. If the reader of this message is not the
intended recipient, you are on notice that any use, disclosure, copying
or distribution of this message, in any form, is strictly prohibited. If
you have received this message in error, please immediately notify the
sender and/or Syncsort and destroy all copies of this message in your
possession, custody or control.

 

 


Mime
View raw message