Return-Path: Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: (qmail 99218 invoked from network); 18 Jul 2010 22:11:58 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 18 Jul 2010 22:11:58 -0000 Received: (qmail 182 invoked by uid 500); 18 Jul 2010 22:11:57 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 99980 invoked by uid 500); 18 Jul 2010 22:11:57 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 99972 invoked by uid 99); 18 Jul 2010 22:11:56 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 18 Jul 2010 22:11:56 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of yuzhihong@gmail.com designates 209.85.216.48 as permitted sender) Received: from [209.85.216.48] (HELO mail-qw0-f48.google.com) (209.85.216.48) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 18 Jul 2010 22:11:50 +0000 Received: by qwd7 with SMTP id 7so2231770qwd.35 for ; Sun, 18 Jul 2010 15:10:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=5vBI2l4QeuezT70RxTljsYRslKwI/iDJ++YH6N7xBIE=; b=cXQVF00DSd9kp+supwN/+GEpGDk8Slcb7ST9DlEhGjGDn6ZHBaCeAJbuJyQM0ymPtX T+4aOmO50BQdfgiOTvO6asLDXU1jBDgqWF3bqkQgIbDNQEN5riEhifVt0WlvPRKCpvxY 7jlYH8y4eEdi7lJiIb3ssDdIwlmenTDtkZVik= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=i1DLe96g/uhVXRfbnGfQvMfcMjbhhu5P1fVFM7NCmZIsHeF0p4luwJhkXs+oiwPisH axWTFfKKbRuOMWxMpvd+IKFXqc4JUPmsoCUE6R/mG0ZDowF3bETWSgr/GwkwnYQIUgWx 011nKVGjwUsYb5mYLQkYHZQ4Edsk/bLfIEsvo= MIME-Version: 1.0 Received: by 10.224.78.41 with SMTP id i41mr3436351qak.133.1279491035090; Sun, 18 Jul 2010 15:10:35 -0700 (PDT) Received: by 10.229.75.85 with HTTP; Sun, 18 Jul 2010 15:10:34 -0700 (PDT) In-Reply-To: <441D899DB0E264409344AFC3D1998EEA01229EA5@email4.us.syncsort.com> References: <441D899DB0E264409344AFC3D1998EEA01229E9B@email4.us.syncsort.com> <441D899DB0E264409344AFC3D1998EEA01229EA5@email4.us.syncsort.com> Date: Sun, 18 Jul 2010 15:10:34 -0700 Message-ID: Subject: Re: Error: java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.get From: Ted Yu To: mapreduce-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=00c09f8e59fea3d28c048bb0b8e9 X-Virus-Checked: Checked by ClamAV on apache.org --00c09f8e59fea3d28c048bb0b8e9 Content-Type: text/plain; charset=ISO-8859-1 Have you looked at https://issues.apache.org/jira/browse/MAPREDUCE-969 ? It points to https://issues.apache.org/jira/browse/HADOOP-4744 See if HADOOP-4744 is contained in the hadoop version you're running. I am reading cdh3b2 source code. Please confirm that *ReduceTask.java:2683 *is this line: List loc = mapLocations.get(host); If so, you can print the value of host, or value of URI u which host is derived. Cheers On Fri, Jul 16, 2010 at 9:42 AM, Chinni, Ravi wrote: > Hadoop version: 0.20.2 > > > > *From:* Ted Yu [mailto:yuzhihong@gmail.com] > *Sent:* Friday, July 16, 2010 12:34 PM > *To:* mapreduce-user@hadoop.apache.org > *Subject:* Re: Error: java.lang.NullPointerException at > java.util.concurrent.ConcurrentHashMap.get > > > > What version of hadoop are you using ? > > On Fri, Jul 16, 2010 at 8:56 AM, Chinni, Ravi > wrote: > > I am trying to run the terasort example with a small input on a 4 node > cluster. I just did the minimal configuration (fs.default.name, master, > slaves etc.), but did not do anything specific to terasort. I am getting the > following java.lang.NullPointerException on running terasort example: > > > > > > 10/07/16 11:48:29 INFO terasort.TeraSort: starting > > 10/07/16 11:48:29 INFO mapred.FileInputFormat: Total input paths to process > : 1 > > 10/07/16 11:48:29 INFO util.NativeCodeLoader: Loaded the native-hadoop > library > > 10/07/16 11:48:29 INFO zlib.ZlibFactory: Successfully loaded & initialized > native-zlib library > > 10/07/16 11:48:29 INFO compress.CodecPool: Got brand-new compressor > > Making 1 from 10 records > > Step size is 10.0 > > 10/07/16 11:48:30 INFO mapred.JobClient: Running job: job_201006251254_0010 > > 10/07/16 11:48:31 INFO mapred.JobClient: map 0% reduce 0% > > 10/07/16 11:48:38 INFO mapred.JobClient: map 50% reduce 0% > > 10/07/16 11:48:39 INFO mapred.JobClient: map 100% reduce 0% > > 10/07/16 11:48:43 INFO mapred.JobClient: Task Id : > attempt_201006251254_0010_r_000000_0, Status : FAILED > > *Error: java.lang.NullPointerException* > > * at > java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)* > > * at > org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683) > * > > * at > org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605) > * > > > > 10/07/16 11:48:49 INFO mapred.JobClient: Task Id : > attempt_201006251254_0010_r_000000_1, Status : FAILED > > Error: java.lang.NullPointerException > > at > java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768) > > at > org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683) > > at > org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605) > > > > 10/07/16 11:48:57 INFO mapred.JobClient: Task Id : > attempt_201006251254_0010_r_000000_2, Status : FAILED > > Error: java.lang.NullPointerException > > at > java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768) > > at > org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683) > > at > org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605) > > > > > > > > Also, I got similar error from reduce jobs on running a streaming > map-reduce application. In this case also, the map tasks completed > successfully. > > > > > > Any pointers are appreciated. > > > > Thanks, > > *Ravi Chinni* > > > > > > > _____________________________________________________________________________ > > > > ATTENTION: > > > > The information contained in this message (including any files transmitted > with this message) may contain proprietary, trade secret or other > confidential and/or legally privileged information. Any pricing information > contained in this message or in any files transmitted with this message is > always confidential and cannot be shared with any third parties without > prior written approval from Syncsort. This message is intended to be read > only by the individual or entity to whom it is addressed or by their > designee. If the reader of this message is not the intended recipient, you > are on notice that any use, disclosure, copying or distribution of this > message, in any form, is strictly prohibited. If you have received this > message in error, please immediately notify the sender and/or Syncsort and > destroy all copies of this message in your possession, custody or control. > > > --00c09f8e59fea3d28c048bb0b8e9 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Have you looked at https://issues.apache.org/jira/browse/MAPREDUCE-969 ?
It poi= nts to https:= //issues.apache.org/jira/browse/HADOOP-4744

See if HADOOP-4744 is contained in the hadoop version you're runnin= g.

I am reading cdh3b2 source code. Please confirm that ReduceTask.java:2683 is this line:
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 List<MapOutputLocation> loc = =3D mapLocations.get(host);
If so, you can print the value of host, or v= alue of URI u which host is derived.

Cheers

On Fri, Jul 16, 2010 at 9:42 AM, Chinni, Ravi &= lt;rchinni@syncsort.com> wrote:

Hadoop version: 0.20.2

=A0

From:= Ted Yu [mailto:yuzhihong@= gmail.com]
Sent: Friday, July 16, 2010 12:34 PM
To: mapreduce-user@hadoop.apache.org
Subject: Re: Error: java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.get

=A0

What version of hadoo= p are you using ?

On Fri, Jul 16, 2010 at 8:56 AM, Chinni, Ravi <rchinni@syncsort.co= m> wrote:

I am trying= to run the terasort example with a small input on a 4 node cluster. I just did the minimal configuration (fs.default.name, master, slaves etc.), but did not do anything specific to terasort. I am getting th= e following java.lang.NullPointerException on running terasort example:

=A0<= /p>

=A0<= /p>

10/07/16 11= :48:29 INFO terasort.TeraSort: starting

10/07/16 11= :48:29 INFO mapred.FileInputFormat: Total input paths to process : 1

10/07/16 11= :48:29 INFO util.NativeCodeLoader: Loaded the native-hadoop library

10/07/16 11= :48:29 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library

10/07/16 11= :48:29 INFO compress.CodecPool: Got brand-new compressor

Making 1 fr= om 10 records

Step size i= s 10.0

10/07/16 11= :48:30 INFO mapred.JobClient: Running job: job_201006251254_0010

10/07/16 11= :48:31 INFO mapred.JobClient:=A0 map 0% reduce 0%

10/07/16 11= :48:38 INFO mapred.JobClient:=A0 map 50% reduce 0%

10/07/16 11= :48:39 INFO mapred.JobClient:=A0 map 100% reduce 0%

10/07/16 11= :48:43 INFO mapred.JobClient: Task Id : attempt_201006251254_0010_r_000000_0, Status : FAILED

Error: j= ava.lang.NullPointerException

=A0=A0= =A0=A0=A0=A0=A0 at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)

=A0=A0= =A0=A0=A0=A0=A0 at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapE= ventsThread.getMapCompletionEvents(ReduceTask.java:2683)

=A0=A0= =A0=A0=A0=A0=A0 at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(Red= uceTask.java:2605)

=A0<= /p>

10/07/16 11= :48:49 INFO mapred.JobClient: Task Id : attempt_201006251254_0010_r_000000_1, Status : FAILED

Error: java= .lang.NullPointerException

=A0=A0=A0= =A0=A0=A0=A0 at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)

=A0=A0=A0= =A0=A0=A0=A0 at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapC= ompletionEvents(ReduceTask.java:2683)

=A0=A0=A0= =A0=A0=A0=A0 at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(Red= uceTask.java:2605)

=A0<= /p>

10/07/16 11= :48:57 INFO mapred.JobClient: Task Id : attempt_201006251254_0010_r_000000_2, Status : FAILED

Error: java= .lang.NullPointerException

=A0=A0=A0= =A0=A0=A0=A0 at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)

=A0=A0=A0= =A0=A0=A0=A0 at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapC= ompletionEvents(ReduceTask.java:2683)

=A0=A0=A0= =A0=A0=A0=A0 at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(Red= uceTask.java:2605)

=A0<= /p>

=A0<= /p>

=A0<= /p>

Also, I got= similar error from reduce jobs on running a streaming map-reduce application. In this case also, the map tasks complete= d successfully.

=A0<= /p>

=A0<= /p>

Any pointer= s are appreciated.

=A0<= /p>

Thanks,

Ravi Chinni

=A0<= /p>

=A0

____________________________________________________= _________________________

=A0

ATTENTION:<= /p>

=A0

The information contained in this message (including any files transmitted with this message) may contain proprietary, trade secret or other=A0 confidentia= l and/or legally privileged information. Any pricing information contained in this message or in any files transmitted with this message=A0is always confidential and cannot be shared with any third parties without prior writ= ten approval from Syncsort. This message is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice tha= t any use, disclosure, copying or distribution of this message, in any form, = is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or Syncsort=A0and destroy all copies of t= his message in your possession, custody or control.

=A0


--00c09f8e59fea3d28c048bb0b8e9--