Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7C18BDD7A for ; Fri, 31 Aug 2012 21:12:05 +0000 (UTC) Received: (qmail 86816 invoked by uid 500); 31 Aug 2012 21:12:00 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 86703 invoked by uid 500); 31 Aug 2012 21:12:00 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 86693 invoked by uid 99); 31 Aug 2012 21:12:00 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 31 Aug 2012 21:12:00 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [216.32.181.185] (HELO ch1outboundpool.messaging.microsoft.com) (216.32.181.185) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 31 Aug 2012 21:11:51 +0000 Received: from mail141-ch1-R.bigfish.com (10.43.68.248) by CH1EHSOBE016.bigfish.com (10.43.70.66) with Microsoft SMTP Server id 14.1.225.23; Fri, 31 Aug 2012 21:11:28 +0000 Received: from mail141-ch1 (localhost [127.0.0.1]) by mail141-ch1-R.bigfish.com (Postfix) with ESMTP id A2405400174 for ; Fri, 31 Aug 2012 21:11:28 +0000 (UTC) X-Forefront-Antispam-Report: CIP:163.181.249.109;KIP:(null);UIP:(null);IPV:NLI;H:ausb3twp02.amd.com;RD:none;EFVD:NLI X-SpamScore: 0 X-BigFish: VPS0(zzc89bhc85dhzz1202hzz8275bh8275dhz2dh668h839hd25hf0ah107ah1155h) Received: from mail141-ch1 (localhost.localdomain [127.0.0.1]) by mail141-ch1 (MessageSwitch) id 1346447486927899_19925; Fri, 31 Aug 2012 21:11:26 +0000 (UTC) Received: from CH1EHSMHS030.bigfish.com (snatpool2.int.messaging.microsoft.com [10.43.68.232]) by mail141-ch1.bigfish.com (Postfix) with ESMTP id E0DCE1A0043 for ; Fri, 31 Aug 2012 21:11:26 +0000 (UTC) Received: from ausb3twp02.amd.com (163.181.249.109) by CH1EHSMHS030.bigfish.com (10.43.70.30) with Microsoft SMTP Server id 14.1.225.23; Fri, 31 Aug 2012 21:11:25 +0000 X-WSS-ID: 0M9N1IZ-02-0KF-02 X-M-MSG: Received: from sausexedgep02.amd.com (sausexedgep02-ext.amd.com [163.181.249.73]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ausb3twp02.amd.com (Axway MailGate 3.8.1) with ESMTP id 2B571C80C0 for ; Fri, 31 Aug 2012 16:11:22 -0500 (CDT) Received: from SAUSEXDAG05.amd.com (163.181.55.6) by sausexedgep02.amd.com (163.181.36.59) with Microsoft SMTP Server (TLS) id 8.3.192.1; Fri, 31 Aug 2012 16:11:44 -0500 Received: from SAUSEXDAG02.amd.com ([fe80::ed3c:9786:3083:dd99]) by sausexdag05.amd.com ([fe80::94d8:2d17:10c5:6039%20]) with mapi id 14.01.0323.003; Fri, 31 Aug 2012 16:11:23 -0500 From: "Devireddy, Bhaskar" To: "user@hadoop.apache.org" Subject: Shuffle BAD_ID errors Thread-Topic: Shuffle BAD_ID errors Thread-Index: Ac2HvSeQPY7p4n1TR2GFejbW9yythA== Date: Fri, 31 Aug 2012 21:11:23 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.224.9.104] Content-Type: multipart/alternative; boundary="_000_F9E7EB9FCF93FD4990932DAE8E7C064901E980C1sausexdag02amdc_" MIME-Version: 1.0 X-OriginatorOrg: amd.com X-Virus-Checked: Checked by ClamAV on apache.org --_000_F9E7EB9FCF93FD4990932DAE8E7C064901E980C1sausexdag02amdc_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable We are noticing shuffle BAD_ID errors using hadoop trunk and Yarn. The stac= k traces are as follows and any ideas why these exceptions are happening an= d how to fix them. java.lang.IllegalArgumentException: Encoded byte size for String was -42, w= hich is outside of 0..1000 range. at org.apache.hadoop.io.WritableUtils.readStringSafely(WritableUtils= .java:477) at org.apache.hadoop.mapreduce.task.reduce.ShuffleHeader.readFields(= ShuffleHeader.java:60) at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fet= cher.java:363) at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetc= her.java:323) at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:= 179) java.lang.IllegalArgumentException: TaskAttemptId string : =EF=BF=BD=EF=BF= =BD^Z=EF=BF=BD=EF=BF=BDm%attempt_1346265984657_000 is not properly formed at org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.j= ava:187) at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fet= cher.java:364) at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetc= her.java:323) at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:= 179) java.lang.IllegalArgumentException: TaskAttemptId string : @ is not properl= y formed at org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.j= ava:187) at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fet= cher.java:364) at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetc= her.java:323) at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:= 179) Thanks Bhaskar Devireddy --_000_F9E7EB9FCF93FD4990932DAE8E7C064901E980C1sausexdag02amdc_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

We are noticing shuffle BAD_ID errors using hadoo= p trunk and Yarn. The stack traces are as follows and any ideas why these e= xceptions are happening and how to fix them.

 

java.lang.IllegalArgumentException: Encoded byte = size for String was -42, which is outside of 0..1000 range.

       at org.apach= e.hadoop.io.WritableUtils.readStringSafely(WritableUtils.java:477)

       at org.apach= e.hadoop.mapreduce.task.reduce.ShuffleHeader.readFields(ShuffleHeader.java:= 60)

       at org.apach= e.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:363)=

       at org.apach= e.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:323)<= /o:p>

       at org.apach= e.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:179)

 

 

java.lang.IllegalArgumentException: TaskAttemptId= string : =EF=BF=BD=EF=BF=BD^Z=EF=BF=BD=EF=BF=BDm%attempt_1346265984657_000= is not properly formed

       at org.apach= e.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:187)=

       at org.apach= e.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:364)=

       at org.apach= e.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:323)<= /o:p>

       at org.apach= e.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:179)

 

 

java.lang.IllegalArgumentException: TaskAttemptId= string : @ is not properly formed

       at org.apach= e.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:187)=

       at org.apach= e.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:364)=

       at org.apach= e.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:323)<= /o:p>

       at org.apach= e.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:179)

 

Thanks

Bhaskar Devireddy

 

--_000_F9E7EB9FCF93FD4990932DAE8E7C064901E980C1sausexdag02amdc_--