Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3361A10D67 for ; Fri, 12 Jul 2013 12:35:24 +0000 (UTC) Received: (qmail 55858 invoked by uid 500); 12 Jul 2013 12:35:18 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 55690 invoked by uid 500); 12 Jul 2013 12:35:17 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 55683 invoked by uid 99); 12 Jul 2013 12:35:16 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 12 Jul 2013 12:35:16 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of prameshfrz@gmail.com designates 209.85.192.175 as permitted sender) Received: from [209.85.192.175] (HELO mail-pd0-f175.google.com) (209.85.192.175) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 12 Jul 2013 12:35:12 +0000 Received: by mail-pd0-f175.google.com with SMTP id 4so8547526pdd.6 for ; Fri, 12 Jul 2013 05:34:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=syZ6Y1ItOQ75Q2GwGYCJ0g3AWAaibNBFKPltAFJcYNY=; b=0gXg2AXp1K+GSdsYNF3zVzJK7JdFR/R4NLpaxlVUx1xvsmZTbAIx3JJzbD9ae1XHf3 ooSwQyPWBKapkRmr8WqR9WeUuAA68VpheTem6X16M5PbCMENzTw2ul2rPw0x8G041hl7 uFB6wO8rf3W7iqeMZ1kDekYzstH4476r9XB/TJRLfMB53xEAJ/LWmCqa4AVAX5v1Ieov 13IMxLZoG5h7EZhkgATF7VnCVwrobn/ubTbtSWc9uqagJZY8vyM+KBMeCVaoNKp6StBi 5FJsk/mV/qHqvU3op7qq9zPPJ+DAtxNmki5HEzE3afrNrprcsbEfr/ksRm6A8l27YN3r Iw0A== MIME-Version: 1.0 X-Received: by 10.68.103.131 with SMTP id fw3mr15216677pbb.65.1373632492071; Fri, 12 Jul 2013 05:34:52 -0700 (PDT) Received: by 10.66.84.131 with HTTP; Fri, 12 Jul 2013 05:34:52 -0700 (PDT) In-Reply-To: <06006DDA5A27D541991944AC4117E7A96E1D5F19@szxeml560-mbx.china.huawei.com> References: <06006DDA5A27D541991944AC4117E7A96E1D5EDE@szxeml560-mbx.china.huawei.com> <06006DDA5A27D541991944AC4117E7A96E1D5F19@szxeml560-mbx.china.huawei.com> Date: Fri, 12 Jul 2013 18:04:52 +0530 Message-ID: Subject: Re: Taktracker in namenode failure From: Ram To: "user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=047d7b5d99e1bdf09404e14fbc9a X-Virus-Checked: Checked by ClamAV on apache.org --047d7b5d99e1bdf09404e14fbc9a Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi, The problem is with jar file only, to check run any other MR job or sample wordcount job on namenode tasktracker, if it is running no problem with namenode tasktracker, if not running there may be problem with tasktracker configuration, then compare with other node tasktracker configuration. i.e tasktracker configuration means mapred configuration. Hi, From, Ramesh. On Fri, Jul 12, 2013 at 3:37 PM, Devaraj k wrote: > I think, there is mismatch of jar=92s coming in the classpath for the ma= p > tasks when it runs in different machines. You can find out this, by givin= g > some unique name for your Mapper class, Job Submit class and then submit > the Job.**** > > ** ** > > Thanks**** > > Devaraj k**** > > ** ** > > *From:* Ramya S [mailto:ramyas@suntecgroup.com] > *Sent:* 12 July 2013 15:27 > *To:* user@hadoop.apache.org > *Subject:* RE: Taktracker in namenode failure**** > > ** ** > > Both the map output value class configured and the output value written > from the mapper is Text class. So there is no mismatch in the value class= . > **** > > **** > > But when the same MR program is run with 2 tasktrackers(without > tasktracker in namenode) exception is not occuring.**** > > **** > > The problem is only with the tasktracker running in the namenode.**** > > **** > > **** > > **** > > *Thanks & Regards***** > > **** > > *Ramya.S***** > > ** ** > ------------------------------ > > *From:* Devaraj k [mailto:devaraj.k@huawei.com ] > *Sent:* Fri 7/12/2013 3:04 PM > *To:* user@hadoop.apache.org > *Subject:* RE: Taktracker in namenode failure**** > > Could you tell, what is the Map Output Value class you are configuring > while submitting Job and what is the type of the value writing from the > Mapper. If both of these mismatches then it will trow the below error.***= * > > **** > > Thanks**** > > Devaraj k**** > > **** > > *From:* Ramya S [mailto:ramyas@suntecgroup.com ] > *Sent:* 12 July 2013 14:46 > *To:* user@hadoop.apache.org > *Subject:* Taktracker in namenode failure**** > > **** > > Hi,**** > > **** > > Why only tasktracker in namenode faill during job execution with error.*= * > ** > > I have attached the snapshot of error screen with this mail**** > > java.io.IOException: Type mismatch in value from map: expected org.apache= .hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable**** > > at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTa= sk.java:1019)**** > > at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapT= ask.java:691)**** > > at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskI= nputOutputContext.java:80)**** > > at WordCount$TokenizerMapper.map(WordCount.java:30)**** > > at WordCount$TokenizerMapper.map(WordCount.java:19)**** > > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)**** > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764= )**** > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)**** > > at org.apache.hadoop.mapred.Child$4.run(Child.java:255)**** > > at java.security.AccessController.doPrivileged(Native Method)**** > > at javax.security.auth.Subject.doAs(Subject.java:416)**** > > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroup= Information.java:1121)**** > > at org.apache.hadoop.mapred.Child.main(Child.java:249)**** > > **** > > but this same task is reassigned to another tasktracker and getting exec= uted. why?**** > > **** > > **** > > *Best Regards,***** > > *Ramya***** > --047d7b5d99e1bdf09404e14fbc9a Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
Hi,
=A0 =A0 The problem is with jar file only, t= o check run any other MR job or sample wordcount job on namenode tasktracke= r, if it is running no problem with namenode tasktracker, if not running th= ere may be problem with tasktracker configuration, then compare with other = node tasktracker configuration. i.e tasktracker configuration means mapred = configuration.

Hi= ,
=A0 =A0


From,
Ramesh.




On Fri, Jul 12, 2013 at 3:37 PM, Devaraj= k <devaraj.k@huawei.com> wrote:

I think, there is mismatc= h of jar=92s coming in the classpath for the map tasks when it runs in diff= erent machines. You can find out this, by giving some unique name for your Mapper class, Job Submit class and then submit the Job.

=A0<= /p>

Thanks

Devaraj k

=A0<= /p>

From: Ramya S = [mailto:ramyas@= suntecgroup.com]

Sent: 12 July 2013 15:27
To: user= @hadoop.apache.org
Subject: RE: Taktracker in namenode fai= lure

=A0

Both the=A0map output value =A0class conf= igured and the output value=A0 written from the mapper is=A0Text class. So = there is no mismatch in the value class.

=A0

=A0But when the same MR program is run wi= th 2 tasktrackers(without tasktracker in namenode) exception is not occurin= g.

=A0

The problem is only=A0with the tasktracke= r running in the namenode.

=A0

=A0

=A0

Thanks & Regards

=A0

Ra= mya.S

=A0

From:<= /span> Devaraj k [mailto:devaraj.k@huawei.com]
Sent: Fri 7/12/2013 3:04 PM
To: user= @hadoop.apache.org
Subject: RE: Taktracker in namenode failure

Could= you tell, what is the Map Output Value class you are configuring while sub= mitting Job and what is the type of the value writing from the Mapper. If b= oth of these mismatches then it will trow the below error.

=A0

Thank= s

Devar= aj k

=A0

From:= Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 14:46
To: user= @hadoop.apache.org
Subject: Taktracker in namenode failure

=A0

Hi,

=A0

Why only tasktracke= r in namenode faill during=A0 job execution with error.

I have attached the= snapshot of error screen with this mail

java.io.IOException: Type mismatch in value from map: expected org.apa=
che.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable=

=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.mapred.MapTask$MapOutputBuf=
fer.collect(MapTask.java:1019)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.mapred.MapTask$NewOutputCol=
lector.write(MapTask.java:691)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.mapreduce.TaskInputOutputCo=
ntext.write(TaskInputOutputContext.java:80)
=A0=A0=A0=A0=A0=A0=A0 at WordCount$TokenizerMapper.map(WordCount.java:=
30)
=A0=A0=A0=A0=A0=A0=A0 at WordCount$TokenizerMapper.map(WordCount.java:=
19)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.mapreduce.Mapper.run(Mapper=
.java:144)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.mapred.MapTask.runNewMapper=
(MapTask.java:764)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.mapred.MapTask.run(MapTask.=
java:370)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.mapred.Child$4.run(Child.ja=
va:255)
=A0=A0=A0=A0=A0=A0=A0 at java.security.AccessController.doPrivileged(N=
ative Method)
=A0=A0=A0=A0=A0=A0=A0 at javax.security.auth.Subject.doAs(Subject.java=
:416)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.security.UserGroupInformati=
on.doAs(UserGroupInformation.java:1121)
=A0=A0=A0=A0=A0=A0=A0 at org.apache.hadoop.mapred.Child.main(Child.jav=
a:249)
=A0
but=A0 this same task is reassigned to another tasktracker and getting=
 executed. why?

=A0

=A0

Best Regards,

Ramya


--047d7b5d99e1bdf09404e14fbc9a--