Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D2637F37C for ; Fri, 22 Mar 2013 06:39:11 +0000 (UTC) Received: (qmail 26595 invoked by uid 500); 22 Mar 2013 06:39:06 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 26189 invoked by uid 500); 22 Mar 2013 06:39:06 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 26148 invoked by uid 99); 22 Mar 2013 06:39:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Mar 2013 06:39:05 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [98.138.229.50] (HELO nm32-vm2.bullet.mail.ne1.yahoo.com) (98.138.229.50) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Mar 2013 06:38:57 +0000 Received: from [98.138.226.178] by nm32.bullet.mail.ne1.yahoo.com with NNFMP; 22 Mar 2013 06:38:36 -0000 Received: from [98.138.88.237] by tm13.bullet.mail.ne1.yahoo.com with NNFMP; 22 Mar 2013 06:38:36 -0000 Received: from [127.0.0.1] by omp1037.mail.ne1.yahoo.com with NNFMP; 22 Mar 2013 06:38:36 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 766505.55375.bm@omp1037.mail.ne1.yahoo.com Received: (qmail 67955 invoked by uid 60001); 22 Mar 2013 06:38:36 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1363934316; bh=7DIT4JbgoFsVjJOx0jHX6xvtaonuRK8Gaykb2EkNlVI=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=Jf75f3uHLioYrFARd/jDxFfQhZiKzWBWALGKjSZGlPA7QD74lag0Qz6v2Vl4OrpENnPdjUf/d+H+MRekZSBs7QcQM56EWkzmAn0/QW2LQJvq3DAcJwbCd3KY5OFCRhke5zpjBwb1nubtpM/beflgrF1SX9MwZ5FM/Kv8BOWIlx0= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=xr86ZZt/iWP4txTyT8r3YPuYGcNI4qnMlgTCTTlx1vWnHTuY7n2z9CP23zUCDyvyL5NyPnUM2hkbJyT+F+T2y9CHBHsceoQiNW+NNk5nX6Wa/g2imbDo+vf7Lb8PdSfuOjTLOzFQKosD3eL0k7Xa/x+JrSrBuoMXZjAyZf/t12c=; X-YMail-OSG: RhcxAKMVM1l4UAiG_kbY1Olm2KIZxiTGYRGpfkMm2hvBkUE iOERgOwHl7lEhQoa_yAga5glhrriAiNgIQNNdgQuBGQjveqI3FELaxYgPnb9 xRLk7PtLMzhMtRZ8gdGYmhWTs7NEwStxUfRwQlFJwv953eUe_ptpjs43.aNX QC86WPQei7.o.0ZVp.5sPQUnkW8Ms5c9ucYOSxYGBekfl_Nte.ryQ5sfUE0U C_xrRLi8kH9pEUuruT3.h_7GziuWmWmqkY5KIhVCVTU4lEJaKrFCH1tXLmdQ YUhDzoh.5QJu57Cp3l0CPWW.mOHOtanZDuinAQVgRrjoEjj8GsYPp3hF2j4N 30R2JOpksJ.4K21CVxrzvvXzZ3x1Tnx1wb11_s9KzjTs5yyeUlsTD4jPDA8U egY9vi_DbaQokLGizBkj.cnEwgFrsnCh9azOC77A_rqln5xpFV_a1VbBwT6G c6s9aEWdtkV.D.RC2Zn5eeFTJSrVz_tchcCnoCky6qiMfRNywsXXV9lA8L6P Y3b9qAA-- Received: from [202.76.242.44] by web122403.mail.ne1.yahoo.com via HTTP; Thu, 21 Mar 2013 23:38:36 PDT X-Rocket-MIMEInfo: 002.001,SGksCsKgCkkgYW0gdHJ5aW5nIHRvIGRvIGFuIG91dGVyIGpvaW4gb24gdG8gaW5wdXQgZmlsZXMuCkNhbiBhbnlvbmUgaGVscCBtZSB0byBmaW5kIG91dCB0aGUgcHJvYmxlbSBoZXJlPz8KwqAKQnV0IHdoaWxlIGpvaW5pbmcgdGhlIFR1cGxlV3JpdGFibGUgdmFsdWUgaW4gdGhlIG1hcHBlciBpcyBub3QgZ2V0dGluZyBjbGVhbmVkIHVwIGFuZCBzbyBpcyB1c2luZyB0aGUgcHJldmlvdXMgdmFsdWVzIG9mIGEgZGlmZmVyZW50CmtleS4KwqAKVGhlIGNvZGUgSSB1c2VkIGlzIDogKMKgIOKAmHBsaXN04oCZIGkBMAEBAQE- X-Mailer: YahooMailWebService/0.8.138.524 Message-ID: <1363934316.66704.YahooMailNeo@web122403.mail.ne1.yahoo.com> Date: Thu, 21 Mar 2013 23:38:36 -0700 (PDT) From: devansh kumar Reply-To: devansh kumar Subject: TupleWritable value in mapper Not getting cleaned up ( using CompositeInputFormat ) To: "hdfs-dev@hadoop.apache.org" , "mapreduce-dev@hadoop.apache.org" , "user@hadoop.apache.org" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="942553102-2084872953-1363934316=:66704" X-Virus-Checked: Checked by ClamAV on apache.org --942553102-2084872953-1363934316=:66704 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi,=0A=C2=A0=0AI am trying to do an outer join on to input files.=0ACan any= one help me to find out the problem here??=0A=C2=A0=0ABut while joining the= TupleWritable value in the mapper is not getting cleaned up and so is usin= g the previous values of a different=0Akey.=0A=C2=A0=0AThe code I used is := (=C2=A0 =E2=80=98plist=E2=80=99 is containing the set=0Aof paths to be tak= en as input )=0A=C2=A0=0AjobConf.setInputFormat(CompositeInputFormat.class)= ;=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 =0AjobConf.set("mapred.join.expr",=0ACompositeInputFormat.comp= ose(op, inputFormatClass,plist.toArray(newPath[0])));=0AjobConf.setOutputFo= rmat(outputFormatClass);=0A=C2=A0=0Ainp1:=0A=C2=A0=0Aanil1=C2=A0=C2=A0=C2= =A0=C2=A0=0A10=0Aanil2=C2=A0=C2=A0=C2=A0=C2=A0=0A20=0Aanil3=C2=A0=C2=A0=C2= =A0=C2=A0=0A30=0Adev1=C2=A0=C2=A0=C2=A0=C2=A0 40=0Adev2=C2=A0=C2=A0=C2=A0= =C2=A0 50=0A=C2=A0=0Ainp2:=0A=C2=A0=0Aanil1=C2=A0=C2=A0=C2=A0=C2=A0=0A100= =0Adev1=C2=A0=C2=A0=C2=A0=C2=A0=0A400=0Adev2=C2=A0=C2=A0=C2=A0=C2=A0=0A500= =0Adev3=C2=A0=C2=A0=C2=A0=C2=A0=0A600=0A=C2=A0=0A=C2=A0=0Aouter join output= :=0A=C2=A0=0Aanil1=C2=A0=C2=A0=C2=A0=C2=A0=0A10,100=0Aanil2=C2=A0=C2=A0=C2= =A0=C2=A0=0A20,100=0Aanil3=C2=A0=C2=A0=C2=A0=C2=A0=0A30,100=0Adev1=C2=A0=C2= =A0=C2=A0=C2=A0=0A40,400=0Adev2=C2=A0=C2=A0=C2=A0=C2=A0=0A50,500=0Adev3=C2= =A0=C2=A0=C2=A0=C2=A0=0A50,600=0A=C2=A0=0AActually It should be, right?=0A= =C2=A0=0Aanil1=C2=A0=C2=A0=C2=A0=C2=A0 10,100=0Aanil2=C2=A0=C2=A0=C2=A0=C2= =A0 20=0Aanil3=C2=A0=C2=A0=C2=A0=C2=A0 30=0Adev1=C2=A0=C2=A0=C2=A0=C2=A0 40= ,400=0Adev2=C2=A0=C2=A0=C2=A0=C2=A0 50,500=0Adev3=C2=A0=C2=A0=C2=A0=C2=A0 6= 00=0A=C2=A0=0ARegards,=0ADevansh Rusia --942553102-2084872953-1363934316=:66704 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Hi,
=0A=0A
 
=0A=0A
I am trying to do an outer join on t= o input files.
Can anyone h= elp me to find out the problem here??
=0A=0A
<= o:p> 
=0A=0A
But while joining the = TupleWritable = value in the mapper=0Ais not getting cleaned up and so is using the = previous values of a different=0Akey.
=0A=0A
 
=0A=0A
The code I = used is : (  =E2=80=98plist=E2=80=99 is containing the set=0Aof paths = to be taken as input )
=0A=0A
=  
=0A=0A
jobConf.setInputFormat(CompositeInputFor= mat.class);        &= nbsp;     =0A
=0A=0A
jobConf.set("mapred.join.expr",=0ACompositeInputFormat.comp= ose(op, inputFormatClass,plist.toArray(new= Path[0])));<= o:p>
=0A=0A
jobConf.setOutputFormat(outputForma= tClass);
=0A=0A
 <= /o:p>
=0A=0A
inp1:
=0A=0A
=  
=0A=0A
anil1   =  =0A10
=0A=0A
= anil2    =0A20
=0A=0A
anil3    =0A30
=0A=0A
dev1  &nbs= p;  40
=0A=0A
= dev2     50
=0A=0A
 
=0A=0A
inp2:
=0A=0A
 =0A=0A
anil= 1    =0A100
=0A=0A
dev1    = ;=0A400
=0A=0A
dev2    =0A500
=0A=0A
= dev3    =0A600
=0A=0A
 
=0A=0A
=  
=0A=0A
outer join output:
=0A=0A =
=0A=0A
anil1    =0A10,100
=0A=0A<= div class=3D"MsoNormal" style=3D"background-position: initial initial; back= ground-repeat: initial initial;">anil2 &nb= sp;  =0A20,100
=0A=0A
anil3    =0A30= ,100
=0A=0A
dev1    =0A40,400
=0A=0A
= dev2    =0A50,500
=0A=0A
dev3   = ; =0A50,600
=0A=0A
 
=0A=0A
Actually It should = be, right?
=0A=0A
<= span style=3D"font-size:10.0pt;=0Afont-family:"Arial","sans-= serif";color:#222222"> 
=0A=0A
anil1     10= ,100
=0A=0A
anil2     20
=0A=0A<= div class=3D"MsoNormal">anil3   =   30
=0A=0A
dev1     40,400=0A=0A
dev2 &nbs= p;   50,500
=0A=0A
dev3     600
=0A=0A
 
=0A=0A
Regards,
=0A=0A
Devansh Rusia
--942553102-2084872953-1363934316=:66704--