hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohamed Trad <riadh.t...@inria.fr>
Subject Re: puzzled at the output
Date Tue, 02 Oct 2012 17:35:21 GMT
I agree with Bertrand. Try disabling the combiner.

Envoyé de mon iPhone

Le 2 oct. 2012 à 19:02, Bertrand Dechoux <dechouxb@gmail.com> a écrit :

> Combiner? And you are only using 'Text' as type?
> 
> Please do a real test with a specified input. We can only guess.
> 
> Bertrand
> 
> On Tue, Oct 2, 2012 at 6:52 PM, Chris Nauroth <cnauroth@hortonworks.com> wrote:
>> Is there also a Mapper?  Is there any chance that logic in the Mapper wrapped the
values with the tags too, so that the records were already wrapped when they entered the reducer
logic?
>> 
>> Thank you,
>> --Chris
>> 
>> On Tue, Oct 2, 2012 at 9:01 AM, Kartashov, Andy <Andy.Kartashov@mpac.ca> wrote:
>>> I want:
>>> 
>>> Key  <tag>
>>>           Value1
>>>           Value2
>>>         </tag>
>>> 
>>> I get double tags:
>>> Key  <tag><tag>
>>>           Value1
>>>           Value2
>>>         </tag></tag>
>>> 
>>> Here is my last proposition that also failed in Reduce.
>>> 
>>> ...
>>> public void reduce (.....
>>> ....
>>> StringBuilder sb = new StringBuilder();
>>> while (values.hasNext()){
>>>      sb.append(values.next().toString());
>>>     }
>>> 
>>> output.collect(key, new Text("\n<tag>"+sb.toString()+"</tag>" ));
>>> 
>>> Kind regards,
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> -----Original Message-----
>>> From: Harsh J [mailto:harsh@cloudera.com]
>>> Sent: Tuesday, October 02, 2012 10:03 AM
>>> To: user@hadoop.apache.org
>>> Subject: Re: puzzled at the output
>>> 
>>> Hi,
>>> 
>>> Could you clarify your post to show what you expect your code to have actually
printed and what it has printed?
>>> 
>>> On Tue, Oct 2, 2012 at 7:01 PM, Kartashov, Andy <Andy.Kartashov@mpac.ca>
wrote:
>>> > Guys, have been stretching my head for the past couple of days.  Why
>>> > are my tags duplicated while the content they wrap around i.e.my
>>> > StringBuilder sb is not?
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > My Reduce code is:
>>> >
>>> > while (values.hasNext()){
>>> >
>>> >     sb.append(values.next().toString());
>>> >
>>> >    }
>>> >
>>> >     output.collect(key,new
>>> > Text("\n<aaa>\n"+sb.toString()+"\n</bbbb>\n" ));
>>> >
>>> >
>>> >
>>> > The out put:
>>> >
>>> > 2459
>>> >
>>> > <aaa>
>>> >
>>> >
>>> >
>>> > <aaa>
>>> >
>>> >         <value....
>>> >
>>> >                 .....
>>> >
>>> >                 ......
>>> >
>>> > </bbbb>
>>> >
>>> >
>>> >
>>> > </bbbb>
>>> >
>>> >
>>> >
>>> > NOTICE: This e-mail message and any attachments are confidential,
>>> > subject to copyright and may be privileged. Any unauthorized use,
>>> > copying or disclosure is prohibited. If you are not the intended
>>> > recipient, please delete and contact the sender immediately. Please
>>> > consider the environment before printing this e-mail. AVIS : le
>>> > présent courriel et toute pièce jointe qui l'accompagne sont
>>> > confidentiels, protégés par le droit d'auteur et peuvent être couverts
>>> > par le secret professionnel. Toute utilisation, copie ou divulgation
>>> > non autorisée est interdite. Si vous n'êtes pas le destinataire prévu
de ce courriel, supprimez-le et contactez immédiatement l'expéditeur.
>>> > Veuillez penser à l'environnement avant d'imprimer le présent courriel
>>> 
>>> 
>>> 
>>> --
>>> Harsh J
>>> NOTICE: This e-mail message and any attachments are confidential, subject to
copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited.
If you are not the intended recipient, please delete and contact the sender immediately. Please
consider the environment before printing this e-mail. AVIS : le présent courriel et toute
pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent
être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée
est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et
contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer
le présent courriel
> 
> 
> 
> -- 
> Bertrand Dechoux
Mime
View raw message