hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Yehdego <dtyehd...@miners.utep.edu>
Subject RE: Reducer to concatenate string values
Date Tue, 20 Sep 2011 06:04:47 GMT

Hi Kai, 
Many thanks for your response. I will look at the links you sent me and I will be back to
you. 

Regards, 

Daniel T. Yehdego
Computational Science Program 
University of Texas at El Paso, UTEP 
dtyehdego@miners.utep.edu

> Subject: Re: Reducer to concatenate string values
> From: k@123.org
> Date: Tue, 20 Sep 2011 07:53:56 +0200
> To: common-user@hadoop.apache.org
> 
> Hi Daniel,
> 
> the values for a single key will be passed to reduce() in a non-predictable order. Actually,
when running the same job on the same data again, the order is most likely different every
time.
> 
> If you want the values to be in a sorted way, you need to apply a 'secondary sort'. The
basic idea is to attach your values to the key, and then benefit from the sorting Hadoop does
on the key.
> 
> However, you need to write some code to make that happen. Josh wrote a nice series of
articles on it, and you will find more if you google for "secondary sort".
> 
> http://www.cloudera.com/blog/2011/03/simple-moving-average-secondary-sort-and-mapreduce-part-1/
> http://www.cloudera.com/blog/2011/03/simple-moving-average-secondary-sort-and-mapreduce-part-2/
> http://www.cloudera.com/blog/2011/04/simple-moving-average-secondary-sort-and-mapreduce-part-3/
> 
> Kai
> 
> Am 20.09.2011 um 07:43 schrieb Daniel Yehdego:
> 
> > 
> > Good evening, 
> > I have a certain value output from a mapper and I want to concatenate the string
outputs using a Reducer (one reducer).But the order of the concatenated string values is not
in order. How can I use a reducer that receives a value from a mapper output and concatenate
the strings in order. waiting your response and thanks in advance.  
> > 
> > Regards, 
> > 
> > Daniel T. Yehdego
> > Computational Science Program 
> > University of Texas at El Paso, UTEP 
> > dtyehdego@miners.utep.edu 		 	   		  
> 
> -- 
> Kai Voigt
> k@123.org
> 
> 
> 
> 
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message