hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Afflictedd2 <flethu...@gmail.com>
Subject Filtering while mapping or reducing?
Date Sun, 10 Apr 2011 15:29:27 GMT

Hi everyone,

I need to add the frequencies of two elements with the same last letter,
however
I don't want single elements in the output of this map reduce. I don't know
where
the filtering of those elements should either be done in the mapping
function or
the reduce function?

For example, if I have the following pairs:

{"f2B"=>4,
 "f1A"=>3,
 "f2C"=>5,
 "f1B"=>4,
 "f2D"=>7,
 "f1C"=>5,
 "f2E"=>8,
 "f1D"=>7,
 "f1F"=>8,
 "f2A"=>3}

I want the following pairs out

{"B"=>4,
 "A"=>3,
 "C"=>5,
 "B"=>4,
 "D"=>7,
 "C"=>5,
 "E"=>8,
 "D"=>7,
 "F"=>8,
 "A"=>3}

but I don't want E and F in my final result of reduced pairs

{"B"=>8,
 "A"=>6,
 "C"=>10,
 "D"=>14,
}
-- 
View this message in context: http://old.nabble.com/Filtering-while-mapping-or-reducing--tp31364272p31364272.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.


Mime
View raw message