hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From unmesha sreeveni <unmeshab...@gmail.com>
Subject Finding mamimum value in reducer
Date Tue, 24 Jun 2014 09:14:45 GMT
I have a scenario.

Output from previous job1 is http://pastebin.com/ADa8fTGB.

In next job2 I need to get/find i key having maximum value.

eg i=3, 3 keys having maximum value.
(i will be a custom parameter)

How to approach this.

Should we calculated max() in job2 mapper as there will be unique keys(as
the output is coming from previous reducer)

or

find max in second jobs reducer.But again how to find i keys?

I tried in this way
Instead of emiting "value" as value in reducer.I emitted value as key so I
can get the values in ascending order. And I wrote the next MR job.where
mapper simply emits the key/value.

Reducer finds the max of key But again I am stuck that cannot be done as we
try to get the id , because id is only unique,Values are not uniqe

How to solve this.

-- 
*Thanks & Regards *


*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Center for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/

Mime
View raw message