hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-2811) Adding Multiple Reducers implementations.
Date Tue, 22 Jul 2014 18:30:40 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Allen Wittenauer updated MAPREDUCE-2811:
----------------------------------------

    Description: Like HADOOP-372, we have a multi format Reducer too. Someone suggested that
if we need different reducers and map implementations(like what i need) I was better of by
writing 2 jobs. I dont quite agree. I am calculating 2 big matrices that must be calculated
in the map step, summed in the reducers multiplied and then written to a file. The First mapper
sums a matrix  based on the i,j th index(key) into the file and the second mapper adds the
N*1  dimension vector that uses a new line as key. These keys must be passed as such to the
reduce process.  (was: Like the Patch released here https://issues.apache.org/jira/browse/HADOOP-372
can we have a multi format Reducer too. Someone suggested that if we need different reducers
and map implementations(like what i need) I was better of by writing 2 jobs. I dont quite
agree. I am calculating 2 big matrices that must be calculated in the map step, summed in
the reducers multiplied and then written to a file. The First mapper sums a matrix  based
on the i,j th index(key) into the file and the second mapper adds the N*1  dimension vector
that uses a new line as key. These keys must be passed as such to the reduce process.)

> Adding Multiple Reducers implementations.
> -----------------------------------------
>
>                 Key: MAPREDUCE-2811
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2811
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Sidharth Gupta
>
> Like HADOOP-372, we have a multi format Reducer too. Someone suggested that if we need
different reducers and map implementations(like what i need) I was better of by writing 2
jobs. I dont quite agree. I am calculating 2 big matrices that must be calculated in the map
step, summed in the reducers multiplied and then written to a file. The First mapper sums
a matrix  based on the i,j th index(key) into the file and the second mapper adds the N*1
 dimension vector that uses a new line as key. These keys must be passed as such to the reduce
process.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message