hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mehmet Tepedelenlioglu <mehmets...@gmail.com>
Subject Re: providing the same input to more than one Map task
Date Fri, 22 Apr 2011 21:59:36 GMT
There is a way:


Are you working with a sparse matrix, or a full one?

On Apr 22, 2011, at 2:33 PM, aanghelescu wrote:

> Hi all,
> I am trying to perform matrix-vector multiplication using Hadoop. 
> So I have matrix M in a file, and vector v in another file. Obviously, files
> are of different sizes. Is it possible to make it so that each Map task will
> get the whole vector v and a chunk of matrix M? I know how my map and reduce
> functions should look like, but I don't know how to format the input. 
> Basically I want my map function to output key-value pairs (i,m[i,j]*v(j)),
> where i is the row number, and j the column number; v(j) is the jth element
> in v. And the reduce function will sum up all the values with the same key -
> i, and that will be the ith element of my result vector. 
> Or can you suggest another way to do it?
> Thanks,
> Alexandra
> -- 
> View this message in context: http://old.nabble.com/providing-the-same-input-to-more-than-one-Map-task-tp31459012p31459012.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message