hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tran Son <son...@yahoo.com>
Subject Re: Re: Inverse of a matrix using Map - Reduce
Date Thu, 04 Feb 2010 10:56:08 GMT
Hi,
As far as i know, inversion of matrix need a lot of loops which are not supported well in
Hadoop MapRed. Hadoop MapRed is working well with block algorithms, especially for simple
operations such are addition, transposition and possibly multiplication. However, with inversion,
there is no (as I search up to now) algorithms support blocking, i mean working in each small
parts of matrix and combine to final result. There are several algorithms such as Gaussian
(as you said) or Csanky, but i think you will need more conplex implementation with ChainMapper/ChainReducer
and/or using multiple contrained job.
However, I think it does not effective much and convenient. So I am developing another version
of Map Reduce which support staging of Reducer: 1 job = Mapper Reducer*. I test it with Csanky
algorithm and it work quite well but I'm still on the way to improve the scheduling mechanism.





________________________________
From: "aa225@buffalo.edu" <aa225@buffalo.edu>
To: common-user@hadoop.apache.org; aa225@buffalo.edu; Ganesh Swami <ganesh@iamganesh.com>
Sent: Thursday, February 4, 2010 3:57:39
Subject: Re: Re: Inverse of a matrix using Map - Reduce

Hi,
   Any idea how this method will scale for dense matrices ?The kind of matrices I
am going to be working with are 500,000*500,000. Will this be a problem. Also
have you used this patch ?

Best Regards from Buffalo

Abhishek Agrawal

SUNY- Buffalo
(716-435-7122)

On Wed 02/03/10  1:41 AM , Ganesh Swami ganesh@iamganesh.com sent:
> What about the Moore-Penrose inverse?
> 
> http://en.wikipedia.org/wiki/Moore-Penrose_pseudoinverse
> 
> The pseudo-inverse coincides with the regular inverse when the matrix
> is non-singular. Moreover, it can be computed using the SVD.
> 
> Here's a patch for a MapReduce version of the SVD:
> https://issues.apache.org/jira/browse/MAHOUT-180
> Ganesh
> 
> On Tue, Feb 2, 2010 at 10:11 PM,  <aa225@buffa
> lo.edu> wrote:> Hello People,
> > Â  Â  Â 
> Â  Â  Â My name is Abhishek Agrawal. For
> the last few days I have been trying> to figure out how to calculate the
inverse of a
> matrix using Map Reduce. Matrix> inversion has 2 common approaches. Gaussian-
> Jordan and the cofactor of transpose> method. But both of them dont seem to be
suited
> too well for Map- Reduce.> Gaussian Jordan involves blocking co factoring a
> matrix requires repeated> calculation of determinant.
> >
> > Can some one give me any pointers so as to how
> to solve this problem ?>
> > Best Regards from Buffalo
> >
> > Abhishek Agrawal
> >
> > SUNY- Buffalo
> > (716-435-7122)
> >
> >
> >
> >
> 
> 
> 
> 
> 


      New Email names for you! 
Get the Email name you&#39;ve always wanted on the new @ymail and @rocketmail. 
Hurry before someone else does!
http://mail.promotions.yahoo.com/newdomains/aa/
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message