Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Subject: Re: Inverse of a matrix using Map - Reduce
Mime-Version: 1.0 (Apple Message framework v1077)
Content-Type: multipart/signed; boundary=Apple-Mail-142--778836213;
 protocol="application/pkcs7-signature"; micalg=sha1
From: Brian Bockelman <bbockelm@cse.unl.edu>
In-Reply-To: <13685.1265255859@buffalo.edu>
Date: Thu, 4 Feb 2010 07:46:12 -0600
Cc: <aa225@buffalo.edu>, "Ganesh Swami" <ganesh@iamganesh.com>
Message-Id: <D4E6B4C6-14AC-4694-8E2A-854A8CDD2508@cse.unl.edu>
References: <13685.1265255859@buffalo.edu>
To: common-user@hadoop.apache.org

--Apple-Mail-142--778836213
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=iso-8859-1

Hey Abhishek,

Why would you want to fully invert a matrix that large?

How is it preconditioned?  What is the condition number of the matrix?

Why not just use ScaLAPACK?  It's a hairy beast, but you should =
definitely consider it.

Brian

On Feb 3, 2010, at 9:57 PM, aa225@buffalo.edu wrote:

> Hi,
>   Any idea how this method will scale for dense matrices ?The kind of =
matrices I
> am going to be working with are 500,000*500,000. Will this be a =
problem. Also
> have you used this patch ?
>=20
> Best Regards from Buffalo
>=20
> Abhishek Agrawal
>=20
> SUNY- Buffalo
> (716-435-7122)
>=20
> On Wed 02/03/10  1:41 AM , Ganesh Swami ganesh@iamganesh.com sent:
>> What about the Moore-Penrose inverse?
>>=20
>> http://en.wikipedia.org/wiki/Moore-Penrose_pseudoinverse
>>=20
>> The pseudo-inverse coincides with the regular inverse when the matrix
>> is non-singular. Moreover, it can be computed using the SVD.
>>=20
>> Here's a patch for a MapReduce version of the SVD:
>> https://issues.apache.org/jira/browse/MAHOUT-180
>> Ganesh
>>=20
>> On Tue, Feb 2, 2010 at 10:11 PM,  <aa225@buffa
>> lo.edu> wrote:> Hello People,
>>> =C2  =C2  =C2=20
>> =C2  =C2  =C2 My name is Abhishek Agrawal. For
>> the last few days I have been trying> to figure out how to calculate =
the
> inverse of a
>> matrix using Map Reduce. Matrix> inversion has 2 common approaches. =
Gaussian-
>> Jordan and the cofactor of transpose> method. But both of them dont =
seem to be
> suited
>> too well for Map- Reduce.> Gaussian Jordan involves blocking co =
factoring a
>> matrix requires repeated> calculation of determinant.
>>>=20
>>> Can some one give me any pointers so as to how
>> to solve this problem ?>
>>> Best Regards from Buffalo
>>>=20
>>> Abhishek Agrawal
>>>=20
>>> SUNY- Buffalo
>>> (716-435-7122)
>>>=20
>>>=20
>>>=20
>>>=20
>>=20
>>=20
>>=20
>>=20
>>=20


--Apple-Mail-142--778836213
Content-Disposition: attachment;
	filename=smime.p7s
Content-Type: application/pkcs7-signature;
	name=smime.p7s
Content-Transfer-Encoding: base64

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIICjCCA/gw
ggLgoAMCAQICASkwDQYJKoZIhvcNAQEFBQAwdTETMBEGCgmSJomT8ixkARkWA25ldDESMBAGCgmS
JomT8ixkARkWAkVTMQ4wDAYDVQQKEwVFU25ldDEgMB4GA1UECxMXQ2VydGlmaWNhdGUgQXV0aG9y
aXRpZXMxGDAWBgNVBAMTD0VTbmV0IFJvb3QgQ0EgMTAeFw0wMjEyMDUwODAwMDBaFw0xMzAxMjUw
ODAwMDBaMGkxEzARBgoJkiaJk/IsZAEZFgNvcmcxGDAWBgoJkiaJk/IsZAEZFghET0VHcmlkczEg
MB4GA1UECxMXQ2VydGlmaWNhdGUgQXV0aG9yaXRpZXMxFjAUBgNVBAMTDURPRUdyaWRzIENBIDEw
ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC09dYjYaPbCD5mtbiQb7Ka3y1qAm0ZcqKC
FciWcfe8Kwcuy9tjHuIsLf9ZItdkDW4xy8sua9nJlx3KlwjtumTMtOtg35KZCknUd8KM4VGTSFdL
VG9AbNayef76caVCGM1+jyF0Lq03kauGOPTcNfZe1TZa3e1c9rc8ljV5OSWa/mfsCACyS5zFIWu0
yIDNyJdf+n0hwaPN53wllpJ30taD+JBjQ7h2k4xRWzeaznLOb9OztZVRA/1sVze+iczFh2xwa4Vd
Gy0eIIPw1pfvYwxO36rm0S109qvbsNlaroPRbxerPKakQLpKe034Xcx7gBPqUk/FxoRRWin5EWN3
rz9LAgMBAAGjgZ4wgZswDgYDVR0PAQH/BAQDAgGGMBEGCWCGSAGG+EIBAQQEAwIAhzAdBgNVHQ4E
FgQUyhkdEo5upDhdQtQxDgjb2Y0XDV0wHwYDVR0jBBgwFoAUvF1NSC/4NZRZq1yJSz7RsjoUAeow
DwYDVR0TAQH/BAUwAwEB/zAlBgNVHREEHjAcgRpET0VHcmlkcy1DQS0xQGRvZWdyaWRzLm9yZzAN
BgkqhkiG9w0BAQUFAAOCAQEAZNVrIDLqe39CEOiJt7Q7EpBPhAihMvDTSf/42u0SMbUmChww4mLm
ph5DBghZUVF8Yn59kRZMn1QLOtO1HzLqvAvPITacZVPlJgG2IXzlR636YghZFAycbIUEOJDBHR4v
tQO1KDxgZwvAbtmKIoxvhUCq2xsfFt9kCBBn+JYtQ6O5LsBJq3PmuubeMcc7mbQAfJZ7h/3Qghgk
FIhmE1+LBXPJbkuP8vgfg6h2BKoAf5TFfZECgGZKimfN110tBvfedGZwYYd3/GsJc83B0JN1gny0
gqNVPm392UchXGeBRrHnm2gkhIkr48Oq6EmNGV9/a6XfbplQW/JWbtPVPWkaizCCBAowggLyoAMC
AQICAwCB+zANBgkqhkiG9w0BAQUFADBpMRMwEQYKCZImiZPyLGQBGRYDb3JnMRgwFgYKCZImiZPy
LGQBGRYIRE9FR3JpZHMxIDAeBgNVBAsTF0NlcnRpZmljYXRlIEF1dGhvcml0aWVzMRYwFAYDVQQD
Ew1ET0VHcmlkcyBDQSAxMB4XDTA5MDYwMjE5NDExM1oXDTEwMDYwMjE5NDExM1owYTETMBEGCgmS
JomT8ixkARkWA29yZzEYMBYGCgmSJomT8ixkARkWCGRvZWdyaWRzMQ8wDQYDVQQLEwZQZW9wbGUx
HzAdBgNVBAMTFkJyaWFuIEJvY2tlbG1hbiA1MDQzMDcwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
ggEKAoIBAQDPWEl7hBiuFRVBSY4SwvG0HpkCZi74a0BeD0tNARgxoQVJ7jhJjR3G4y8ino0/5axt
2EEfIWUE+DVpV37IWOQl8q/wdvicnhbfjByxBbq4sfWPLepU7+Kd8k1FKHRHermARn9VxEkFLrLB
Gp7O5EX4mFHDaQy+Vv0thtA+m4qKoM+DA/8cOkJA5Rn6ZS/v/vtBzJh9HimVnhBx4+rw2cvKN+7r
lKsm7qTn9TCZmrQ97CvBEXSkHS11m8vYF6ZwcTgSCJM0M9nnX5JilupQO1vDICXSUZeWX2xpsqeL
x1PFGWgDaYXxFGtTRt2Qc9EPwf9Dr72xGPbKN8u5HylpOMDnAgMBAAGjgcIwgb8wEQYJYIZIAYb4
QgEBBAQDAgWgMA4GA1UdDwEB/wQEAwIF4DAfBgNVHSMEGDAWgBTKGR0Sjm6kOF1C1DEOCNvZjRcN
XTAYBgNVHSAEETAPMA0GCyqGSIb3TAMHAQMAMD4GA1UdHwQ3MDUwM6AxoC+GLWh0dHA6Ly9jcmwu
ZG9lZ3JpZHMub3JnLzFjM2YyY2E4LzFjM2YyY2E4LmNybDAfBgNVHREEGDAWgRRiYm9ja2VsbUBj
c2UudW5sLmVkdTANBgkqhkiG9w0BAQUFAAOCAQEAp6KjcWnfnH/MGlUkUWstE9gtPeymHp+2r4zI
w8JXigncJh/8qpSZqBcVhD24WFowI95otblrKYNZKW9f2G/hWwDSxZFqHhCDxFO12vDthrzOc3EH
CwypJPvIlZPt/E/x93XruzPxJwPz84DKKuPoJAMeNlADbd+92YtRr2y+VuMpgZaebMAoeCdWH8Cq
Y8xheNMajf8uiImBbatDuCu7qRvhwgxsMNLHEt4h853K1Zc181RlFGXG1+uL/Q/8VeKiASiCu+7L
1zpfLg7OCr6rJHb5S7wU+CeAvzSqmyy0fd2mwPeiX7huK+Cw4UjaB3yGKItzWT+KQJnV//wcSrzZ
dTGCAv0wggL5AgEBMHAwaTETMBEGCgmSJomT8ixkARkWA29yZzEYMBYGCgmSJomT8ixkARkWCERP
RUdyaWRzMSAwHgYDVQQLExdDZXJ0aWZpY2F0ZSBBdXRob3JpdGllczEWMBQGA1UEAxMNRE9FR3Jp
ZHMgQ0EgMQIDAIH7MAkGBSsOAwIaBQCgggFiMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJ
KoZIhvcNAQkFMQ8XDTEwMDIwNDEzNDYxMlowIwYJKoZIhvcNAQkEMRYEFORIxtTTQkxFGEtWnOB3
s6RKddqwMH8GCSsGAQQBgjcQBDFyMHAwaTETMBEGCgmSJomT8ixkARkWA29yZzEYMBYGCgmSJomT
8ixkARkWCERPRUdyaWRzMSAwHgYDVQQLExdDZXJ0aWZpY2F0ZSBBdXRob3JpdGllczEWMBQGA1UE
AxMNRE9FR3JpZHMgQ0EgMQIDAIH7MIGBBgsqhkiG9w0BCRACCzFyoHAwaTETMBEGCgmSJomT8ixk
ARkWA29yZzEYMBYGCgmSJomT8ixkARkWCERPRUdyaWRzMSAwHgYDVQQLExdDZXJ0aWZpY2F0ZSBB
dXRob3JpdGllczEWMBQGA1UEAxMNRE9FR3JpZHMgQ0EgMQIDAIH7MA0GCSqGSIb3DQEBAQUABIIB
AC8QQ3FQtdfaRRz+L7W3/iWx/zfbTVdKZH8S896HMgIkGPs0vcV5h9vSgCAvaZCfecCHcy/TD4vs
Ee62srbbhOwsmqTCnVi0GxiiGqG1l0O1ra+cnwvaESXa9RlgfyBr8dr/R1ufOdiSu0kMPH6awdah
Vbw0VWPLDKB2CaQhaOdWFM2YL3t/QSPAwnDCMyomqbysRAZ2wvdDuV46he1JeVBTqAtcF+jx2IBt
TS5QfnYHXYzxyKyppVPX2eImylEdoqV13dpPEMlrH+l5d4ynU4nrAzakwiufCsb1uPMRf+lzbibJ
Te7mXxaWQ2hW6GEbSrRWOS0QB3hRXtnwSMSNCGAAAAAAAAA=

--Apple-Mail-142--778836213--