mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From peng <pc...@uowmail.edu.au>
Subject Re: Does SSVD supports eigendecomposition of non-symmetric & non-positive-semidefinitive matrix better than Lanczos?
Date Tue, 18 Feb 2014 21:47:13 GMT
Thanks a lot Sebastian, Ted and Dmitriy, I'll try Giraph for a 
performance benchmark.
You are right, power iteration is just the most simple form of Lanczos, 
it shouldn't be in the scope.

On Tue 18 Feb 2014 03:59:57 AM EST, Sebastian Schelter wrote:
> You can also use giraph for a superfast PageRank implementation. Giraph
> even runs on standard hadoop clusters.
>
> Pagerank is usually computed by power iteration, which is much simpler than
> lanczos or ssvd and only gives the eigenvector associated with the largest
> eigenvalue.
> Am 18.02.2014 09:33 schrieb "Peng Cheng" <pc175@uowmail.edu.au>:
>
>> Really? I guess PageRank in mahout was removed due to inherited network
>> bottleneck of mapreduce. But I didn't know MLlib has the implementation. Is
>> mllib implementation based on Lanczos or SSVD? Just curious...
>>
>> On 17/02/2014 11:11 PM, Dmitriy Lyubimov wrote:
>>
>>> I bet page rank in mllib in spark finds stationary distribution much
>>> faster.
>>> On Feb 17, 2014 1:33 PM, "peng" <pc175@uowmail.edu.au> wrote:
>>>
>>>   Agreed, and this is the case where Lanczos algorithm is obsolete.
>>>> My point is: if SSVD is unable to find the eigenvector of asymmetric
>>>> matrix (this is a common formulation of PageRank, and some random walks,
>>>> and many other things), then we still have to rely on large-scale Lanczos
>>>> algorithm.
>>>>
>>>> On Mon 17 Feb 2014 04:25:16 PM EST, Ted Dunning wrote:
>>>>
>>>>   For the symmetric case, SVD is eigen decomposition.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Feb 17, 2014 at 1:12 PM, peng <pc175@uowmail.edu.au> wrote:
>>>>>
>>>>>    If SSVD is not designed for such eigenvector problem. Then I would
>>>>> vote
>>>>>
>>>>>> for retaining the Lanczos algorithm.
>>>>>> However, I would like to see the opposite case, I have tested both
>>>>>> algorithms on symmetric case and SSVD is much faster and more accurate
>>>>>> than
>>>>>> its competitor.
>>>>>>
>>>>>> Yours Peng
>>>>>>
>>>>>> On Wed 12 Feb 2014 03:25:47 PM EST, peng wrote:
>>>>>>
>>>>>>    In PageRank I'm afraid I have no other option than eigenvector
>>>>>>
>>>>>>> \lambda, but not singular vector u & v:) The PageRank in
Mahout was
>>>>>>> removed with other graph-based algorithm.
>>>>>>>
>>>>>>> On Tue 11 Feb 2014 06:34:17 PM EST, Ted Dunning wrote:
>>>>>>>
>>>>>>>    SSVD is very probably better than Lanczos for any large
>>>>>>> decomposition.
>>>>>>>
>>>>>>>>      That said, it does SVD, not eigen decomposition which
means that
>>>>>>>> the
>>>>>>>> question of symmetrical matrices or positive definiteness
doesn't
>>>>>>>> much
>>>>>>>> matter.
>>>>>>>>
>>>>>>>> Do you really need eigen-decomposition?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Feb 11, 2014 at 2:55 PM, peng <pc175@uowmail.edu.au>
wrote:
>>>>>>>>
>>>>>>>>     Just asking for possible replacement of our Lanczos-based
PageRank
>>>>>>>>
>>>>>>>>   implementation. - Peng
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>
>

Mime
View raw message