Thanks a lot Sebastian, Ted and Dmitriy, I'll try Giraph for a
performance benchmark.
You are right, power iteration is just the most simple form of Lanczos,
it shouldn't be in the scope.
On Tue 18 Feb 2014 03:59:57 AM EST, Sebastian Schelter wrote:
> You can also use giraph for a superfast PageRank implementation. Giraph
> even runs on standard hadoop clusters.
>
> Pagerank is usually computed by power iteration, which is much simpler than
> lanczos or ssvd and only gives the eigenvector associated with the largest
> eigenvalue.
> Am 18.02.2014 09:33 schrieb "Peng Cheng" <pc175@uowmail.edu.au>:
>
>> Really? I guess PageRank in mahout was removed due to inherited network
>> bottleneck of mapreduce. But I didn't know MLlib has the implementation. Is
>> mllib implementation based on Lanczos or SSVD? Just curious...
>>
>> On 17/02/2014 11:11 PM, Dmitriy Lyubimov wrote:
>>
>>> I bet page rank in mllib in spark finds stationary distribution much
>>> faster.
>>> On Feb 17, 2014 1:33 PM, "peng" <pc175@uowmail.edu.au> wrote:
>>>
>>> Agreed, and this is the case where Lanczos algorithm is obsolete.
>>>> My point is: if SSVD is unable to find the eigenvector of asymmetric
>>>> matrix (this is a common formulation of PageRank, and some random walks,
>>>> and many other things), then we still have to rely on largescale Lanczos
>>>> algorithm.
>>>>
>>>> On Mon 17 Feb 2014 04:25:16 PM EST, Ted Dunning wrote:
>>>>
>>>> For the symmetric case, SVD is eigen decomposition.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Feb 17, 2014 at 1:12 PM, peng <pc175@uowmail.edu.au> wrote:
>>>>>
>>>>> If SSVD is not designed for such eigenvector problem. Then I would
>>>>> vote
>>>>>
>>>>>> for retaining the Lanczos algorithm.
>>>>>> However, I would like to see the opposite case, I have tested both
>>>>>> algorithms on symmetric case and SSVD is much faster and more accurate
>>>>>> than
>>>>>> its competitor.
>>>>>>
>>>>>> Yours Peng
>>>>>>
>>>>>> On Wed 12 Feb 2014 03:25:47 PM EST, peng wrote:
>>>>>>
>>>>>> In PageRank I'm afraid I have no other option than eigenvector
>>>>>>
>>>>>>> \lambda, but not singular vector u & v:) The PageRank in
Mahout was
>>>>>>> removed with other graphbased algorithm.
>>>>>>>
>>>>>>> On Tue 11 Feb 2014 06:34:17 PM EST, Ted Dunning wrote:
>>>>>>>
>>>>>>> SSVD is very probably better than Lanczos for any large
>>>>>>> decomposition.
>>>>>>>
>>>>>>>> That said, it does SVD, not eigen decomposition which
means that
>>>>>>>> the
>>>>>>>> question of symmetrical matrices or positive definiteness
doesn't
>>>>>>>> much
>>>>>>>> matter.
>>>>>>>>
>>>>>>>> Do you really need eigendecomposition?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Feb 11, 2014 at 2:55 PM, peng <pc175@uowmail.edu.au>
wrote:
>>>>>>>>
>>>>>>>> Just asking for possible replacement of our Lanczosbased
PageRank
>>>>>>>>
>>>>>>>> implementation.  Peng
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>
>
