giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ameya Vilankar <ameya.vilan...@gmail.com>
Subject Re: Waking up all the vertices after every vertex calls vote to halt
Date Thu, 21 Nov 2013 23:49:48 GMT
Nice suggestion. I will try that out. Thanks a ton.


On Thu, Nov 21, 2013 at 3:43 PM, Claudio Martella <
claudio.martella@gmail.com> wrote:

> The simplest thing, is that you get a flag for each vertex to signal
> whether they are really active. If not, they return. This means that
> vertices never really vote to halt. Computationally, it does not cost you
> much more than this check. You can play the rest of the logics with some
> aggregators and the master compute.
>
>
> On Thu, Nov 21, 2013 at 11:57 PM, Ameya Vilankar <ameya.vilankar@gmail.com
> > wrote:
>
>> Hi,
>> I have implemented Alternating Least Squares on top apache giraph. On the
>> edge, I store the type of the edge. Edges can be either a training edge or
>> testing edge. When I run the algorithm, I use only the ratings on the
>> training edge to tune the vectors on the vertices.
>> The algorithm ends in one of the two scenarios:
>> 1. All the vertices have tuned their vector with in the tolerable error.
>> At this point there are no active vertices since everyone has called vote
>> to halt.
>> 2. We reached the maximum number of supersteps. At this point, some
>> vertices are active since they received messages from the last superstep.
>>
>> I have written an Aggregator that counts the training error along this
>> process. But now, I want to calculate the prediction/testing error which is
>> along the testing labelled edges. But there are either no active vertices
>> or few active vertices at this point in my algorithm. I need all the
>> vertices to send their vectors along all of their testing edges to compute
>> the testing error and send it to a error sum aggregator. For this I need to
>> activate all the vertices.
>> Hope it is clear to you now.
>>
>> Thanks,
>> Ameya.
>> Zynga
>>
>>
>> On Thu, Nov 21, 2013 at 2:45 PM, Claudio Martella <
>> claudio.martella@gmail.com> wrote:
>>
>>> Hi Ameya,
>>>
>>> I'm not sure I understand the problem correctly. The maximum number of
>>> supersteps allows you to halt the computation when that threshold is
>>> reached. The RMSE can be computed within the master compute.
>>>
>>> What do you want to achieve exactly?
>>>
>>>
>>> On Thu, Nov 21, 2013 at 10:47 PM, Ameya Vilankar <
>>> ameya.vilankar@gmail.com> wrote:
>>>
>>>> Hi,
>>>> I am implementing a machine learning algorithm on top giraph. The
>>>> algorithm converges when all the vertices call voteToHalt or some max
>>>> number of supersteps have completed.
>>>> I want to calculate the RMSE error  after the algorithm has converged.
>>>> But the problem is either all the vertices have called vote to halt or (in
>>>> the case where we reach max supersteps) only some of them are still active.
>>>> I need to reactivate or wake up all the vertices. Is there any way in
>>>> giraph that I could do this?
>>>>
>>>> Thanks,
>>>> Ameya Vilankar
>>>> Zynga
>>>>
>>>
>>>
>>>
>>> --
>>>    Claudio Martella
>>>    claudio.martella@gmail.com
>>>
>>
>>
>
>
> --
>    Claudio Martella
>    claudio.martella@gmail.com
>

Mime
View raw message