giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: Waking up all the vertices after every vertex calls vote to halt
Date Thu, 21 Nov 2013 23:43:37 GMT
The simplest thing, is that you get a flag for each vertex to signal
whether they are really active. If not, they return. This means that
vertices never really vote to halt. Computationally, it does not cost you
much more than this check. You can play the rest of the logics with some
aggregators and the master compute.


On Thu, Nov 21, 2013 at 11:57 PM, Ameya Vilankar
<ameya.vilankar@gmail.com>wrote:

> Hi,
> I have implemented Alternating Least Squares on top apache giraph. On the
> edge, I store the type of the edge. Edges can be either a training edge or
> testing edge. When I run the algorithm, I use only the ratings on the
> training edge to tune the vectors on the vertices.
> The algorithm ends in one of the two scenarios:
> 1. All the vertices have tuned their vector with in the tolerable error.
> At this point there are no active vertices since everyone has called vote
> to halt.
> 2. We reached the maximum number of supersteps. At this point, some
> vertices are active since they received messages from the last superstep.
>
> I have written an Aggregator that counts the training error along this
> process. But now, I want to calculate the prediction/testing error which is
> along the testing labelled edges. But there are either no active vertices
> or few active vertices at this point in my algorithm. I need all the
> vertices to send their vectors along all of their testing edges to compute
> the testing error and send it to a error sum aggregator. For this I need to
> activate all the vertices.
> Hope it is clear to you now.
>
> Thanks,
> Ameya.
> Zynga
>
>
> On Thu, Nov 21, 2013 at 2:45 PM, Claudio Martella <
> claudio.martella@gmail.com> wrote:
>
>> Hi Ameya,
>>
>> I'm not sure I understand the problem correctly. The maximum number of
>> supersteps allows you to halt the computation when that threshold is
>> reached. The RMSE can be computed within the master compute.
>>
>> What do you want to achieve exactly?
>>
>>
>> On Thu, Nov 21, 2013 at 10:47 PM, Ameya Vilankar <
>> ameya.vilankar@gmail.com> wrote:
>>
>>> Hi,
>>> I am implementing a machine learning algorithm on top giraph. The
>>> algorithm converges when all the vertices call voteToHalt or some max
>>> number of supersteps have completed.
>>> I want to calculate the RMSE error  after the algorithm has converged.
>>> But the problem is either all the vertices have called vote to halt or (in
>>> the case where we reach max supersteps) only some of them are still active.
>>> I need to reactivate or wake up all the vertices. Is there any way in
>>> giraph that I could do this?
>>>
>>> Thanks,
>>> Ameya Vilankar
>>> Zynga
>>>
>>
>>
>>
>> --
>>    Claudio Martella
>>    claudio.martella@gmail.com
>>
>
>


-- 
   Claudio Martella
   claudio.martella@gmail.com

Mime
View raw message