Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6FB0C101EA for ; Thu, 21 Nov 2013 23:50:16 +0000 (UTC) Received: (qmail 85266 invoked by uid 500); 21 Nov 2013 23:50:16 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 85233 invoked by uid 500); 21 Nov 2013 23:50:16 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 85220 invoked by uid 99); 21 Nov 2013 23:50:16 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 Nov 2013 23:50:16 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of ameya.vilankar@gmail.com designates 74.125.82.180 as permitted sender) Received: from [74.125.82.180] (HELO mail-we0-f180.google.com) (74.125.82.180) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 Nov 2013 23:50:09 +0000 Received: by mail-we0-f180.google.com with SMTP id u56so464489wes.25 for ; Thu, 21 Nov 2013 15:49:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=H5WE3+FF9L6cjzVCQVtsuNGgcQMUTW5V0Caz17WMBJo=; b=WFqsniJ060pkvrYg4aHlmGPN8omqjheDztYSZoWzSI8+jBfZZwtdUxDMi+D33lJJbb w6AK9JgBHSQhs70gbzF/8JSN0PKX54ScNeuCFvQDBIMi1pLTNiGjGTSyNJrEGkCz85DX 4w/HujMnYk0gdbuVbsw6vmxQLNCJzK2jQUHtwl/Hkdzym4TT2c53lsrFywq1A6vq86Po MI5QIvDkoyCCWzCcFS3Tjw1G++wn15SNZ92MMdaJ9qhNB9z8edUFdmSLeBlxTypgcCRv 4qsGK6tG9UTpkCMx/Nt9TxWYiMOJ/h+rzqS9bd0EBOP65tjCUu5hw5UcOZbIT7L44QWa WFdw== MIME-Version: 1.0 X-Received: by 10.180.208.49 with SMTP id mb17mr130431wic.64.1385077788778; Thu, 21 Nov 2013 15:49:48 -0800 (PST) Received: by 10.216.119.132 with HTTP; Thu, 21 Nov 2013 15:49:48 -0800 (PST) In-Reply-To: References: Date: Thu, 21 Nov 2013 15:49:48 -0800 Message-ID: Subject: Re: Waking up all the vertices after every vertex calls vote to halt From: Ameya Vilankar To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=001a11c37e30963d3e04ebb88d7e X-Virus-Checked: Checked by ClamAV on apache.org --001a11c37e30963d3e04ebb88d7e Content-Type: text/plain; charset=ISO-8859-1 Nice suggestion. I will try that out. Thanks a ton. On Thu, Nov 21, 2013 at 3:43 PM, Claudio Martella < claudio.martella@gmail.com> wrote: > The simplest thing, is that you get a flag for each vertex to signal > whether they are really active. If not, they return. This means that > vertices never really vote to halt. Computationally, it does not cost you > much more than this check. You can play the rest of the logics with some > aggregators and the master compute. > > > On Thu, Nov 21, 2013 at 11:57 PM, Ameya Vilankar > wrote: > >> Hi, >> I have implemented Alternating Least Squares on top apache giraph. On the >> edge, I store the type of the edge. Edges can be either a training edge or >> testing edge. When I run the algorithm, I use only the ratings on the >> training edge to tune the vectors on the vertices. >> The algorithm ends in one of the two scenarios: >> 1. All the vertices have tuned their vector with in the tolerable error. >> At this point there are no active vertices since everyone has called vote >> to halt. >> 2. We reached the maximum number of supersteps. At this point, some >> vertices are active since they received messages from the last superstep. >> >> I have written an Aggregator that counts the training error along this >> process. But now, I want to calculate the prediction/testing error which is >> along the testing labelled edges. But there are either no active vertices >> or few active vertices at this point in my algorithm. I need all the >> vertices to send their vectors along all of their testing edges to compute >> the testing error and send it to a error sum aggregator. For this I need to >> activate all the vertices. >> Hope it is clear to you now. >> >> Thanks, >> Ameya. >> Zynga >> >> >> On Thu, Nov 21, 2013 at 2:45 PM, Claudio Martella < >> claudio.martella@gmail.com> wrote: >> >>> Hi Ameya, >>> >>> I'm not sure I understand the problem correctly. The maximum number of >>> supersteps allows you to halt the computation when that threshold is >>> reached. The RMSE can be computed within the master compute. >>> >>> What do you want to achieve exactly? >>> >>> >>> On Thu, Nov 21, 2013 at 10:47 PM, Ameya Vilankar < >>> ameya.vilankar@gmail.com> wrote: >>> >>>> Hi, >>>> I am implementing a machine learning algorithm on top giraph. The >>>> algorithm converges when all the vertices call voteToHalt or some max >>>> number of supersteps have completed. >>>> I want to calculate the RMSE error after the algorithm has converged. >>>> But the problem is either all the vertices have called vote to halt or (in >>>> the case where we reach max supersteps) only some of them are still active. >>>> I need to reactivate or wake up all the vertices. Is there any way in >>>> giraph that I could do this? >>>> >>>> Thanks, >>>> Ameya Vilankar >>>> Zynga >>>> >>> >>> >>> >>> -- >>> Claudio Martella >>> claudio.martella@gmail.com >>> >> >> > > > -- > Claudio Martella > claudio.martella@gmail.com > --001a11c37e30963d3e04ebb88d7e Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Nice suggestion. I will try that out. Thanks a ton.
<= div class=3D"gmail_extra">

On Thu, Nov 21= , 2013 at 3:43 PM, Claudio Martella <claudio.martella@gmail.com> wrote:
The simplest thing, is that= you get a flag for each vertex to signal whether they are really active. I= f not, they return. This means that vertices never really vote to halt. Com= putationally, it does not cost you much more than this check. You can play = the rest of the logics with some aggregators and the master compute.


On Thu, Nov 21, 2013 at 11:57 PM, Ameya = Vilankar <ameya.vilankar@gmail.com> wrote:
Hi,
I have implemented = Alternating Least Squares on top apache giraph. On the edge, I store the ty= pe of the edge. Edges can be either a training edge or testing edge. When I= run the algorithm, I use only the ratings on the training edge to tune the= vectors on the vertices.=A0
The algorithm ends in one of the two scenarios:
1. All the v= ertices have tuned their vector with in the tolerable error. At this point = there are no active vertices since everyone has called vote to halt.
2. We reached the maximum number of supersteps. At this point, some ve= rtices are active since they received messages from the last superstep.

I have written an Aggregator that counts the training= error along this process. But now, I want to calculate the prediction/test= ing error which is along the testing labelled edges. But there are either n= o active vertices or few active vertices at this point in my algorithm. I n= eed all the vertices to send their vectors along all of their testing edges= to compute the testing error and send it to a error sum aggregator. For th= is I need to activate all the vertices.
Hope it is clear to you now.

Thanks,
Ameya.
Zynga

On Thu, Nov 21, 2013 at 2:45 PM, Claudio M= artella <claudio.martella@gmail.com> wrote:
Hi Ameya,

I'm not sure I understand the problem correctly. The maximum number o= f supersteps allows you to halt the computation when that threshold is reac= hed. The RMSE can be computed within the master compute.

What do you want to achieve exactly?




<= /div>--
=A0 =A0Claudio = Martella
=A0 =A0claudio.martella@gmail.com=A0 =A0
--001a11c37e30963d3e04ebb88d7e--