Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4712DD26D for ; Mon, 10 Sep 2012 20:21:26 +0000 (UTC) Received: (qmail 44997 invoked by uid 500); 10 Sep 2012 20:21:26 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 44962 invoked by uid 500); 10 Sep 2012 20:21:26 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 44951 invoked by uid 99); 10 Sep 2012 20:21:26 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 10 Sep 2012 20:21:25 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of apache.mailbox@gmail.com designates 209.85.216.52 as permitted sender) Received: from [209.85.216.52] (HELO mail-qa0-f52.google.com) (209.85.216.52) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 10 Sep 2012 20:21:19 +0000 Received: by qabg14 with SMTP id g14so1194462qab.11 for ; Mon, 10 Sep 2012 13:20:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=tgTFWnGqOrSPLyqSe2BmgJ7mG+lUdfdisXS785hS0b4=; b=CLevj4WVVeB8WfuFAGKza2rYHLa9zLq7/bLcLE2EK9qNb1RiZZpK3iMuChH5DLJmbF mev5sh32WrnMyMeHIU+I9WtHeuLkezWsgAmholTL5YzGzLJM4iUuySuT3T2AiA4KovOp jZLTacgTqx8H2tZ8qG5MtxT9HwF6YFfcEvKsjrkOiYC/FSL7u9DIAp6735TKd6esG8ul ovEaPWAyNl+3bsbialtsFflhj7Xjm3zd+1m4BjQm+9FETYo5QnqEQtPSHMwn574DzEkM G+o6Nqy6/PEkYuie8Rqb/QvtvxI6Da9ACpxUelR203ixIgFUDPfwSojzRchi858nSDVj vvQQ== MIME-Version: 1.0 Received: by 10.229.137.85 with SMTP id v21mr4088990qct.17.1347308457954; Mon, 10 Sep 2012 13:20:57 -0700 (PDT) Received: by 10.49.15.129 with HTTP; Mon, 10 Sep 2012 13:20:57 -0700 (PDT) In-Reply-To: References: Date: Mon, 10 Sep 2012 13:20:57 -0700 Message-ID: Subject: Re: How to register aggregators with the 'new' Giraph? From: Eli Reisman To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=00235452e9600a0eaa04c95eb2d6 --00235452e9600a0eaa04c95eb2d6 Content-Type: text/plain; charset=ISO-8859-1 Hi Paolo, I would say Alessandro and Maja are probably to go-to people as far as up to date Aggregator best practices, there have been a lot of changes in that area of the code lately. If they don't field this one I'd be happy to take a look myself. Welcome back! Eli On Mon, Sep 10, 2012 at 12:36 PM, Paolo Castagna wrote: > Hi, > first and foremost, thanks for all the work and improvements on Giraph. > I went away from computers for a while (personal reasons) and changed > job, now I am back and playing with Giraph when I can. > > I updated my little examples (overall, it was easy and quick, here the > changes [1]. Just in case others are in a similar situation and want > to have a look). > > I am not sure I get the 'new' aggregators and in particular how I can > 'register' them. My tests failing confirm my non understanding! And > forgive me if I come here and ask such a simple question. > > Here is what I used to do [2]: > > public class PageRankVertexWorkerContext extends WorkerContext { > > private static final Logger log = > LoggerFactory.getLogger(PageRankVertexWorkerContext.class); > > public static double errorPrevious = Double.MAX_VALUE; > public static double danglingPrevious = 0d; > > @SuppressWarnings("unchecked") > @Override > public void preApplication() throws InstantiationException, > IllegalAccessException { > log.debug("preApplication()"); > registerAggregator("dangling-current", SumAggregator.class); > registerAggregator("error-current", SumAggregator.class); > registerAggregator("pagerank-sum", SumAggregator.class); > registerAggregator("vertices-count", LongSumAggregator.class); > > > ((Aggregator)getAggregator("error-current")).setAggregatedValue( > new DoubleWritable( Double.MAX_VALUE ) ); > } > > [...] > > > Here is what I am trying to do now [3]: > > public class PageRankVertexWorkerContext extends WorkerContext { > > private static final Logger log = > LoggerFactory.getLogger(PageRankVertexWorkerContext.class); > > public static double errorPrevious = Double.MAX_VALUE; > public static double danglingPrevious = 0d; > > // TODO: double check this... how is calling initialize()? > public static class SimplePageRankVertexMasterCompute extends > DefaultMasterCompute { > @Override > public void initialize() throws InstantiationException, > IllegalAccessException { > registerAggregator("dangling-current", DoubleSumAggregator.class); > registerAggregator("error-current", DoubleSumAggregator.class); > registerAggregator("pagerank-sum", DoubleSumAggregator.class); > registerAggregator("vertices-count", LongSumAggregator.class); > } > } > > [...] > > > I am not convinced someone is actually calling the initialize() method > and there must be something I am missing (yesterday was late, after a > long day at work). > > Anyway, is there a place/example where I can learn how to use > Aggregators with the new Giraph? > > Thanks again and it's good to see Giraph mailing list and JIRA 'brewing' > ;-) > > Paolo > > > [1] > https://github.com/castagna/jena-grande/commit/3edc0a7780f5e7c25d37956c158d878b590858b5#src/main/java/org/apache/jena/grande/giraph/pagerank/PageRankVertexWorkerContext.java > [2] > https://github.com/castagna/jena-grande/blob/2fa8a1b879a464d8e3db84e78edd539c70274e7c/src/main/java/org/apache/jena/grande/giraph/pagerank/PageRankVertexWorkerContext.java > [3] > https://github.com/castagna/jena-grande/blob/3edc0a7780f5e7c25d37956c158d878b590858b5/src/main/java/org/apache/jena/grande/giraph/pagerank/PageRankVertexWorkerContext.java > --00235452e9600a0eaa04c95eb2d6 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi Paolo,

I would say Alessandro and Maja are probably to go-to peop= le as far as up to date Aggregator best practices, there have been a lot of= changes in that area of the code lately. If they don't field this one = I'd be happy to take a look myself. Welcome back!

Eli

On Mon, Sep 10, 2012 at 12:36 PM,= Paolo Castagna <castagna.lists@gmail.com> wrote:
=
Hi,
first and foremost, thanks for all the work and improvements on Giraph.
I went away from computers for a while (personal reasons) and changed
job, now I am back and playing with Giraph when I can.

I updated my little examples (overall, it was easy and quick, here the
changes [1]. Just in case others are in a similar situation and want
to have a look).

I am not sure I get the 'new' aggregators and in particular how I c= an
'register' them. My tests failing confirm my non understanding! And=
forgive me if I come here and ask such a simple question.

Here is what I used to do [2]:

public class PageRankVertexWorkerContext extends WorkerContext {

=A0 private static final Logger log =3D
LoggerFactory.getLogger(PageRankVertexWorkerContext.class);

=A0 public static double errorPrevious =3D Double.MAX_VALUE;
=A0 public static double danglingPrevious =3D 0d;

=A0 @SuppressWarnings("unchecked")
=A0 @Override
=A0 public void preApplication() throws InstantiationException,
IllegalAccessException {
=A0 =A0 log.debug("preApplication()");
=A0 =A0 registerAggregator("dangling-current", SumAggregator.clas= s);
=A0 =A0 registerAggregator("error-current", SumAggregator.class);=
=A0 =A0 registerAggregator("pagerank-sum", SumAggregator.class);<= br> =A0 =A0 registerAggregator("vertices-count", LongSumAggregator.cl= ass);

=A0 =A0 ((Aggregator<DoubleWritable>)getAggregator("error-curren= t")).setAggregatedValue(
new DoubleWritable( Double.MAX_VALUE ) );
=A0 }

=A0 [...]


Here is what I am trying to do now [3]:

public class PageRankVertexWorkerContext extends WorkerContext {

=A0 private static final Logger log =3D
LoggerFactory.getLogger(PageRankVertexWorkerContext.class);

=A0 public static double errorPrevious =3D Double.MAX_VALUE;
=A0 public static double danglingPrevious =3D 0d;

=A0 // TODO: double check this... how is calling initialize()?
=A0 public static class SimplePageRankVertexMasterCompute extends
DefaultMasterCompute {
=A0 =A0 @Override
=A0 =A0 public void initialize() throws InstantiationException,
IllegalAccessException {
=A0 =A0 =A0 registerAggregator("dangling-current", DoubleSumAggre= gator.class);
=A0 =A0 =A0 registerAggregator("error-current", DoubleSumAggregat= or.class);
=A0 =A0 =A0 registerAggregator("pagerank-sum", DoubleSumAggregato= r.class);
=A0 =A0 =A0 registerAggregator("vertices-count", LongSumAggregato= r.class);
=A0 =A0 }
=A0 }

=A0 [...]


I am not convinced someone is actually calling the initialize() method
and there must be something I am missing (yesterday was late, after a
long day at work).

Anyway, is there a place/example where I can learn how to use
Aggregators with the new Giraph?

Thanks again and it's good to see Giraph mailing list and JIRA 'bre= wing' ;-)

Paolo


=A0[1] https://github= .com/castagna/jena-grande/commit/3edc0a7780f5e7c25d37956c158d878b590858b5#s= rc/main/java/org/apache/jena/grande/giraph/pagerank/PageRankVertexWorkerCon= text.java
=A0[2] https://github.c= om/castagna/jena-grande/blob/2fa8a1b879a464d8e3db84e78edd539c70274e7c/src/m= ain/java/org/apache/jena/grande/giraph/pagerank/PageRankVertexWorkerContext= .java
=A0[3] https://github.c= om/castagna/jena-grande/blob/3edc0a7780f5e7c25d37956c158d878b590858b5/src/m= ain/java/org/apache/jena/grande/giraph/pagerank/PageRankVertexWorkerContext= .java

--00235452e9600a0eaa04c95eb2d6--