giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Reisman <apache.mail...@gmail.com>
Subject Re: How to register aggregators with the 'new' Giraph?
Date Mon, 10 Sep 2012 20:20:57 GMT
Hi Paolo,

I would say Alessandro and Maja are probably to go-to people as far as up
to date Aggregator best practices, there have been a lot of changes in that
area of the code lately. If they don't field this one I'd be happy to take
a look myself. Welcome back!

Eli

On Mon, Sep 10, 2012 at 12:36 PM, Paolo Castagna
<castagna.lists@gmail.com>wrote:

> Hi,
> first and foremost, thanks for all the work and improvements on Giraph.
> I went away from computers for a while (personal reasons) and changed
> job, now I am back and playing with Giraph when I can.
>
> I updated my little examples (overall, it was easy and quick, here the
> changes [1]. Just in case others are in a similar situation and want
> to have a look).
>
> I am not sure I get the 'new' aggregators and in particular how I can
> 'register' them. My tests failing confirm my non understanding! And
> forgive me if I come here and ask such a simple question.
>
> Here is what I used to do [2]:
>
> public class PageRankVertexWorkerContext extends WorkerContext {
>
>   private static final Logger log =
> LoggerFactory.getLogger(PageRankVertexWorkerContext.class);
>
>   public static double errorPrevious = Double.MAX_VALUE;
>   public static double danglingPrevious = 0d;
>
>   @SuppressWarnings("unchecked")
>   @Override
>   public void preApplication() throws InstantiationException,
> IllegalAccessException {
>     log.debug("preApplication()");
>     registerAggregator("dangling-current", SumAggregator.class);
>     registerAggregator("error-current", SumAggregator.class);
>     registerAggregator("pagerank-sum", SumAggregator.class);
>     registerAggregator("vertices-count", LongSumAggregator.class);
>
>
> ((Aggregator<DoubleWritable>)getAggregator("error-current")).setAggregatedValue(
> new DoubleWritable( Double.MAX_VALUE ) );
>   }
>
>   [...]
>
>
> Here is what I am trying to do now [3]:
>
> public class PageRankVertexWorkerContext extends WorkerContext {
>
>   private static final Logger log =
> LoggerFactory.getLogger(PageRankVertexWorkerContext.class);
>
>   public static double errorPrevious = Double.MAX_VALUE;
>   public static double danglingPrevious = 0d;
>
>   // TODO: double check this... how is calling initialize()?
>   public static class SimplePageRankVertexMasterCompute extends
> DefaultMasterCompute {
>     @Override
>     public void initialize() throws InstantiationException,
> IllegalAccessException {
>       registerAggregator("dangling-current", DoubleSumAggregator.class);
>       registerAggregator("error-current", DoubleSumAggregator.class);
>       registerAggregator("pagerank-sum", DoubleSumAggregator.class);
>       registerAggregator("vertices-count", LongSumAggregator.class);
>     }
>   }
>
>   [...]
>
>
> I am not convinced someone is actually calling the initialize() method
> and there must be something I am missing (yesterday was late, after a
> long day at work).
>
> Anyway, is there a place/example where I can learn how to use
> Aggregators with the new Giraph?
>
> Thanks again and it's good to see Giraph mailing list and JIRA 'brewing'
> ;-)
>
> Paolo
>
>
>  [1]
> https://github.com/castagna/jena-grande/commit/3edc0a7780f5e7c25d37956c158d878b590858b5#src/main/java/org/apache/jena/grande/giraph/pagerank/PageRankVertexWorkerContext.java
>  [2]
> https://github.com/castagna/jena-grande/blob/2fa8a1b879a464d8e3db84e78edd539c70274e7c/src/main/java/org/apache/jena/grande/giraph/pagerank/PageRankVertexWorkerContext.java
>  [3]
> https://github.com/castagna/jena-grande/blob/3edc0a7780f5e7c25d37956c158d878b590858b5/src/main/java/org/apache/jena/grande/giraph/pagerank/PageRankVertexWorkerContext.java
>

Mime
View raw message