giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jake Mannix <>
Subject Re: On pre/post Application/Superstep contract
Date Fri, 30 Sep 2011 15:23:04 GMT
Remember that there's already a "singleton"-like object available to all
vertices: the GraphState object, which has a handle on the GraphMapper.

Maybe this is the right place to get your handle on the FSDataOutputStream?


On Fri, Sep 30, 2011 at 7:25 AM, Claudio Martella <> wrote:

> Hello,
> to my understanding pre/post Application/Superstep methods are called
> ONCE on a "fake" vertex on each worker (the so called
> representativeVertex). This means that these methods should not depend
> on any specific-vertex data.
> As I'm trying to sort out my Emitter, I thought I could create one
> FSDataOutputStream per worker which each Vertex belonging to that
> worker could share (which would be even thread-safe as each worker is
> not parallel).
> The questions are:
> 1) how to share the FSDataOutputFormat object created at
> preApplication() (and closed at postApplication()) created by this
> representativeVertex?
> 2) about the filename, I'd be happy to have access to the Worker Id so
> to create an outputfile filename as with happens with reducers and
> part files by FileOutputFormat (i.e. <userdefinedfilename>-workerid).
> The "best" idea i have in my mind right now is to use the calling
> vertex (the representativeVertex) hashCode as the id, and create an
> external Singleton where i can request register and request the
> outputfiles similarly to what happens with Aggregators now, and by
> passing the *this* reference as an index to this map. Any better idea?
> :)
> --
>     Claudio Martella

View raw message