giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Saltz <sal...@gmail.com>
Subject Re: how do I maintain a cached List across supersteps?
Date Tue, 16 Sep 2014 22:09:24 GMT
Hey Matt,

If you need to share data between all vertices that persists across
supersteps and that's created/determined at runtime, I believe an
aggregator is the best way to do this. Then you can declare an instance
variable within the Computation class and within the preSuperstep method of
the class you can use getAggregatedValue to set the value of the variable.
Unless you can afford to just reparse the argument within the Computation
class at each superstep, in which case you can use getConf() and give that
a try.

Best,
Matthew
El 16/09/2014 21:43, "Matthew Cornell" <matt@matthewcornell.org> escribió:

> Hi Folks. I have a custom argument that's passed into my Giraph job that
> needs parsing. The parsed value is accessed by my Vertex#compute. To avoid
> excessive GC I'd like to cache the parsing results. What's a good way to do
> so? I looked at using the ImmutableClassesGiraphConfiguration returned by
> getConf(), but it supports only String properties. I looked at using my
> custom MasterCompute to manage it, but I couldn't find how to access the
> master compute instance from the vertex. My last idea is to use (abuse?) an
> aggregator to do this. I'd appreciate your thoughts! -- matt
>
> --
> Matthew Cornell | matt@matthewcornell.org | 413-626-3621 | 34 Dickinson
> Street, Amherst MA 01002 | matthewcornell.org
>

Mime
View raw message