giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: how do I maintain a cached List across supersteps?
Date Wed, 17 Sep 2014 15:49:31 GMT
I would use a workercontext, it is shared and persistent during computation
by all vertices in a worker. If it's readonly, you won't have to manage
concurrency.

On Tue, Sep 16, 2014 at 9:42 PM, Matthew Cornell <matt@matthewcornell.org>
wrote:

> Hi Folks. I have a custom argument that's passed into my Giraph job that
> needs parsing. The parsed value is accessed by my Vertex#compute. To avoid
> excessive GC I'd like to cache the parsing results. What's a good way to do
> so? I looked at using the ImmutableClassesGiraphConfiguration returned by
> getConf(), but it supports only String properties. I looked at using my
> custom MasterCompute to manage it, but I couldn't find how to access the
> master compute instance from the vertex. My last idea is to use (abuse?) an
> aggregator to do this. I'd appreciate your thoughts! -- matt
>
> --
> Matthew Cornell | matt@matthewcornell.org | 413-626-3621 | 34 Dickinson
> Street, Amherst MA 01002 | matthewcornell.org
>



-- 
   Claudio Martella

Mime
View raw message