giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Cornell <m...@matthewcornell.org>
Subject Re: how do I maintain a cached List across supersteps?
Date Wed, 17 Sep 2014 23:18:17 GMT
Thanks to Claudio and Matthew, I went with the WorkerContext solution. Note
that I wrote a MasterCompute.validate() to verify the correct WorkerContext
class was set. Otherwise I was worried my cast would fail. -- matt

On Wed, Sep 17, 2014 at 11:49 AM, Claudio Martella <
claudio.martella@gmail.com> wrote:

> I would use a workercontext, it is shared and persistent during
> computation by all vertices in a worker. If it's readonly, you won't have
> to manage concurrency.
>
> On Tue, Sep 16, 2014 at 9:42 PM, Matthew Cornell <matt@matthewcornell.org>
> wrote:
>
>> Hi Folks. I have a custom argument that's passed into my Giraph job that
>> needs parsing. The parsed value is accessed by my Vertex#compute. To avoid
>> excessive GC I'd like to cache the parsing results. What's a good way to do
>> so? I looked at using the ImmutableClassesGiraphConfiguration returned by
>> getConf(), but it supports only String properties. I looked at using my
>> custom MasterCompute to manage it, but I couldn't find how to access the
>> master compute instance from the vertex. My last idea is to use (abuse?) an
>> aggregator to do this. I'd appreciate your thoughts! -- matt
>>
>> --
>> Matthew Cornell | matt@matthewcornell.org | 413-626-3621 | 34 Dickinson
>> Street, Amherst MA 01002 | matthewcornell.org
>>
>
>
>
> --
>    Claudio Martella
>
>



-- 
Matthew Cornell | matt@matthewcornell.org | 413-626-3621 | 34 Dickinson
Street, Amherst MA 01002 | matthewcornell.org

Mime
View raw message