giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avery Ching <>
Subject Re: Workers input splits and MasterCompute communication
Date Mon, 19 Aug 2013 17:17:29 GMT
That makes sense, since the Context doesn't have a real InputSplit (it's 
a Giraph one - see BspInputSplit).

What information are you trying to get out of the input splits? Giraph 
workers can process an arbitrary number of input splits (0 or more), so 
I don't think this will be useful.

You can use Configuration if you need to set some information at runtime.


On 8/19/13 9:14 AM, Marco Aurelio Barbosa Fagnani Lotz wrote:
> Hello all :)
> I am having problems calling getContext().getInputSplit(); inside the 
> compute() method in the workers.
> It always returns as if it didn't get any split at all, since 
> inputSplit.getLocations() returns without the hosts that should have 
> that split as local and inputSplit.getLength() returns 0.
> Should there be any initialization to the Workers context so that I 
> can get this information?
> Is there anyway to access the jobContext from the workers or the Master?
> Best Regards,
> Marco Lotz
> ------------------------------------------------------------------------
> *From:* Marco Aurelio Barbosa Fagnani Lotz <>
> *Sent:* 17 August 2013 20:20
> *To:*
> *Subject:* Workers input splits and MasterCompute communication
> Hello all :)
> In what class the workers actually get the input file splits from the 
> file system?
> Is it possible to a MasterCompute class object to have 
> access/communication with the workers in that job? I though about 
> using aggregators, but then I assumed that aggregators actually work 
> with vertices compute() (and related methods) and not with the worker 
> itself.
> When I mean workers I don't mean the vertices in each worker, but the 
> object that runs the compute for all the vertices in that worker.
> Best Regards,
> Marco Lotz

View raw message