giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avery Ching <>
Subject Re: writing/emitting to HDFS
Date Thu, 22 Sep 2011 14:07:10 GMT
There are some methods in Vertex (i.e. preApplication(), preSuperstep(), 
postApplication(), postSuperstep()) that can be overidden to do anything 
you like, for instance write out some data to an HDFS file.  We have an 
open issue on outputting Aggregator values that is unassigned if you'd 
like to take a look at it as well 

Hope this helps,


On 9/22/11 7:34 AM, Claudio Martella wrote:
> Hello list,
> I have the need to emit to HDFS once in a while some Text. This
> doesn't happen necessarily at the end of the computation and I might
> need to emit something more complex than just the VertexValue, so I'd
> like more control than what the VertexWriter gives me.
> What do you suggest I might do to obtain a handler to a HDFS file (it
> can be in parts aswell) to write to?
> Is there any code I can start looking at?
> Thanks!
> Claudio

View raw message