hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Evans <ev...@yahoo-inc.com>
Subject Re: Keeping intermediate results
Date Mon, 07 Nov 2011 16:18:21 GMT
So you are processing a graph looking for a specific node with a given value.  The special
key is about all I can think of or if you don't need the rest of the graph you can look at
just outputting the nodes that you care about.  If you are doing graph processing you might
want to look at http://incubator.apache.org/giraph/  Even if you don't use their tool they
might be able to answer your question a lot better then I can.

--Bobby Evans

On 11/6/11 10:21 AM, "Yaron Gonen" <yaron.gonen@gmail.com> wrote:

Suppose I have chained M/R jobs that traverse a graph and look for nodes with a specific value.
Every time a Map encounters that value, I'd like to keep that node in the final result.
I can of course save it with a special key and use a condition in the Reducer, but is there
a more formal or elegant way?


View raw message