mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Spreitzer <mspre...@us.ibm.com>
Subject Re: shortest-path maintenance
Date Fri, 20 Apr 2012 17:37:40 GMT
I am trying to understand the comparative virtues of map-reduce vs. the 
programming model found in Pregel.  Both cite BSP as inspiration, but 
Pregel's model includes iteration, per-component state, and selective 
activation --- all of which are absent in map-reduce.  While one could 
implement Pregel (or pretty much anything else) on top of Hadoop with 
sufficient additional client code, I am trying to compare the quality of 
doing that with instead taking the programming model of Pregel, simplified 
to apply to key-value data instead of graphs, as the fundamental 
abstraction and building other things (as needed/desired) on top of that. 
As noted in other replies, the additional features of the Pregel model 
look like they could be very beneficial for solving the problem I posed. I 
would like to do an actual comparison.  So I want to know the best way to 
solve this maintenance problem using map-reduce.

Thanks,
Mike



From:   Saikat Kanjilal <sxk1969@hotmail.com>
To:     "user@mahout.apache.org" <user@mahout.apache.org>
Cc:     "user@mahout.apache.org" <user@mahout.apache.org>
Date:   04/20/2012 12:21 PM
Subject:        Re: shortest-path maintenance



Is it a requirement to use map reduce?  Also how does mahout play into 
this?  Potentially you could build mappers that could reference an in 
memory graph and have an API that pre calculates dijkstra or astar up 
front.  You could then add or remove a node as part of the reduce process 
that references this graph and recalculates dijkstra or astar in a closed 
feedback loop.  However its not obvious to me that mapreduce is the 
appropriate tool to do this.

Some more context into the problem and how mahout fits in would be great.

Sent from my iPhone




Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message