hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "edward yoon" <edw...@udanax.org>
Subject Re: graph data representation for mapreduce
Date Fri, 01 Feb 2008 23:57:05 GMT
I'm not familiar with graph algorithms, but i guess it will need a
matrix, list, multi-list structures, it also will take a time
complexity.

Therefore, I think Map/Reduce can be solution even if a development
could be complicated. I also recommend a Hbase and Matrix package.

On 2/2/08, Peter W. <peter@marketingbrokers.com> wrote:
> Cam,
>
> Making a directed graph in Hadoop is not
> very difficult but traversing live might be
> since the result is a separate file.
>
> Basically, you kick out a destination node
> as your key in the mapper and from nodes as
> intermediate values. Concatenate from values in
> the reducer assigning weights for each edge.
>
> Assigned edge scores come from a computation
> done in the reducer or number passed by key.
>
> This gives a simple but weighted from/to
> depiction and can be experimented with and
> improved by subsequent passes or REST style
> calls in the mapper for mysqldb weights.
>
> Later,
>
> Peter W.
>
> Cam Bazz wrote:
>
> > Hello,
> >
> > I have been long interested in storing graphs, in databases, object
> > databases and lucene like indexes.
> > ....
> >
> > Has anyone done any work on storing and processing graphs with map
> > reduce?
> > If I were to start, where would I start from. I am interested in
> > finding
> > shortest paths in a large graph.
>
>


-- 
B. Regards,
Edward yoon @ NHN, corp.

Mime
View raw message