hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antonio Piccolboni <anto...@piccolboni.info>
Subject Re: Friends of friends with MapReduce
Date Mon, 20 Dec 2010 17:39:41 GMT
For an easy solution, use hive. Let's say your record contains userid and
friendid and the table is called friends
Then you would do
select A.userid , B.friendid from friends A join friends B on (A.friendid =
B user.id)

This is on top of my mind, sorry if some details are off, but I've done it
in the past on large datasets (~100M rows).That's it. Do that in java and
tell me if it isn't at least 50 lines of code.


A

On Sun, Dec 19, 2010 at 6:28 PM, Praveen Bathala <pbathalac@gmail.com>wrote:

> Hi,
>
> I am trying to write a MapReduce code to find friends of friends in a
> social network with MapReduce.
> my data snippet :
>
> 1       41
> 1       7
> 1       100
> 2       64
> 2       65
> 2       42
> 2       86
> 3       54
> 3       24
> 3       16
> 3       43
> 4       39
> 4       52
>
> Here map() goes through line by line, now I get "41" as friend to "1" and I
> want to go to all "41"'s to get his friends. how can I do that.
> Can I jump to any line I need in Mapper class?
>
> Please, Help me with this...
>
> Thanks in advance
> + Praveen

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message