hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shanu Shushmita <shanu.sushmit...@gmail.com>
Subject Re: Collecting MAP output in a Iterator
Date Mon, 20 Aug 2012 21:52:12 GMT
This article has a good example, which might help:

http://www.ibm.com/developerworks/java/library/j-javadev2-15/index.html

SS
On Aug 20, 2012, at 2:46 PM, Siddharth Tiwari wrote:

> Hi Bertrand,
> 
> You got me spot on,
> I wanted to do a cross Join.
> I have some data related to a particular type of payment card, we need to find all types
of txn user did during the day. So in one file I have something linke below :-
> 
> USER MERCHANT ANMT TXN TS LAT LONG   -- actual big log
> 
> so we want a report want a report which shows how many events happened at what day.
> so that way we can see how active was teh user in aparticular hour on a particular day
> 
> 
> 
> 
> *------------------------*
> Cheers !!!
> Siddharth Tiwari
> Have a refreshing day !!!
> "Every duty is holy, and devotion to duty is the highest form of worship of God.” 
> "Maybe other people will try to limit me but I don't limit myself"
> 
> 
> Date: Mon, 20 Aug 2012 22:28:50 +0200
> Subject: Re: Collecting MAP output in a Iterator
> From: dechouxb@gmail.com
> To: user@hadoop.apache.org
> 
> The question is why do you want to do that? How will you use the output?
> You really want a cross join? This a very expensive operation (in terms of memory).
> 
> Could you provide more context?
> 
> Regards
> 
> Bertrand
> 
> On Mon, Aug 20, 2012 at 10:24 PM, Lance Norskog <goksron@gmail.com> wrote:
> The reducer has to open MAP2 explicitly. If it is a regular file, it
> has to be in distributed cache.
> 
> On Mon, Aug 20, 2012 at 12:56 PM, Siddharth Tiwari
> <siddharth.tiwari@live.com> wrote:
> >
> > Hi
> >
> > I have two mappers MAP1 and MAP2, which collect data from two different
> > files,
> > In reducer I want to traverse all keys and values of MAP2 for each key and
> > value of  MAP1.
> > How can I achieve it in one Reducer.
> >
> > Please suggest
> >
> >
> > *------------------------*
> > Cheers !!!
> > Siddharth Tiwari
> > Have a refreshing day !!!
> > "Every duty is holy, and devotion to duty is the highest form of worship of
> > God.”
> > "Maybe other people will try to limit me but I don't limit myself"
> 
> 
> 
> --
> Lance Norskog
> goksron@gmail.com
> 
> 
> 
> -- 
> Bertrand Dechoux


Mime
View raw message