hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siddharth Tiwari <siddharth.tiw...@live.com>
Subject RE: Collecting MAP output in a Iterator
Date Mon, 20 Aug 2012 21:46:54 GMT

Hi Bertrand,

You got me spot on,
I wanted to do a cross Join.
I have some data related to a particular type of payment card, we need to find all types of
txn user did during the day. So in one file I have something linke below :-

USER MERCHANT ANMT TXN TS LAT LONG   -- actual big log

so we want a report want a report which shows how many events happened at what day.
so that way we can see how active was teh user in aparticular hour on a particular day



*------------------------*

Cheers !!!

Siddharth Tiwari

Have a refreshing day !!!
"Every duty is holy, and devotion to duty is the highest form of worship of God.” 

"Maybe other people will try to limit me but I don't limit myself"


Date: Mon, 20 Aug 2012 22:28:50 +0200
Subject: Re: Collecting MAP output in a Iterator
From: dechouxb@gmail.com
To: user@hadoop.apache.org

The question is why do you want to do that? How will you use the output?You really want a
cross join? This a very expensive operation (in terms of memory).
Could you provide more context?

Regards
Bertrand

On Mon, Aug 20, 2012 at 10:24 PM, Lance Norskog <goksron@gmail.com> wrote:

The reducer has to open MAP2 explicitly. If it is a regular file, it

has to be in distributed cache.



On Mon, Aug 20, 2012 at 12:56 PM, Siddharth Tiwari

<siddharth.tiwari@live.com> wrote:

>

> Hi

>

> I have two mappers MAP1 and MAP2, which collect data from two different

> files,

> In reducer I want to traverse all keys and values of MAP2 for each key and

> value of  MAP1.

> How can I achieve it in one Reducer.

>

> Please suggest

>

>

> *------------------------*

> Cheers !!!

> Siddharth Tiwari

> Have a refreshing day !!!

> "Every duty is holy, and devotion to duty is the highest form of worship of

> God.”

> "Maybe other people will try to limit me but I don't limit myself"







--

Lance Norskog

goksron@gmail.com



-- 
Bertrand Dechoux

 		 	   		  
Mime
View raw message