Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B1E1468C for ; Mon, 20 Aug 2012 21:47:26 +0000 (UTC) Received: (qmail 15660 invoked by uid 500); 20 Aug 2012 21:47:22 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 15580 invoked by uid 500); 20 Aug 2012 21:47:21 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 15573 invoked by uid 99); 20 Aug 2012 21:47:21 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 20 Aug 2012 21:47:21 +0000 X-ASF-Spam-Status: No, hits=3.2 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of siddharth.tiwari@live.com designates 65.54.61.90 as permitted sender) Received: from [65.54.61.90] (HELO snt0-omc2-s39.snt0.hotmail.com) (65.54.61.90) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 20 Aug 2012 21:47:15 +0000 Received: from SNT142-W54 ([65.55.90.72]) by snt0-omc2-s39.snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Mon, 20 Aug 2012 14:46:54 -0700 Message-ID: Content-Type: multipart/alternative; boundary="_3ce698b7-45d7-44f3-8b61-978a0537f8d0_" X-Originating-IP: [180.151.41.42] From: Siddharth Tiwari To: USers Hadoop , Subject: RE: Collecting MAP output in a Iterator Date: Mon, 20 Aug 2012 21:46:54 +0000 Importance: Normal In-Reply-To: References: ,, MIME-Version: 1.0 X-OriginalArrivalTime: 20 Aug 2012 21:46:54.0822 (UTC) FILETIME=[50B1C860:01CD7F1D] X-Virus-Checked: Checked by ClamAV on apache.org --_3ce698b7-45d7-44f3-8b61-978a0537f8d0_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hi Bertrand=2C You got me spot on=2C I wanted to do a cross Join. I have some data related to a particular type of payment card=2C we need to= find all types of txn user did during the day. So in one file I have somet= hing linke below :- USER MERCHANT ANMT TXN TS LAT LONG -- actual big log so we want a report want a report which shows how many events happened at w= hat day. so that way we can see how active was teh user in aparticular hour on a par= ticular day *------------------------* Cheers !!! Siddharth Tiwari Have a refreshing day !!! "Every duty is holy=2C and devotion to duty is the highest form of worship = of God.=94=20 "Maybe other people will try to limit me but I don't limit myself" Date: Mon=2C 20 Aug 2012 22:28:50 +0200 Subject: Re: Collecting MAP output in a Iterator From: dechouxb@gmail.com To: user@hadoop.apache.org The question is why do you want to do that? How will you use the output?You= really want a cross join? This a very expensive operation (in terms of mem= ory). Could you provide more context? Regards Bertrand On Mon=2C Aug 20=2C 2012 at 10:24 PM=2C Lance Norskog w= rote: The reducer has to open MAP2 explicitly. If it is a regular file=2C it has to be in distributed cache. On Mon=2C Aug 20=2C 2012 at 12:56 PM=2C Siddharth Tiwari wrote: > > Hi > > I have two mappers MAP1 and MAP2=2C which collect data from two different > files=2C > In reducer I want to traverse all keys and values of MAP2 for each key an= d > value of MAP1. > How can I achieve it in one Reducer. > > Please suggest > > > *------------------------* > Cheers !!! > Siddharth Tiwari > Have a refreshing day !!! > "Every duty is holy=2C and devotion to duty is the highest form of worshi= p of > God.=94 > "Maybe other people will try to limit me but I don't limit myself" -- Lance Norskog goksron@gmail.com --=20 Bertrand Dechoux = --_3ce698b7-45d7-44f3-8b61-978a0537f8d0_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
Hi Bertrand=2C

Yo= u got me spot on=2C
I wanted to do a cross Join.
I have some data rel= ated to a particular type of payment card=2C we need to find all types of t= xn user did during the day. So in one file I have something linke below :-<= br>
USER MERCHANT ANMT TXN TS LAT LONG =3B =3B -- actual big log=

so we want a report want a report which shows how many events happened at what day.
so tha= t way we can see how active was teh user in aparticular hour on a particula= r day




*-----------------= -------*
= Cheers !!!
 = Siddharth Tiwari Have a r= efreshing day !!!"Every duty is holy=2C and devotion to duty is the highest form of wors= hip of God.=94
"Maybe other people will try to limit me but I do= n't limit myself"


Date: Mon=2C 20 Aug 2012 22:28:50 +0200
Subject: Re: Collecting MA= P output in a Iterator
From: dechouxb@gmail.com
To: user@hadoop.apach= e.org

The question is why do you want to do that? How will you use t= he output?
You really want a cross join? This a very expensive operatio= n (in terms of memory).

Could you provide more con= text?

Regards

Bertrand

On Mon=2C Aug 20=2C 2012 at 10:24 PM=2C Lance Norsk= og <=3Bgoksron@gma= il.com>=3B wrote:
The reducer has to open MAP2 explicitly. If it is a regul= ar file=2C it
has to be in distributed cache.

On Mon=2C Aug 20=2C 2012 at 12:56 PM=2C Siddharth Tiwari
<=3Bsiddharth.tiwari@live.co= m>=3B wrote:
>=3B
>=3B Hi
>=3B
>=3B I have two mappers MAP1 and MAP2=2C which collect data from two diff= erent
>=3B files=2C
>=3B In reducer I want to traverse all keys and values of MAP2 for each k= ey and
>=3B value of  =3BMAP1.
>=3B How can I achieve it in one Reducer.
>=3B
>=3B Please suggest
>=3B
>=3B
>=3B *------------------------*
>=3B Cheers !!!
>=3B Siddharth Tiwari
>=3B Have a refreshing day !!!
>=3B "Every duty is holy=2C and devotion to duty is the highest form of w= orship of
>=3B God.=94
>=3B "Maybe other people will try to limit me but I don't limit myself"


--
Lance Norskog
goksron@gmail.com



-- Bertrand Dechoux
= --_3ce698b7-45d7-44f3-8b61-978a0537f8d0_--