Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6A9E54A31 for ; Mon, 23 May 2011 19:14:57 +0000 (UTC) Received: (qmail 37612 invoked by uid 500); 23 May 2011 19:14:56 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 37578 invoked by uid 500); 23 May 2011 19:14:56 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 37570 invoked by uid 99); 23 May 2011 19:14:56 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 May 2011 19:14:56 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of mspreitz@us.ibm.com designates 32.97.182.143 as permitted sender) Received: from [32.97.182.143] (HELO e3.ny.us.ibm.com) (32.97.182.143) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 May 2011 19:14:47 +0000 Received: from d01relay04.pok.ibm.com (d01relay04.pok.ibm.com [9.56.227.236]) by e3.ny.us.ibm.com (8.14.4/8.13.1) with ESMTP id p4NIqR36016100 for ; Mon, 23 May 2011 14:52:27 -0400 Received: from d01av05.pok.ibm.com (d01av05.pok.ibm.com [9.56.224.195]) by d01relay04.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id p4NJEQOa110140 for ; Mon, 23 May 2011 15:14:26 -0400 Received: from d01av05.pok.ibm.com (loopback [127.0.0.1]) by d01av05.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id p4NJEQIw020489 for ; Mon, 23 May 2011 15:14:26 -0400 Received: from d01ml604.pok.ibm.com (d01ml604.pok.ibm.com [9.56.227.90]) by d01av05.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id p4NJEQVT020486 for ; Mon, 23 May 2011 15:14:26 -0400 In-Reply-To: References: To: mapreduce-user@hadoop.apache.org MIME-Version: 1.0 Subject: Re: Stupid questions about combiners in ...hadoop.mapreduce X-KeepSent: 39D1345B:A3D09716-85257899:00694B54; type=4; name=$KeepSent X-Mailer: Lotus Notes Release 8.5.1 September 28, 2009 From: Mike Spreitzer Message-ID: Date: Mon, 23 May 2011 15:14:25 -0400 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Build V853_CD4_03082011|March 08, 2011) at 05/23/2011 15:14:25, Serialize complete at 05/23/2011 15:14:25 Content-Type: multipart/alternative; boundary="=_alternative 0069B0CF85257899_=" This is a multipart message in MIME format. --=_alternative 0069B0CF85257899_= Content-Type: text/plain; charset="US-ASCII" Question 1 remains: What happens if one invocation of a combiner outputs more than one value? My main interest in question 2 was about instances not classes, so let me rephrase question 2 this way: What happens if an output key object is not equal to the input key object (even though both are of the same class)? Even for question 3, I did not exactly see an answer to "what happens" --- only a statement that I should not exercise that case. Thanks, Mike From: Ted Yu To: mapreduce-user@hadoop.apache.org Date: 05/23/2011 03:04 PM Subject: Re: Stupid questions about combiners in ...hadoop.mapreduce Questions 2 and 3 can be answered relatively easily: Remember, the output of the combiner is going to be consumed by the reducer. So the output key/vlaue classes of the combiner have to align with the input key/vlaue classes of the reducer. On Mon, May 23, 2011 at 11:32 AM, Mike Spreitzer wrote: In general, the Java interfaces say that one invocation of a combiner (technically, a Class) can output multiple (key,value) pairs. So: What happens if one invocation of a combiner outputs more than one value? What happens if an output key is different from the input key? What happens if an output value is of a different class than the class of the input values? Thanks, Mike Spreitzer --=_alternative 0069B0CF85257899_= Content-Type: text/html; charset="US-ASCII" Question 1 remains: What happens if one invocation of a combiner outputs more than one value?

My main interest in question 2 was about instances not classes, so let me rephrase question 2 this way: What happens if an output key object is not equal to the input key object (even though both are of the same class)?

Even for question 3, I did not exactly see an answer to "what happens" --- only a statement that I should not exercise that case.

Thanks,
Mike



From:        Ted Yu <yuzhihong@gmail.com>
To:        mapreduce-user@hadoop.apache.org
Date:        05/23/2011 03:04 PM
Subject:        Re: Stupid questions about combiners in ...hadoop.mapreduce



Questions 2 and 3 can be answered relatively easily:
Remember, the output of the combiner is going to be consumed by the reducer.
So the output key/vlaue classes of the combiner have to align with the input key/vlaue classes of the reducer.
On Mon, May 23, 2011 at 11:32 AM, Mike Spreitzer <mspreitz@us.ibm.com> wrote:
In general, the Java interfaces say that one invocation of a combiner (technically, a Class<? extends Reducer>) can output multiple (key,value) pairs.  So:

What happens if one invocation of a combiner outputs more than one value?

What happens if an output key is different from the input key?

What happens if an output value is of a different class than the class of the input values?

Thanks,
Mike Spreitzer

--=_alternative 0069B0CF85257899_=--