Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A27A3413D for ; Mon, 23 May 2011 19:03:52 +0000 (UTC) Received: (qmail 8837 invoked by uid 500); 23 May 2011 19:03:51 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 8799 invoked by uid 500); 23 May 2011 19:03:51 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 8791 invoked by uid 99); 23 May 2011 19:03:51 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 May 2011 19:03:51 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of yuzhihong@gmail.com designates 209.85.210.48 as permitted sender) Received: from [209.85.210.48] (HELO mail-pz0-f48.google.com) (209.85.210.48) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 May 2011 19:03:44 +0000 Received: by pzk10 with SMTP id 10so3852346pzk.35 for ; Mon, 23 May 2011 12:03:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=o6PDTLihsiSzAKq9Us/U/Hpzmk83w1ibSoGFoo7ZLZo=; b=l6HPpijVZf12O0MCUuHR45hKvbNujzBlkeg4PB5deQS/DTuXdOqjLZDBPiH2TfFPDJ pU9NTjAb5qYwGEvHavRtUh4x+fuGwE951FYV4R8wjuW3+Lk+jzAOho34Nh1RQcgmXDMp S8kF870KJP48EbzYhGTrEhbiUx9o6I6oOuxmE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=PEY25uhCwhOMYNW3jJmsYJKk9LQBKo97FQadCwJ5j+xnV7eVBEYKnVDK4pUCHdnwXb wSWB88fMa1ccSzSHOMKH/mxkRr5BgK1HepQTaUbyC1UUe1NiNVvMb3pV4p1lKpAawH1j 4CP8CraIO+U5x5gAfLF+xeGxPgVdGLLPsazgM= MIME-Version: 1.0 Received: by 10.68.9.196 with SMTP id c4mr2302494pbb.461.1306177402651; Mon, 23 May 2011 12:03:22 -0700 (PDT) Received: by 10.68.58.228 with HTTP; Mon, 23 May 2011 12:03:22 -0700 (PDT) In-Reply-To: References: Date: Mon, 23 May 2011 12:03:22 -0700 Message-ID: Subject: Re: Stupid questions about combiners in ...hadoop.mapreduce From: Ted Yu To: mapreduce-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=bcaec52162311944f404a3f620f5 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec52162311944f404a3f620f5 Content-Type: text/plain; charset=ISO-8859-1 Questions 2 and 3 can be answered relatively easily: Remember, the output of the combiner is going to be consumed by the reducer. So the output key/vlaue classes of the combiner have to align with the input key/vlaue classes of the reducer. On Mon, May 23, 2011 at 11:32 AM, Mike Spreitzer wrote: > In general, the Java interfaces say that one invocation of a combiner > (technically, a Class) can output multiple > (key,value) pairs. So: > > What happens if one invocation of a combiner outputs more than one value? > > What happens if an output key is different from the input key? > > What happens if an output value is of a different class than the class of > the input values? > > Thanks, > Mike Spreitzer > --bcaec52162311944f404a3f620f5 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Questions 2 and 3 can be answered relatively easily:
Remember, the outpu= t of the combiner is going to be consumed by the reducer.
So the output = key/vlaue classes of the combiner have to align with the input key/vlaue cl= asses of the reducer.

On Mon, May 23, 2011 at 11:32 AM, Mike Sprei= tzer <mspreitz@= us.ibm.com> wrote:
In general, the Java interfaces say th= at one invocation of a combiner (technically, a Cl= ass<? extends Reducer>) can output mu= ltiple (key,value) pairs. =A0So:

What happens if one invocation of = a combiner outputs more than one value?

What happens if an output key is d= ifferent from the input key?

What happens if an output value is= of a different class than the class of the input values?

Thanks,
Mike Sprei= tzer

--bcaec52162311944f404a3f620f5--