Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DDE949CD2 for ; Sat, 20 Dec 2014 17:14:11 +0000 (UTC) Received: (qmail 8867 invoked by uid 500); 20 Dec 2014 17:14:06 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 8729 invoked by uid 500); 20 Dec 2014 17:14:06 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 8718 invoked by uid 99); 20 Dec 2014 17:14:05 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 20 Dec 2014 17:14:05 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [87.230.46.220] (HELO vwp3725.webpack.hosteurope.de) (87.230.46.220) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 20 Dec 2014 17:13:38 +0000 Received: from dslb-188-108-180-250.188.108.pools.vodafone-ip.de ([188.108.180.250] helo=[192.168.2.103]); authenticated by vwp3725.webpack.hosteurope.de running ExIM with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) id 1Y2Nae-00031N-A0; Sat, 20 Dec 2014 18:13:16 +0100 Content-Type: multipart/alternative; boundary="Apple-Mail=_4B0A38B0-F9D4-48FD-86E9-B5351F6AC40D" Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: Combiner doesn't take effect if there is no reducer phrase From: Kai Voigt X-Priority: 3 In-Reply-To: <37be11df.16c37.14a689e1a12.Coremail.bit1129@163.com> Date: Sat, 20 Dec 2014 18:13:15 +0100 Message-Id: <3B763968-FFF3-4D3F-800C-7E93EA341FA0@123.org> References: <37be11df.16c37.14a689e1a12.Coremail.bit1129@163.com> To: user@hadoop.apache.org X-Mailer: Apple Mail (2.1993) X-bounce-key: webpack.hosteurope.de;k@123.org;1419095637;a7fe0999; X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_4B0A38B0-F9D4-48FD-86E9-B5351F6AC40D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Your observation is correct, the combine code will not execute if you = disable the reduce phase, i.e. set numbers of reducers to 0. Actually, the combine code might not be executed with a reduce phase, or = it might be executed more than once on the map output. It=E2=80=99s the = framework=E2=80=99s decision, so don=E2=80=99t put anything into the = combine code you expect to run. Kai > Am 20.12.2014 um 17:53 schrieb Todd : >=20 > Hi, Hadoopers, > It looks to me that when I provide the combiner but doesn't provide = the reducer,the combiner doesn't take effect or combiner doesn't get run = at all. > I don't understand here because the combiner is running on the Mapper = node, so could some one please explain the behavior of Combiner?Thanks. Kai Voigt Am Germaniahafen 1 = k@123.org 24143 Kiel = +49 160 96683050 Germany = @KaiVoigt --Apple-Mail=_4B0A38B0-F9D4-48FD-86E9-B5351F6AC40D Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Your observation is correct, the combine code will not = execute if you disable the reduce phase, i.e. set numbers of reducers to = 0.

Actually, the = combine code might not be executed with a reduce phase, or it might be = executed more than once on the map output. It=E2=80=99s the = framework=E2=80=99s decision, so don=E2=80=99t put anything into the = combine code you expect to run.

Kai

Am = 20.12.2014 um 17:53 schrieb Todd <bit1129@163.com>:

Hi, Hadoopers,
It looks to me = that when I provide the combiner but doesn't provide the reducer,the = combiner doesn't take effect or combiner doesn't get run at all.
I don't understand here because the combiner is running on = the Mapper node, so could some one please explain the behavior of = Combiner?Thanks.

Kai Voigt Am = Germaniahafen 1 k@123.org
= 24143 Kiel +49 160 = 96683050
= Germany = @KaiVoigt

= --Apple-Mail=_4B0A38B0-F9D4-48FD-86E9-B5351F6AC40D--