Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6BB5A10550 for ; Wed, 8 Jan 2014 07:33:23 +0000 (UTC) Received: (qmail 91225 invoked by uid 500); 8 Jan 2014 07:32:58 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 90958 invoked by uid 500); 8 Jan 2014 07:32:57 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 90951 invoked by uid 99); 8 Jan 2014 07:32:56 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Jan 2014 07:32:56 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of justlooks@gmail.com designates 209.85.216.43 as permitted sender) Received: from [209.85.216.43] (HELO mail-qa0-f43.google.com) (209.85.216.43) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Jan 2014 07:32:51 +0000 Received: by mail-qa0-f43.google.com with SMTP id k15so1535102qaq.30 for ; Tue, 07 Jan 2014 23:32:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=80qBQgijIGUY8Xz76zHdjqjkJ8bY4/h/QcXHwgILC1s=; b=MohvaqI4vreIlGB1et3UinwS3Tnf8bcI1zma8ei7sd5aNyWiI5OQzPqX6RFJZKpFJU LzcpIDCA9MBp54dRSGijduiIQRrHuS65VbBve1QKJbb/jikYSfpUtTe3KF6yJvbKSHas ko44LAi+MzX40SWahJe/ilf66PhCqbNKyvTy0xcHPpytO80mVk3AZlBnMXeEViGIdTqo 9QjHshzkTh6bKfFc1D3ocNMabUkEhASvf8Vn7bjxEiF/pkBQ9ZlqaI2ZcgiCLJ1LGxWq YDVOwUJyayew07rWm+UhkOCZtq8IszY6WF6/KnwXsUztJcbeFxTiJ6JiXM6UYoK2EHo/ 8I1g== MIME-Version: 1.0 X-Received: by 10.49.75.137 with SMTP id c9mr5634646qew.27.1389166350781; Tue, 07 Jan 2014 23:32:30 -0800 (PST) Received: by 10.140.20.37 with HTTP; Tue, 7 Jan 2014 23:32:30 -0800 (PST) Date: Wed, 8 Jan 2014 15:32:30 +0800 Message-ID: Subject: issue about how to assiging map output to reducer? From: ch huang To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=047d7bea33aedf6e8404ef707ec4 X-Virus-Checked: Checked by ClamAV on apache.org --047d7bea33aedf6e8404ef707ec4 Content-Type: text/plain; charset=ISO-8859-1 hi,maillist: i look the containers log from " hadoop fs -cat /var/log/hadoop-yarn/apps/root/logs/application_1388730279827_2770/CHBM221_50853" and log say it get 25 map output , and assiging 7 to fetcher 5, assiging 7 to fetcher 4 and assiging 11 to fetcher 3,my question is why not " assiging 8 to fetcher 5, assiging 8 to fetcher 4 and assiging 9 to fetcher 3 " ? 2014-01-08 11:28:00,346 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1388730279827_2770_r_000000_0: Got 25 new map-outputs 2014-01-08 11:28:00,348 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Assiging CHBM223:8080 with 7 to fetcher#5 2014-01-08 11:28:00,349 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: assigned 7 of 7 to CHBM223:8080 to fetcher#5 2014-01-08 11:28:00,349 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Assiging CHBM222:8080 with 7 to fetcher#4 2014-01-08 11:28:00,349 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: assigned 7 of 7 to CHBM222:8080 to fetcher#4 2014-01-08 11:28:00,352 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Assiging CHBM221:8080 with 11 to fetcher#3 2014-01-08 11:28:00,352 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: assigned 11 of 11 to CHBM221:8080 to fetcher#3 --047d7bea33aedf6e8404ef707ec4 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
hi,maillist:
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 i look the containers log from "= ; hadoop fs -cat /var/log/hadoop-yarn/apps/root/logs/application_1388730279= 827_2770/CHBM221_50853"
=A0
and log say it get 25 map output , and assiging 7 to fetcher 5, assigi= ng 7 to fetcher 4 and assiging=A011 to fetcher 3,my question is why not
"=A0assiging=A08 to fetcher 5, assiging=A08 to fetcher 4 and assi= ging=A09 to fetcher 3=A0" ?
=A0
2014-01-08 11:28:00,346 INFO [EventFetcher for fetching Map Completion= Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1388= 730279827_2770_r_000000_0: Got 25 new map-outputs
2014-01-08 11:28:00,34= 8 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler= : Assiging CHBM223:8080 with 7 to fetcher#5
2014-01-08 11:28:00,349 INFO [fetcher#5] org.apache.hadoop.mapreduce.task.r= educe.ShuffleScheduler: assigned 7 of 7 to CHBM223:8080 to fetcher#5
201= 4-01-08 11:28:00,349 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.redu= ce.ShuffleScheduler: Assiging CHBM222:8080 with 7 to fetcher#4
2014-01-08 11:28:00,349 INFO [fetcher#4] org.apache.hadoop.mapreduce.task.r= educe.ShuffleScheduler: assigned 7 of 7 to CHBM222:8080 to fetcher#4
201= 4-01-08 11:28:00,352 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.redu= ce.ShuffleScheduler: Assiging CHBM221:8080 with 11 to fetcher#3
2014-01-08 11:28:00,352 INFO [fetcher#3] org.apache.hadoop.mapreduce.task.r= educe.ShuffleScheduler: assigned 11 of 11 to CHBM221:8080 to fetcher#3
<= /div> --047d7bea33aedf6e8404ef707ec4--