From user-return-17921-apmail-hadoop-user-archive=hadoop.apache.org@hadoop.apache.org Fri Dec 12 06:38:52 2014 Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 554969338 for ; Fri, 12 Dec 2014 06:38:52 +0000 (UTC) Received: (qmail 48436 invoked by uid 500); 12 Dec 2014 06:38:47 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 48282 invoked by uid 500); 12 Dec 2014 06:38:47 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 48272 invoked by uid 99); 12 Dec 2014 06:38:45 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 12 Dec 2014 06:38:45 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,MIME_QP_LONG_LINE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of navaz.enc@gmail.com designates 209.85.218.46 as permitted sender) Received: from [209.85.218.46] (HELO mail-oi0-f46.google.com) (209.85.218.46) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 12 Dec 2014 06:38:18 +0000 Received: by mail-oi0-f46.google.com with SMTP id h136so4844143oig.19 for ; Thu, 11 Dec 2014 22:36:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:message-id:thread-topic :mime-version:content-type; bh=ONvp3TaetakG7KyjP1ocReJiMcrM/L5qPQIkNldugfI=; b=E/8cieyMRPUPh5FK1lkMVRxtuhADyIwyJXOM6caESU39i0FmQ0F8U4Fzc5NgGPRWez Y7xip5OjmF/n2VbaYHlQtUwlYOzetMA/pCGfSULE3WIcPBIl9xl7qid6V4cyqqCBre3a kHLdcclc+EsrJmNjQj3J9JZM2SgvjKEs2LJBTZfheA/ZpcPKfe8ryEnJMHieqv5NSpqj kSfPMqdZK2NEB+O0yz8zBWL9YWFsjOsHxlhFIGqSEvTGwf85jfcO/wKTptsxUZFdur+0 UnysTK+d3o7we0ihPLaxJdQ3bMvmvbuwEdOOTJbV+myZvk+XZrjbBXrjaOUq7VSE4hKj 3qLQ== X-Received: by 10.202.72.85 with SMTP id v82mr8789915oia.14.1418366203972; Thu, 11 Dec 2014 22:36:43 -0800 (PST) Received: from [10.0.0.4] (c-98-201-145-147.hsd1.tx.comcast.net. [98.201.145.147]) by mx.google.com with ESMTPSA id l10sm231119oev.7.2014.12.11.22.36.41 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 11 Dec 2014 22:36:42 -0800 (PST) User-Agent: Microsoft-MacOutlook/14.4.5.141003 Date: Fri, 12 Dec 2014 00:36:39 -0600 Subject: Where the output of mappers are saved ? From: Abdul Navaz To: "user@hadoop.apache.org" Message-ID: Thread-Topic: Where the output of mappers are saved ? Mime-version: 1.0 Content-type: multipart/alternative; boundary="B_3501189401_5850518" X-Virus-Checked: Checked by ClamAV on apache.org > This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3501189401_5850518 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Hello, I am interested in efficiently manage the Hadoop shuffling traffic and utilize the network bandwidth effectively. To do this I want to know how much shuffling traffic generated by each Datanodes ? Shuffling traffic is nothing but the output of mappers. So where this mapper output is saved ? How can i get the size of mapper output from each datanodes in a real time ? Appreciate your help. Thanks & Regards, Abdul Navaz --B_3501189401_5850518 Content-type: text/html; charset="US-ASCII" Content-transfer-encoding: quoted-printable
Hello,


I am interested in efficient= ly manage the Hadoop shuffling traffic and utilize the network bandwidth eff= ectively. To do this I want to know how much shuffling traffic generated by = each Datanodes ? Shuffling traffic is nothing but the output of mappers. So = where this mapper output is saved ? How can i get the size of mapper output = from each datanodes in a real time ? 

Appreciate your help.

<= p style=3D"margin: 0px 0px 1em; padding: 0px; border: 0px; vertical-align: bas= eline; clear: both; color: rgb(0, 0, 0); font-family: Arial, 'Liberation San= s', 'DejaVu Sans', sans-serif; line-height: 17.8048000335693px; background-c= olor: rgb(255, 255, 255); background-position: initial initial; background-r= epeat: initial initial;">

Thanks & Regards,
=

Abdul Navaz

--B_3501189401_5850518--