Return-Path: 
 <mapreduce-user-return-5623-apmail-hadoop-mapreduce-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 87BBDD2EA
	for <apmail-hadoop-mapreduce-user-archive@minotaur.apache.org>;
 Fri, 24 May 2013 10:54:31 +0000 (UTC)
Received: (qmail 77401 invoked by uid 500); 24 May 2013 10:54:30 -0000
Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org
Received: (qmail 77269 invoked by uid 500); 24 May 2013 10:54:26 -0000
Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:mapreduce-user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:mapreduce-user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:mapreduce-user@hadoop.apache.org>
List-Id: <mapreduce-user.hadoop.apache.org>
Reply-To: mapreduce-user@hadoop.apache.org
Delivered-To: mailing list mapreduce-user@hadoop.apache.org
Received: (qmail 77222 invoked by uid 99); 24 May 2013 10:54:25 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 May 2013 10:54:25 +0000
X-ASF-Spam-Status: No, hits=1.7 required=5.0
	tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of psdc1978@gmail.com designates
 209.85.212.172 as permitted sender)
Received: from [209.85.212.172] (HELO mail-wi0-f172.google.com)
 (209.85.212.172)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 May 2013 10:54:20 +0000
Received: by mail-wi0-f172.google.com with SMTP id m6so1371961wiv.17
        for <mapreduce-user@hadoop.apache.org>;
 Fri, 24 May 2013 03:53:59 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:from:date:message-id:subject:to:content-type;
        bh=tcTaGPthq6uRm9qDhhLE+lyY2ls5lcB7eIa3kd4Fvh4=;
        b=bu5xprzFgYiZB95UkJPfDyUTkGJ0i4XUbz3JSKexVb7VA1aIB6+kSiQJ/hRcmy3ZTZ
         rk/PgJ2Oj7f77PLSEIEfva9Gk3oCT/Z0s82uKNFXSjNdM52VnrG+Q96M3wh087OYtURP
         cZLZh7FxfYGMB+T1/fqfspLhSOqkiSi85nim0yaMs1BUPVLIqDLDZCE9+9cLJMl57EOk
         bsT2p/rxzQgzApensUkF+mgn4spscfVq0f1uAAHxaGaYb5MAvZEjukRtkK1eHQnbp+Zq
         e5TT15GSbAY4VuC1cTPsmUb4IfxDRBCinS1e4mdbu5BvuUD7eEFTDc3wp2xM1oWTksjD
         JGtg==
X-Received: by 10.180.205.178 with SMTP id lh18mr841105wic.47.1369392839045;
 Fri, 24 May 2013 03:53:59 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.180.184.107 with HTTP; Fri, 24 May 2013 03:53:29 -0700 (PDT)
From: =?ISO-8859-1?Q?Pedro_S=E1_da_Costa?= <psdc1978@gmail.com>
Date: Fri, 24 May 2013 12:53:29 +0200
Message-ID: 
 <CAJ8VZfWm-uGyuPgcs1kBaPu9mkKrBnsSgFmUyMOp+na4Gcv5Fg@mail.gmail.com>
Subject: HDFS counters
To: mapreduce-user <mapreduce-user@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=001a11c37b60bab86d04dd749dea
X-Virus-Checked: Checked by ClamAV on apache.org

--001a11c37b60bab86d04dd749dea
Content-Type: text/plain; charset=ISO-8859-1

I am analyzing some HDFS counters, and I have these questions?

1 - The "HDFS: Number of bytes read" as long as the map tasks read data
from the HDFS, or it is a pre-calculated sum before the mappers start to
read?

2 - With these metrics, it was written some data in the HDFS before the map
tasks start. Does anyone have an opinion if it is possible the map tasks
write the intermediate output in thi HDFS? This happens because this job
defined by the user forces to (I don't know what this job does)?

<mapcompletion>map() completion: 0.9946828</mapcompletion>
<redcompletion>reduce() completion: 0.0</redcompletion>
<hdfs>HDFS: Number of bytes read=314470180</hdfs>
<hdfs>HDFS: Number of bytes written=313912087</hdfs>

-- 
Best regards,

--001a11c37b60bab86d04dd749dea
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I am analyzing some HDFS counters, and I have these questi=
ons?<br><div><div><br>1 - The &quot;HDFS: Number of bytes read&quot; as lon=
g as the map tasks read data from the HDFS, or it is a pre-calculated sum b=
efore the mappers start to read?<br clear=3D"all">

<br>2 - With these metrics, it was written some data in the HDFS before the=
 map tasks start. Does anyone have an opinion if it is possible the map tas=
ks write the intermediate output in thi HDFS? This happens because this job=
 defined by the user forces to (I don&#39;t know what this job does)?<br>

<br>&lt;mapcompletion&gt;map() completion: 0.9946828&lt;/mapcompletion&gt;<=
br>&lt;redcompletion&gt;reduce() completion: 0.0&lt;/redcompletion&gt;<br>&=
lt;hdfs&gt;HDFS: Number of bytes read=3D314470180&lt;/hdfs&gt;<br>&lt;hdfs&=
gt;HDFS: Number of bytes written=3D313912087&lt;/hdfs&gt;<br>

<br>-- <br>Best regards,<br>
</div></div></div>

--001a11c37b60bab86d04dd749dea--