Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of hailong.yang1115@gmail.com
 designates 209.85.210.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=date:from:to:subject:message-id:x-mailer:mime-version:content-type;
        b=W6UHoLTnz1qT5coOxjhLberdS4aq61w9nGmI01v8J6GlVSGFkiWox+rh553uqoVj+n
         GUTLkFp/Ilyohp7QAHdAfnP6cP1Z13F29tOZ7IwWr56xvwXkMI4WbFXxwCoEyuggBr/5
         TkR8NxVaLlVvETELykZLu3O/jeyw7goqlL9gI=
Date: Fri, 10 Jun 2011 13:28:54 +0800
From: "hailong.yang1115" <hailong.yang1115@gmail.com>
To: "general" <general@hadoop.apache.org>
Subject: Problems about the job counters
Message-ID: <201106101328462304641@gmail.com>
Mime-Version: 1.0
Content-Type: multipart/alternative;
	boundary="=====003_Dragon731067674787_====="

--=====003_Dragon731067674787_=====
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit

Dear all,

I am trying to the built-in example wordcount with nearly 15GB input. When the Hadoop job finished, I got the following counters.


CounterMapReduceTotal
Job CountersLaunched reduce tasks001
Rack-local map tasks0035
Launched map tasks002,318
Data-local map tasks002,283
FileSystemCountersFILE_BYTES_READ22,863,580,65617,654,943,34140,518,523,997
HDFS_BYTES_READ154,400,997,4590154,400,997,459
FILE_BYTES_WRITTEN33,490,829,40317,654,943,34151,145,772,744
HDFS_BYTES_WRITTEN02,747,356,7042,747,356,704


My question is what does the FILE_BYTES_READ counter mean? And what is the difference between FILE_BYTES_READ and HDFS_BYTES_READ? In my opinion, all the input is located in HDFS, so where does FILE_BYTES_READ come from during the map phase?


Any help will be appreciated!

Hailong

2011-06-10 


***********************************************
* Hailong Yang, PhD. Candidate 
* Sino-German Joint Software Institute, 
* School of Computer Science&Engineering, Beihang University
* Phone: (86-010)82315908
* Email: hailong.yang1115@gmail.com
* Address: G413, New Main Building in Beihang University, 
*              No.37 XueYuan Road,HaiDian District, 
*              Beijing,P.R.China,100191
***********************************************

--=====003_Dragon731067674787_=====--