Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-dev@hadoop.apache.org
Message-ID: <18898603.138911287048094233.JavaMail.jira@thor>
Date: Thu, 14 Oct 2010 05:21:34 -0400 (EDT)
From: "Ravi Gummadi (JIRA)" <jira@apache.org>
To: mapreduce-dev@hadoop.apache.org
Subject: [jira] Created: (MAPREDUCE-2135) FILE_BYTES_WRITTEN counter in map
 task seems incorrect
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

FILE_BYTES_WRITTEN counter in map task seems incorrect
------------------------------------------------------

                 Key: MAPREDUCE-2135
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2135
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: task
            Reporter: Ravi Gummadi


With MapReduce trunk,
 The FileSystem counter FILE_BYTES_WRITTEN is a lot less than "Map output bytes" counter even when map output compression is OFF. I think this FILE_BYTES_WRITTEN signifies the bytes written to local file system. So it should be more than map output bytes(in the counters shown below, 210 Vs 19200000). Right ?

Here are some counters from map task of wordcount example:


Counters for attempt_201010141448_0001_m_000000_0

FileInputFormatCounters
	BYTES_READ 	9,600,000

FileSystemCounters
	FILE_BYTES_READ 	92
	FILE_BYTES_WRITTEN 	210
	HDFS_BYTES_READ 	9,600,107

Map-Reduce Framework
	Combine input records 	2,400,000
	Combine output records 	8
	CPU_MILLISECONDS 	4,810
	Failed Shuffles 	0
	GC time elapsed (ms) 	73
	Map input records 	600,000
	Map output bytes 	19,200,000
	Map output records 	2,400,000
	Merged Map outputs 	0
	PHYSICAL_MEMORY_BYTES 	131,518,464
	Spilled Records 	16
	SPLIT_RAW_BYTES 	107
	VIRTUAL_MEMORY_BYTES 	581,021,696

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.