Return-Path: X-Original-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 488F97A9C for ; Fri, 22 Jul 2011 11:53:25 +0000 (UTC) Received: (qmail 48929 invoked by uid 500); 22 Jul 2011 11:53:25 -0000 Delivered-To: apmail-hadoop-mapreduce-issues-archive@hadoop.apache.org Received: (qmail 48767 invoked by uid 500); 22 Jul 2011 11:53:22 -0000 Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-issues@hadoop.apache.org Delivered-To: mailing list mapreduce-issues@hadoop.apache.org Received: (qmail 48759 invoked by uid 99); 22 Jul 2011 11:53:21 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Jul 2011 11:53:21 +0000 X-ASF-Spam-Status: No, hits=-2001.1 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Jul 2011 11:53:19 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id E9618563C8 for ; Fri, 22 Jul 2011 11:52:57 +0000 (UTC) Date: Fri, 22 Jul 2011 11:52:57 +0000 (UTC) From: "Ravi Gummadi (JIRA)" To: mapreduce-issues@hadoop.apache.org Message-ID: <370202504.14463.1311335577952.JavaMail.tomcat@hel.zones.apache.org> In-Reply-To: <296088917.14414.1311334017924.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Updated] (MAPREDUCE-2722) Gridmix simulated job's map's hdfsBytesRead counter is wrong when compressed input is used MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/MAPREDUCE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Gummadi updated MAPREDUCE-2722: ------------------------------------ Attachment: MR2722.patch Attaching patch fixing the bug. The new counter values with this patch for the Gridmix simulated job are(for the example case of previous comment): 100MB as hdfsBytesRead and 200MB as MapInputBytes. Please review and provide your comments. > Gridmix simulated job's map's hdfsBytesRead counter is wrong when compressed input is used > ------------------------------------------------------------------------------------------ > > Key: MAPREDUCE-2722 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2722 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/gridmix > Reporter: Ravi Gummadi > Assignee: Ravi Gummadi > Attachments: MR2722.patch > > > When compressed input was used by original job's map task, then the simulated job's map task's hdfsBytesRead counter is wrong if compression emulation is enabled. This issue is because hdfsBytesRead of map task of original job is considered as uncompressed map input size by Gridmix. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira