Return-Path: Delivered-To: apmail-hadoop-core-user-archive@www.apache.org Received: (qmail 55927 invoked from network); 18 Feb 2009 02:26:54 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 18 Feb 2009 02:26:54 -0000 Received: (qmail 7974 invoked by uid 500); 18 Feb 2009 02:26:49 -0000 Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org Received: (qmail 7928 invoked by uid 500); 18 Feb 2009 02:26:48 -0000 Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-user@hadoop.apache.org Delivered-To: mailing list core-user@hadoop.apache.org Received: (qmail 7917 invoked by uid 99); 18 Feb 2009 02:26:48 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Feb 2009 18:26:48 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of nathan@rapleaf.com designates 216.74.32.93 as permitted sender) Received: from [216.74.32.93] (HELO mail.rapleaf.com) (216.74.32.93) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 Feb 2009 02:26:41 +0000 Received: from mail.rapleaf.com (localhost.localdomain [127.0.0.1]) by mail.rapleaf.com (Postfix) with ESMTP id 53DB8125017A for ; Tue, 17 Feb 2009 18:26:20 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=rapleaf.com; q=dns; s=m1; b=YY9AW ds+a+S8O5qSLSXTxGNDVkwevqnDjNkfiUawvCOePon1polnUuTmxfnBq2BgLMfiA 273zcNjnl2w7LJg0sFEWdODfdAkKI1RZzfA5WsLhQyeuxhZlpe2CkAWDs2BPOUVK kEOBkCKyNP+imF3ymAgenmXZgyMfKiehPpZ6e8= Received: from [192.168.1.78] (unknown [192.168.1.78]) by mail.rapleaf.com (Postfix) with ESMTP id 40EB912500A7 for ; Tue, 17 Feb 2009 18:26:20 -0800 (PST) Message-Id: <1BDFF97B-DCB8-4225-A307-747C887AD9FE@rapleaf.com> From: Nathan Marz To: core-user@hadoop.apache.org Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v926) Subject: HDFS bytes read job counters? Date: Tue, 17 Feb 2009 18:25:47 -0800 X-Mailer: Apple Mail (2.926) X-Virus-Checked: Checked by ClamAV on apache.org Hello, I'm seeing very odd numbers from the HDFS job tracker page. I have a job that operates over approximately 200 GB of data (209715200047 bytes to be exact), and HDFS bytes read is 2,103,170,802,501 (2 TB). The "Map input bytes" is set to "209,714,811,510", which is a correct number. The job only took 10 minutes to run, so there's no way that that much data was actually read. Anyone have any idea of what's going on here? Thanks, Nathan Marz