Return-Path: Delivered-To: apmail-hadoop-core-commits-archive@www.apache.org Received: (qmail 18019 invoked from network); 25 Nov 2008 02:55:30 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 25 Nov 2008 02:55:30 -0000 Received: (qmail 15061 invoked by uid 500); 25 Nov 2008 02:55:40 -0000 Delivered-To: apmail-hadoop-core-commits-archive@hadoop.apache.org Received: (qmail 15022 invoked by uid 500); 25 Nov 2008 02:55:40 -0000 Mailing-List: contact core-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-commits@hadoop.apache.org Received: (qmail 15013 invoked by uid 99); 25 Nov 2008 02:55:40 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 24 Nov 2008 18:55:40 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Nov 2008 02:54:23 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 8D0DA238899B; Mon, 24 Nov 2008 18:54:39 -0800 (PST) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r720386 - in /hadoop/core/trunk: CHANGES.txt src/core/org/apache/hadoop/fs/FileSystem.java src/core/org/apache/hadoop/fs/FsShell.java Date: Tue, 25 Nov 2008 02:54:39 -0000 To: core-commits@hadoop.apache.org From: cdouglas@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20081125025439.8D0DA238899B@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: cdouglas Date: Mon Nov 24 18:54:38 2008 New Revision: 720386 URL: http://svn.apache.org/viewvc?rev=720386&view=rev Log: HADOOP-4339. Remove redundant calls from FileSystem/FsShell when generating/processing ContentSummary. Contributed by David Phillips. Modified: hadoop/core/trunk/CHANGES.txt hadoop/core/trunk/src/core/org/apache/hadoop/fs/FileSystem.java hadoop/core/trunk/src/core/org/apache/hadoop/fs/FsShell.java Modified: hadoop/core/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/core/trunk/CHANGES.txt?rev=720386&r1=720385&r2=720386&view=diff ============================================================================== --- hadoop/core/trunk/CHANGES.txt (original) +++ hadoop/core/trunk/CHANGES.txt Mon Nov 24 18:54:38 2008 @@ -129,6 +129,9 @@ HADOOP-4666. Launch reduces only after a few maps have run in the Fair Scheduler. (Matei Zaharia via johan) + HADOOP-4339. Remove redundant calls from FileSystem/FsShell when + generating/processing ContentSummary. (David Phillips via cdouglas) + OPTIMIZATIONS HADOOP-3293. Fixes FileInputFormat to do provide locations for splits Modified: hadoop/core/trunk/src/core/org/apache/hadoop/fs/FileSystem.java URL: http://svn.apache.org/viewvc/hadoop/core/trunk/src/core/org/apache/hadoop/fs/FileSystem.java?rev=720386&r1=720385&r2=720386&view=diff ============================================================================== --- hadoop/core/trunk/src/core/org/apache/hadoop/fs/FileSystem.java (original) +++ hadoop/core/trunk/src/core/org/apache/hadoop/fs/FileSystem.java Mon Nov 24 18:54:38 2008 @@ -693,7 +693,8 @@ // f is a directory long[] summary = {0, 0, 1}; for(FileStatus s : listStatus(f)) { - ContentSummary c = getContentSummary(s.getPath()); + ContentSummary c = s.isDir() ? getContentSummary(s.getPath()) : + new ContentSummary(s.getLen(), 1, 0); summary[0] += c.getLength(); summary[1] += c.getFileCount(); summary[2] += c.getDirectoryCount(); Modified: hadoop/core/trunk/src/core/org/apache/hadoop/fs/FsShell.java URL: http://svn.apache.org/viewvc/hadoop/core/trunk/src/core/org/apache/hadoop/fs/FsShell.java?rev=720386&r1=720385&r2=720386&view=diff ============================================================================== --- hadoop/core/trunk/src/core/org/apache/hadoop/fs/FsShell.java (original) +++ hadoop/core/trunk/src/core/org/apache/hadoop/fs/FsShell.java Mon Nov 24 18:54:38 2008 @@ -654,14 +654,16 @@ System.out.println("Found " + items.length + " items"); int maxLength = 10; + long length[] = new long[items.length]; for (int i = 0; i < items.length; i++) { - String size = String.valueOf(srcFs.getContentSummary(items[i] - .getPath()).getLength()); - if (size.length() > maxLength) maxLength = size.length(); + length[i] = items[i].isDir() ? + srcFs.getContentSummary(items[i].getPath()).getLength() : + items[i].getLen(); + int len = String.valueOf(length[i]).length(); + if (len > maxLength) maxLength = len; } for(int i = 0; i < items.length; i++) { - System.out.printf("%-"+ (maxLength + BORDER) +"d", - srcFs.getContentSummary(items[i].getPath()).getLength()); + System.out.printf("%-"+ (maxLength + BORDER) +"d", length[i]); System.out.println(items[i].getPath()); } }