Return-Path: Delivered-To: apmail-hadoop-common-dev-archive@www.apache.org Received: (qmail 35741 invoked from network); 29 Apr 2010 00:38:15 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 29 Apr 2010 00:38:15 -0000 Received: (qmail 60073 invoked by uid 500); 29 Apr 2010 00:38:13 -0000 Delivered-To: apmail-hadoop-common-dev-archive@hadoop.apache.org Received: (qmail 59941 invoked by uid 500); 29 Apr 2010 00:38:13 -0000 Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-dev@hadoop.apache.org Received: (qmail 59923 invoked by uid 99); 29 Apr 2010 00:38:13 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Apr 2010 00:38:13 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.22] (HELO thor.apache.org) (140.211.11.22) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Apr 2010 00:38:11 +0000 Received: from thor (localhost [127.0.0.1]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o3T0bodd004185 for ; Thu, 29 Apr 2010 00:37:50 GMT Message-ID: <7316965.12841272501470184.JavaMail.jira@thor> Date: Wed, 28 Apr 2010 20:37:50 -0400 (EDT) From: "Hairong Kuang (JIRA)" To: common-dev@hadoop.apache.org Subject: [jira] Created: (HADOOP-6732) Improve FsShell's heap consumption by switching to listStatus that returns an iterator MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org Improve FsShell's heap consumption by switching to listStatus that returns an iterator -------------------------------------------------------------------------------------- Key: HADOOP-6732 URL: https://issues.apache.org/jira/browse/HADOOP-6732 Project: Hadoop Common Issue Type: Improvement Reporter: Hairong Kuang Fix For: 0.22.0 When listing a large directory from the command line using the default heap configuration, FsShell often runs out of memory. This is because all stats of the entries under the directory need to be in memory before printing them. The new API listStatus that returns an iterator of FileStatus, which implemented in HDFS-1091, no longer requires that all entries are fetched first. Thus switching to this new API will greatly improve the use of heap space. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.