hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-6732) Improve FsShell's heap consumption by switching to listStatus that returns an iterator
Date Sat, 22 Sep 2012 09:46:08 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Harsh J updated HADOOP-6732:
----------------------------

    Fix Version/s:     (was: 0.24.0)
    
> Improve FsShell's heap consumption by switching to listStatus that returns an iterator
> --------------------------------------------------------------------------------------
>
>                 Key: HADOOP-6732
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6732
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Hairong Kuang
>            Assignee: Daryn Sharp
>
> When listing a large directory from the command line using the default heap configuration,
FsShell often runs out of memory. This is because all stats of the entries under the directory
need to be in memory before printing them. The new API listStatus that returns an iterator
of FileStatus, which implemented in HDFS-1091, no longer requires that all entries are fetched
first. Thus switching to this new API will greatly improve the use of heap space.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message