hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-6963) Fix FileUtil.getDU. It should not include the size of the directory or follow symbolic links
Date Thu, 05 Apr 2012 20:12:23 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247603#comment-13247603

Hudson commented on HADOOP-6963:

Integrated in Hadoop-Hdfs-trunk-Commit #2081 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2081/])
    Moved HADOOP-6963 to 0.23.3 (Revision 1310022)

     Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1310022
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

> Fix FileUtil.getDU. It should not include the size of the directory or follow symbolic
> --------------------------------------------------------------------------------------------
>                 Key: HADOOP-6963
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6963
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions:, 0.23.1
>            Reporter: Owen O'Malley
>            Assignee: Ravi Prakash
>            Priority: Critical
>             Fix For: 1.0.3, 0.23.3
>         Attachments: HADOOP-6963.branch-1.0.2.patch, HADOOP-6963.branch-1.0.2.patch,
HADOOP-6963.branch-1.patch, HADOOP-6963.branch-23.patch, HADOOP-6963.branch-23.patch, HADOOP-6963.branch-23.patch
> The getDU method should not include the size of the directory. The Java interface says
that the value is undefined and in Linux/Sun it gets the 4096 for the inode. Clearly this
isn't useful.
> It also recursively calls itself. In case the directory has a symbolic link forming a
cycle, getDU keeps spinning in the cycle. In our case, we saw this in the org.apache.hadoop.mapred.JobLocalizer.downloadPrivateCacheObjects
call. This prevented other tasks on the same node from committing, causing the TT to become
effectively useless (because the JT thinks it already has enough tasks running)

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message