hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3203) TaskTracker::localizeJob doesn't provide the correct size to LocalDirAllocator
Date Wed, 09 Apr 2008 17:35:26 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12587272#action_12587272
] 

Tsz Wo (Nicholas), SZE commented on HADOOP-3203:
------------------------------------------------

If we use FileSystem.listStatus(Path), it requires 'r' permission in the parent dir.  It seems
that this additional permission requirement is not necessary.

For saving RPCs, do we really need the size of job.xml?  In most cases, the file size is <
20kB.

> TaskTracker::localizeJob doesn't provide the correct size to LocalDirAllocator
> ------------------------------------------------------------------------------
>
>                 Key: HADOOP-3203
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3203
>             Project: Hadoop Core
>          Issue Type: Bug
>    Affects Versions: 0.15.3
>            Reporter: Chris Douglas
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.18.0
>
>         Attachments: patch-3203.txt
>
>
> In TaskTracker::localizeJob:
> {code}
>     // Get sizes of JobFile and JarFile
>     // sizes are -1 if they are not present.
>     FileSystem fileSystem = FileSystem.get(fConf);
>     FileStatus status[] = fileSystem.listStatus(new Path(jobFile).getParent());
>     long jarFileSize = -1;
>     long jobFileSize = -1;
>     for(FileStatus stat : status) {
>       if (stat.getPath().toString().contains("job.xml")) {
>         jobFileSize = stat.getLen();
>       } else {
>         jobFileSize = -1;
>       }
>       if (stat.getPath().toString().contains("job.jar")) {
>         jarFileSize = stat.getLen();
>       } else {
>         jarFileSize = -1;
>       }
>     }
> {code}
> One or both of jobFileSize and jarFileSize will be -1.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message