hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "omkar vinit joshi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
Date Thu, 21 Mar 2013 00:07:16 GMT

    [ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608430#comment-13608430

omkar vinit joshi commented on YARN-109:

[~mayank_bansal] I looked at the patch. Even though it is not modifying call method logically
but still I see changes there.

Can we simply keep a flag to track all the places in unpack method where renameTo was called
(To indicate not to delete file as it is already deleted). and towards the end before returning
we can delete file based on flag value. For file deletion we can even use File.delete(). I
guess this will make it cleaner and will keep the tmp file handling logic intact as we need
to log an error message in case we are unable to delete the .tmp file.

The code is still not handling situation where resource.getType() == ARCHIVE and it is .jar
or if it is PATTERN and .zip file.

Also I see that entire test code is only testing if the file is a tar file and ARCHIVE type
but not for other extensions / TYPE. I think we need to test all the combinations ( .zip ,
.tar, .jar ) and it is specified as FILE, PATTERN or ARCHIVE.

Also for the test I think you will have to add a timeout parameter. @Test(timeout=<value>)
to avoid -1.

Please let me know your thoughts on this.
> .tmp file is not deleted for localized archives
> -----------------------------------------------
>                 Key: YARN-109
>                 URL: https://issues.apache.org/jira/browse/YARN-109
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 0.23.3, 2.0.0-alpha
>            Reporter: Jason Lowe
>            Assignee: Mayank Bansal
>         Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, YARN-109-trunk.patch
> When archives are localized they are initially created as a .tmp file and unpacked from
that file.  However the .tmp file is not deleted afterwards.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message