hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj K (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5895) FileAlreadyExistsException was thrown : Temporary Index File can not be cleaned up because OutputStream doesn't close properly
Date Tue, 27 May 2014 16:29:01 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14009857#comment-14009857
] 

Devaraj K commented on MAPREDUCE-5895:
--------------------------------------

dos.close() closes *bos* when there are no errors/exceptions(other than IOException) during
flush(). 

It is better to close the *bos* before making it as null, with the similar way as we are trying
to close in the finally. 

And also can you make these two times invocations into single method call using var args i.e
*IOUtils.cleanup(LOG, dos, bos);*.
{code:xml}
IOUtils.cleanup(LOG, dos);
IOUtils.cleanup(LOG, bos);
{code}


> FileAlreadyExistsException was thrown : Temporary Index File can not be cleaned up because
OutputStream doesn't close properly
> ------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5895
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5895
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 3.0.0
>            Reporter: Kousuke Saruta
>         Attachments: MAPREDUCE-5895.patch
>
>
> In TaskLog.java, Temporary Index File is created by following code.
> {code}
> BufferedOutputStream bos =
>   new BufferedOutputStream(
>     SecureIOUtils.createForWrite(tmpIndexFile, 0644));
> DataOutputStream dos = new DataOutputStream(bos);
> {code}
> The code is surrounded by try-finally so if some Exception/ERROR is thrown between constructing
bos and dos, temporary file is not cleaned up.
> I met the situation that when a thread ran, OOM was thrown after bos created and temporary
file is not cleaned up. At different time, another thread executed same logic and fail because
of FileAlreadyExistsException.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message