hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3592) org.apache.hadoop.fs.FileUtil.copy() will leak input streams if the destination can't be opened
Date Fri, 25 Jul 2008 00:37:31 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616724#action_12616724
] 

Raghu Angadi commented on HADOOP-3592:
--------------------------------------

> So the patch needs work or IOUtils needs to provide a rethrow, cos having io leaks left
around is no good either.

Yes, we should not have leaks, of course. I guess the patch needs work. It would be nice to
have a patch that just fixes the leak you found. A clean up patch can be submitted later either
part of this jira or another one.

{{IOUtils.closeStream()}} is a utility but not a replacement for {{close()}}. 

> org.apache.hadoop.fs.FileUtil.copy() will leak input streams if the destination can't
be opened
> -----------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3592
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3592
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.19.0
>            Reporter: Steve Loughran
>            Assignee: Bill de hOra
>            Priority: Minor
>             Fix For: 0.19.0
>
>         Attachments: HADOOP-3592.patch, HADOOP-3592.patch
>
>
> FileUtil.copy()  relies on IOUtils.copyBytes() to close the incoming streams, which it
does. Normally.
> But if dstFS.create() raises any kind of IOException, then the inputstream "in", which
was created in the line above, will never get closed, and hence be leaked.
>       InputStream in = srcFS.open(src);
>       OutputStream out = dstFS.create(dst, overwrite);
>       IOUtils.copyBytes(in, out, conf, true);
> Some try/catch wrapper around the open operations could close the streams if any exception
gets thrown at that point in the copy process.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message