hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yiqun Lin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-13264) Hadoop HDFS - DFSOutputStream close method fails to clean up resources in case no hdfs datanodes are accessible
Date Thu, 16 Jun 2016 04:13:05 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15333042#comment-15333042
] 

Yiqun Lin edited comment on HADOOP-13264 at 6/16/16 4:12 AM:
-------------------------------------------------------------

I have looked the code, there were some other places will not release resources associated
with stream. Like in {{DFSOutputStream#abort}}, {{DFSStripedOutputStream#abort}}. We could
make all fixed in this jira. Who can assign this to me, I'd like to post a patch for this.
In addition, this jira seems better to transform to HDFS ranther Hadoop Common.


was (Author: linyiqun):
I have looked the code, there were some other places will not release resources associated
with stream. Like in {{DFSOutputStream#abort}}, {{DFSStripedOutputStream#abort}}. We could
make all fixed in this jira. Assign this jira to me, post a initial patch for this. In addition,
this jira seems better to transform to HDFS ranther Hadoop Common.

> Hadoop HDFS - DFSOutputStream close method fails to clean up resources in case no hdfs
datanodes are accessible 
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-13264
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13264
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.7.2
>            Reporter: Seb Mo
>
> Using:
> hadoop-hdfs\2.7.2\hadoop-hdfs-2.7.2-sources.jar!\org\apache\hadoop\hdfs\DFSOutputStream.java
> Close method fails when the client can't connect to any data nodes. When re-using the
same DistributedFileSystem in the same JVM, if all the datanodes can't be accessed, then this
causes a memory leak as the DFSClient#filesBeingWritten map is never cleared after that.
> See test program provided by [~sebyonthenet] in comments below.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message