hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4161) HDFS keeps a thread open for every file writer
Date Thu, 08 Nov 2012 21:36:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493513#comment-13493513
] 

Tsz Wo (Nicholas), SZE commented on HDFS-4161:
----------------------------------------------

After updated with HDFS-4168, all the failed tests are now passing.
                
> HDFS keeps a thread open for every file writer
> ----------------------------------------------
>
>                 Key: HDFS-4161
>                 URL: https://issues.apache.org/jira/browse/HDFS-4161
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>    Affects Versions: 1.0.0
>            Reporter: Suresh Srinivas
>            Assignee: Tsz Wo (Nicholas), SZE
>         Attachments: h4161_20121108_b-1.patch
>
>
> In 1.0 release DFSClient uses a thread per file writer. In some use cases (dynamic partions
in hive) that use a large number of file writers a large number of threads are created. The
file writer thread has the following stack:
> {noformat}
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hdfs.DFSClient$LeaseChecker.run(DFSClient.java:1462)
> at java.lang.Thread.run(Thread.java:662)
> {noformat}
> This problem has been fixed in later releases. This jira will post a consolidated patch
from various jiras that addresses the issue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message