hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dennis Cheung (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1836) Thousand of CLOSE_WAIT socket
Date Tue, 26 Apr 2011 06:05:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025084#comment-13025084

Dennis Cheung commented on HDFS-1836:

It creates a singleton FileSystem object which is connected to a HDFS cloud.
And it does nothing with hadoop but one simple thing, copy local files into hadoop.

We use only simple FileSystem.create(path, true) and  IOUtils.copyLarge(). And already applied
IOUtils.closeQuietly(out); in the finally block.

Nothing more than that.

> Thousand of CLOSE_WAIT socket 
> ------------------------------
>                 Key: HDFS-1836
>                 URL: https://issues.apache.org/jira/browse/HDFS-1836
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>    Affects Versions: 0.20.2
>         Environment: Linux 2.6.18-194.32.1.el5 #1 SMP Wed Jan 5 17:52:25 EST 2011 x86_64
x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> Java(TM) SE Runtime Environment (build 1.6.0_23-b05)
> Java HotSpot(TM) 64-Bit Server VM (build 19.0-b09, mixed mode)
>            Reporter: Dennis Cheung
> $ /usr/sbin/lsof -i TCP:50010 | grep -c CLOSE_WAIT
> 4471
> It is better if everything runs normal. 
> However, from time to time there are some "DataStreamer Exception: java.net.SocketTimeoutException"
and "DFSClient.processDatanodeError(2507) | Error Recovery for" can be found from log file
and the number of CLOSE_WAIT socket just keep increasing
> The CLOSE_WAIT handles may remain for hours and days; then "Too many open file" some

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message