Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <31498708.140061280855477198.JavaMail.jira@thor>
Date: Tue, 3 Aug 2010 13:11:17 -0400 (EDT)
From: "Tsz Wo (Nicholas), SZE (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Updated: (HDFS-1325) DFSClient(DFSInputStream) release the
 persistent connection with datanode when no data have been read for a long
 time
In-Reply-To: <13496559.110881280715077054.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HDFS-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tsz Wo (Nicholas), SZE updated HDFS-1325:
-----------------------------------------

    Hadoop Flags: [Incompatible change]

After the timeout, could the client read again?

> DFSClient(DFSInputStream) release the persistent connection with datanode when no data have been read for a long time
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-1325
>                 URL: https://issues.apache.org/jira/browse/HDFS-1325
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs client
>            Reporter: jinglong.liujl
>             Fix For: 0.20.3
>
>         Attachments: dfsclient.patch, toomanyconnction.patch
>
>
> When you use Hbase over hadoop. We found during scanning over a large table ( which has many regions and each region has many store files), there're too many connections has been kept between regionserver (act as DFSClient) and datanode.  Even if the store file has been complete to scanning, the connections can not be closed.
> In our cluster, too many extra connections cause too many system resource has been wasted, which cause system cpu on region server reach to a high level, then bring this region server down.
> After investigating, we found the number of active connection is very small, and the most connection is idle. We add a timeout checker thread into DFSClient, to close this connection.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.