hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Esteban Gutierrez (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7608) hdfs dfsclient newConnectedPeer has no write timeout
Date Thu, 14 May 2015 22:18:23 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14544376#comment-14544376
] 

Esteban Gutierrez commented on HDFS-7608:
-----------------------------------------

Thanks [~cnauroth]. I think just changing newConnectedPeer() to use the one provided by  DFSClient.getDatanodeWriteTimeout()
is good enough. In the DataStreamer we already use it and but we have set the number of nodes
to 2. That should be fine and that gives at least the flexibility to tune it down if required.
Also, I see that if we don't set a write timeout we can run into the issue that was mentioned
in this JIRA and after adding the timeout in the peer I no longer experience this issue. I've
noticed other issues like in Client() where we set up the connection and then the timeout
but that can be addressed in another JIRA.

> hdfs dfsclient  newConnectedPeer has no write timeout
> -----------------------------------------------------
>
>                 Key: HDFS-7608
>                 URL: https://issues.apache.org/jira/browse/HDFS-7608
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: fuse-dfs, hdfs-client
>    Affects Versions: 2.3.0, 2.6.0
>         Environment: hdfs 2.3.0  hbase 0.98.6
>            Reporter: zhangshilong
>            Assignee: Xiaoyu Yao
>         Attachments: HDFS-7608.0.patch, HDFS-7608.1.patch, HDFS-7608.2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> problem:
> hbase compactSplitThread may lock forever on  read datanode blocks.
> debug found:  epollwait timeout set to 0,so epollwait can not  run out.
> cause: in hdfs 2.3.0
> hbase using DFSClient to read and write blocks.
> DFSClient  creates one socket using newConnectedPeer(addr), but has no read or write
timeout. 
> in v 2.6.0,  newConnectedPeer has added readTimeout to deal with the problem,but did
not add writeTimeout. why did not add write Timeout?
> I think NioInetPeer need a default socket timeout,so appalications will no need to force
adding timeout by themselives. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message