hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Walter Su (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9040) Erasure coding: Refactor DFSStripedOutputStream (Move Namenode RPC Requests to Coordinator)
Date Thu, 24 Sep 2015 01:46:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14905659#comment-14905659
] 

Walter Su commented on HDFS-9040:
---------------------------------

bq. The socket timeout is 60s by default.
Yeah. It turns out it's in the test case set to 6s. My mistake. So you wait at most 58s? How
about wait at most halfSocketTimeout? I'm afraid 2s left is not enough for healthy streamers
to start send packets when client machine is high-loaded.
{code}
long remaingTime = socketTimeout > 0 ? socketTimeout : Long.MAX_VALUE;
&& remaingTime > waitInterval * 2) {
{code}

> Erasure coding: Refactor DFSStripedOutputStream (Move Namenode RPC Requests to Coordinator)
> -------------------------------------------------------------------------------------------
>
>                 Key: HDFS-9040
>                 URL: https://issues.apache.org/jira/browse/HDFS-9040
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Walter Su
>            Assignee: Jing Zhao
>         Attachments: HDFS-9040-HDFS-7285.002.patch, HDFS-9040-HDFS-7285.003.patch, HDFS-9040-HDFS-7285.004.patch,
HDFS-9040-HDFS-7285.005.patch, HDFS-9040-HDFS-7285.005.patch, HDFS-9040.00.patch, HDFS-9040.001.wip.patch,
HDFS-9040.02.bgstreamer.patch
>
>
> The general idea is to simplify error handling logic.
> -Proposal 1:-
> -A BlockGroupDataStreamer to communicate with NN to allocate/update block, and StripedDataStreamer
s only have to stream blocks to DNs.-
> Proposal 2:
> See below the [comment|https://issues.apache.org/jira/browse/HDFS-9040?focusedCommentId=14741388&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14741388]
from [~jingzhao].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message