hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinayakumar B (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6247) Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer
Date Fri, 25 Jul 2014 04:17:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14074047#comment-14074047

Vinayakumar B commented on HDFS-6247:

Hi [~clamb], Thanks for taking a look at the patch.

bq. I'm curious about why you are using a 5sec heartbeat interval. That seems small relative
to the timeout on the socket.
 I thought it will be good enough to send the status. Since the total number of block movements
will be limited in balancing by bandwidth, I felt 5 second interval will not going to add
too much traffic.
How much you want to me increase? 30 sec would be fine?

> Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer
> ------------------------------------------------------------------------------------
>                 Key: HDFS-6247
>                 URL: https://issues.apache.org/jira/browse/HDFS-6247
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: balancer, datanode
>    Affects Versions: 2.4.0
>            Reporter: Vinayakumar B
>            Assignee: Vinayakumar B
>         Attachments: HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch
> Currently there is no response sent from target Datanode to Balancer for the replaceBlock()
> Since the Block movement for balancing is throttled, complete block movement will take
time and this could result in timeout at Balancer, which will be trying to read the status
> To Avoid this during replaceBlock() call in in progress Datanode  can send IN_PROGRESS
status messages to Balancer to avoid timeouts and treat BlockMovement as  failed.

This message was sent by Atlassian JIRA

View raw message