hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rakesh R (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-12570) [SPS]: Refactor Co-ordinator datanode logic to track the block storage movements
Date Fri, 06 Oct 2017 04:36:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16194124#comment-16194124
] 

Rakesh R edited comment on HDFS-12570 at 10/6/17 4:35 AM:
----------------------------------------------------------

Thanks [~surendrasingh] for the reviews.

Attached latest patch fixing checkstyle, findbug, test case failure and addressed (1) and
(2) comments of Surendra.

comment-1) => Actually {{attemptedItemInfo.allBlockLocsAttemptedToSatisfy}} attribute is
not required as SPS will re-check the trackId/fileId once it receives movementreport from
DN. So, I've removed this flag.

comment-2) => Done

comment-3) => Interesting thought. In my current patch, kept blk movement status reporting
simple via heartbeat. As we know, IBR is reporting block received and deleted items only.
We need to change the semantics of #blockReceivedAndDeleted call to incorporate the block
move finished status. Let me think for a while and get back to this point.


was (Author: rakeshr):
Thanks [~surendrasingh] for the reviews.

Attached latest patch fixing checkstyle, findbug, test case failure and addressed (1) and
(2) comments of Surendra.

comment-1) => Actually {{attemptedItemInfo.allBlockLocsAttemptedToSatisfy }} attribute
is not required as SPS will re-check the trackId/fileId once it receives movementreport from
DN. So, I've removed this flag.

comment-2) => Done

comment-3) => Interesting thought. In my current patch, kept blk movement status reporting
simple via heartbeat. As we know, IBR is reporting block received and deleted items only.
We need to change the semantics of #blockReceivedAndDeleted call to incorporate the block
move finished status. Let me think for a while and get back to this point.

> [SPS]: Refactor Co-ordinator datanode logic to track the block storage movements
> --------------------------------------------------------------------------------
>
>                 Key: HDFS-12570
>                 URL: https://issues.apache.org/jira/browse/HDFS-12570
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>            Reporter: Rakesh R
>            Assignee: Rakesh R
>         Attachments: HDFS-12570-HDFS-10285-00.patch, HDFS-12570-HDFS-10285-01.patch
>
>
> This task is to refactor the C-DN block storage movements. Basically, the idea is to
move the scheduling and tracking logic to Namenode rather than at the special C-DN. Please
refer the discussion with [~andrew.wang] to understand the [background and the necessity of
refactoring|https://issues.apache.org/jira/browse/HDFS-10285?focusedCommentId=16141060&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16141060].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message