hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rakesh R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9918) Erasure Coding: Sort located striped blocks based on decommissioned states
Date Tue, 29 Mar 2016 19:14:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15216677#comment-15216677
] 

Rakesh R commented on HDFS-9918:
--------------------------------

Thanks [~walter.k.su] for the advice, this makes our logic simple. One observation about the
logic is - after sorting, block indices will not be in ascending order even though all locations
are in the same distance. For example, we have below blkIndices list and assume all the datanodes
are In-service state.
{code}
0, 1, 2, 3, 4, 5, 6, 7, 8, 1, 2
{code}

Here 1 and 2 are duplicated. After sorting it will not maintain the blkIndices in ascending
order and the list can be something like,
{code}
8, 7, 1, 6, 2, 4, 1, 5, 3, 0, 2
{code}

I think this shuffling won't create any issues. I submitted new patch including new test case.

[~zhz], would be great to see your feedback. Thanks!

> Erasure Coding: Sort located striped blocks based on decommissioned states
> --------------------------------------------------------------------------
>
>                 Key: HDFS-9918
>                 URL: https://issues.apache.org/jira/browse/HDFS-9918
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Rakesh R
>            Assignee: Rakesh R
>         Attachments: HDFS-9918-001.patch, HDFS-9918-002.patch, HDFS-9918-003.patch, HDFS-9918-004.patch,
HDFS-9918-005.patch, HDFS-9918-006.patch, HDFS-9918-007.patch
>
>
> This jira is a follow-on work of HDFS-8786, where we do decommissioning of datanodes
having striped blocks.
> Now, after decommissioning it requires to change the ordering of the storage list so
that the decommissioned datanodes should only be last node in list.
> For example, assume we have a block group with storage list:-
> d0, d1, d2, d3, d4, d5, d6, d7, d8, d9
> mapping to indices
> 0, 1, 2, 3, 4, 5, 6, 7, 8, 2
> Here the internal block b2 is duplicated, locating in d2 and d9. If d2 is a decommissioning
node then should switch d2 and d9 in the storage list.
> Thanks [~jingzhao] for the [discussions|https://issues.apache.org/jira/browse/HDFS-8786?focusedCommentId=15180415&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15180415]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message