hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9918) Erasure Coding: Sort located striped blocks based on decommissioned states
Date Mon, 14 Mar 2016 06:56:33 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15192825#comment-15192825

Zhe Zhang commented on HDFS-9918:

Thanks for working on this Rakesh. A few comments:
# When blkList is null or empty, we shouldn't call the sort method
      if (blkList == null || blkList.size() == 0 ||
          blkList.get(0) instanceof LocatedStripedBlock) {
        // sort located striped blocks based on decommissioned states
# For the sorting logic, how about using a {{HashMap}} to map an index to a set of locations,
and for each index, picking the non-decomm location to put first? It will probably use more
memory than the current method but structure is clearer.

# {{sortEachStripedBlock}} should be {{sortStripedBlock}}
# {{locatedblocks}} should be {{locatedBlocks}} -- can change {{sortLocatedBlocks}} too, up
to you.
# Misspell {{decommisioned}}

> Erasure Coding: Sort located striped blocks based on decommissioned states
> --------------------------------------------------------------------------
>                 Key: HDFS-9918
>                 URL: https://issues.apache.org/jira/browse/HDFS-9918
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Rakesh R
>            Assignee: Rakesh R
>         Attachments: HDFS-9918-001.patch
> This jira is a follow-on work of HDFS-8786, where we do decommissioning of datanodes
having striped blocks.
> Now, after decommissioning it requires to change the ordering of the storage list so
that the decommissioned datanodes should only be last node in list.
> For example, assume we have a block group with storage list:-
> d0, d1, d2, d3, d4, d5, d6, d7, d8, d9
> mapping to indices
> 0, 1, 2, 3, 4, 5, 6, 7, 8, 2
> Here the internal block b2 is duplicated, locating in d2 and d9. If d2 is a decommissioning
node then should switch d2 and d9 in the storage list.
> Thanks [~jingzhao] for the [discussions|https://issues.apache.org/jira/browse/HDFS-8786?focusedCommentId=15180415&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15180415]

This message was sent by Atlassian JIRA

View raw message