hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuanbo Liu (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.
Date Fri, 06 Jan 2017 10:20:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15804220#comment-15804220
] 

Yuanbo Liu edited comment on HDFS-11150 at 1/6/17 10:20 AM:
------------------------------------------------------------

uploaded v5 patch for this JIRA.
[~umamaheswararao] / [~rakeshr] please help me review it if you have time. Thanks in advance.
[~umamaheswararao] Due to some scenarios make my restart testing unstable, I've changed some
code of {{StoragePolicySatisfier}}, the basic idea is here:
{code}
expected type = [SSD, DISK, DISK]
current block types = [DISK(datanode A), DISK(datanode B), DISK(datanode C)], their storage
type is disk, and they all can be added into candidate source list.
           ||
expected type = [SSD]
source pairs = [DISK => datanode A, DISK => datanode B, DISK => datanode C]
A, B, C are put into exclude node list.
            ||
if chooseTargetTypeInSameNode = true, then expected type =[], otherwise keep searching by
 chooseTarget
            ||
if the expected type is empty, that means all unsatisfied blocks find target nodes.
{code}


was (Author: yuanbo):
uploaded v5 patch for this JIRA.
[~umamaheswararao] / [~rakeshr] please help me review it if you have time. Thanks in advance.
[~umamaheswararao] Due to some scenarios make my restart testing unstable, I've changed some
code of {{StoragePolicySatisfier}}, the basic idea is here:
{code}
expected type = [SSD, DISK, DISK]
current block types = [DISK(datanode A), DISK(datanode B), DISK(datanode C)]
           ||
expected type = [SSD]
source pairs = [DISK => datanode A, DISK => datanode B, DISK => datanode C]
A, B, C are put into exclude node list.
            ||
if chooseTargetTypeInSameNode = true, then expected type =[], otherwise keep searching by
 chooseTarget
            ||
if the expected type is empty, that means all unsatisfied blocks find target nodes.
{code}

> [SPS]: Provide persistence when satisfying storage policy.
> ----------------------------------------------------------
>
>                 Key: HDFS-11150
>                 URL: https://issues.apache.org/jira/browse/HDFS-11150
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>            Reporter: Yuanbo Liu
>            Assignee: Yuanbo Liu
>         Attachments: HDFS-11150-HDFS-10285.001.patch, HDFS-11150-HDFS-10285.002.patch,
HDFS-11150-HDFS-10285.003.patch, HDFS-11150-HDFS-10285.004.patch, HDFS-11150-HDFS-10285.005.patch,
editsStored, editsStored.xml
>
>
> Provide persistence for SPS in case that Hadoop cluster crashes by accident. Basically
we need to change EditLog and FsImage here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message