hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on
Date Mon, 07 Nov 2016 05:50:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15643205#comment-15643205

Kai Zheng commented on HDFS-10885:

Sorry for the late on this.

bq. Again, StoragePolicySatisfier is kept under BlockManager. BlockManager doesn't have FSNamesystem
reference to call #getFileInfo() and this function is not visible.

It looks to me more natural that we have {{StoragePolicySatisfier}} as a first citizen class
(not just under BM) under NameNode since it needs to access both BM and the name system. I'm
thinking that in future we may extend {{SPS}} to do more NN wide things. 

Or at least, we don't have to put the logic codes related to do MOVER_ID checking in the {{SPS}}
class, instead around some place in the stack of calling {{SPS.start/run}}.

bq. it's quite strange for SPS to create a file even using FSNamesystem APIs
I'm not sure it's a good idea to create the MOVER_ID file here in SPS, because it may be the
duty of {{mover}} to create the flag file. And also, not sure how easy it is to do the creation
in NN without inviting race or tricky condition.

> [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on
> ----------------------------------------------------------------------------------
>                 Key: HDFS-10885
>                 URL: https://issues.apache.org/jira/browse/HDFS-10885
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>    Affects Versions: HDFS-10285
>            Reporter: Wei Zhou
>            Assignee: Wei Zhou
>             Fix For: HDFS-10285
>         Attachments: HDFS-10800-HDFS-10885-00.patch, HDFS-10800-HDFS-10885-01.patch,
HDFS-10800-HDFS-10885-02.patch, HDFS-10885-HDFS-10285.03.patch, HDFS-10885-HDFS-10285.04.patch,
> These two can not work at the same time to avoid conflicts and fight with each other.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message