hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Surendra Singh Lilhore (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11163) Mover should move the file blocks to default storage once policy is unset
Date Tue, 10 Jan 2017 07:49:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15814232#comment-15814232

Surendra Singh Lilhore commented on HDFS-11163:

Thanks [~cnauroth] and [~szetszwo] for comments..

bq. Indeed, the additional RPC is not needed since HdfsLocatedFileStatus already has the resolved
storage policy. We don't need to call getStoragePolicy again.
Yes, HdfsLocatedFileStatus has resolved storage policy. I am calling {{getStoragePolicy}}
because it will give default policy in case of {{BLOCK_STORAGE_POLICY_ID_UNSPECIFIED}}.

  public BlockStoragePolicy getPolicy(byte id) {
    // id == 0 means policy not specified.
    return id == 0? getDefaultPolicy(): policies[id];

we can add one API to get default policy from namenode, so we can avoid {{getStoragePolicy}}
RPC per file.

> Mover should move the file blocks to default storage once policy is unset
> -------------------------------------------------------------------------
>                 Key: HDFS-11163
>                 URL: https://issues.apache.org/jira/browse/HDFS-11163
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: balancer & mover
>    Affects Versions: 2.8.0
>            Reporter: Surendra Singh Lilhore
>            Assignee: Surendra Singh Lilhore
>         Attachments: HDFS-11163-001.patch, HDFS-11163-002.patch
> HDFS-9534 added new API in FileSystem to unset the storage policy. Once policy is unset
blocks should move back to the default storage policy.
> Currently mover is not moving file blocks which have zero storage ID
> {code}
>       // currently we ignore files with unspecified storage policy
>       if (policyId == HdfsConstants.BLOCK_STORAGE_POLICY_ID_UNSPECIFIED) {
>         return;
>       }
> {code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message