hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "surendra singh lilhore (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8219) setStoragePolicy with folder behavior is different after cluster restart
Date Mon, 04 May 2015 12:08:06 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526563#comment-14526563
] 

surendra singh lilhore commented on HDFS-8219:
----------------------------------------------

Little change in the scenario, before putting the file in directory first set the storage
policy for directory.

*Root Cause*

	When we will add edit log for new file, we will get the storage policy from {{INodeFile}}
by {{getStoragePolicyID()}}.
	{code}
	 AddOp op = AddOp.getInstance(cache.get())
	....................
	....................
        .setStoragePolicyId(newNode.getStoragePolicyID());
	{code}

	but this API will return the parent policy If file storagepolicy is UNSPECIFIED.

	{code}
        public byte getStoragePolicyID() {
            byte id = getLocalStoragePolicyID();
            if (id == BLOCK_STORAGE_POLICY_ID_UNSPECIFIED) {
               return this.getParent() != null ?this.getParent().getStoragePolicyID() : id;
            }
            return id;
        }
	{code}

	So in edits log create file will store parent storage policy.


*Fix*
	We should use {{INodeFile.getLocalStoragePolicyID()}}, it will give the current INodeFile
storage policy

> setStoragePolicy with folder behavior is different after cluster restart
> ------------------------------------------------------------------------
>
>                 Key: HDFS-8219
>                 URL: https://issues.apache.org/jira/browse/HDFS-8219
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Peter Shi
>            Assignee: Xiaoyu Yao
>            Priority: Minor
>         Attachments: HDFS-8219.patch, HDFS-8219.unittest-norepro.patch
>
>
> Reproduce steps.
> 1) mkdir named /temp
> 2) put one file A under /temp
> 3) change /temp storage policy to COLD
> 4) use -getStoragePolicy to query file A's storage policy, it is same with /temp
> 5) change /temp folder storage policy again, will see file A's storage policy keep same
with parent folder.
> then restart the cluster.
> do 3) 4) again, will find file A's storage policy is not change while parent folder's
storage policy changes. It behaves different.
> As i debugged, found the code:
> in INodeFile.getStoragePolicyID
> {code}
>   public byte getStoragePolicyID() {
>     byte id = getLocalStoragePolicyID();
>     if (id == BLOCK_STORAGE_POLICY_ID_UNSPECIFIED) {
>       return this.getParent() != null ?
>           this.getParent().getStoragePolicyID() : id;
>     }
>     return id;
>   }
> {code}
> If the file do not have its storage policy, it will use parent's. But after cluster restart,
the file turns to have its own storage policy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message