hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinayakumar B (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11291) Avoid unnecessary edit log for setStoragePolicy() and setReplication()
Date Fri, 20 Jan 2017 10:59:26 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831543#comment-15831543

Vinayakumar B commented on HDFS-11291:

bq. I think this change is not required in unprotectedSetStoragePolicy(). setStoragePolicy()
will just return void.
yes, you are right. it doesnt return anything.

One more change required is, in both cases, where there is no edit txn added, no need to wait
for logSync(). 
So I feel, if want to skip logSync() and still return success response to client, some special
refactor required in return types of {{FSDirAttrOp#setStoragePolicy(..)}} and  {{FSDirAttrOp#setReplication(..)}}
to indicate change not done due to same value and skip logSync() (and audit also, if not required)

Also I have doubt whether its correct to log audit when there is no change done and no edit
txn added.

May be [~andrew.wang]/[~arpitagarwal] can give their view on this.

> Avoid unnecessary edit log for setStoragePolicy() and setReplication()
> ----------------------------------------------------------------------
>                 Key: HDFS-11291
>                 URL: https://issues.apache.org/jira/browse/HDFS-11291
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Surendra Singh Lilhore
>            Assignee: Surendra Singh Lilhore
>         Attachments: HDFS-11291.001.patch, HDFS-11291.002.patch
> We are setting the storage policy for file without checking the current policy of file
for avoiding extra getStoragePolicy() rpc call. Currently namenode is not checking the current
storage policy before setting new one and adding edit logs. I think if the old and new storage
policy is same we can avoid set operation.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message