hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11695) [SPS]: Namenode failed to start while loading SPS xAttrs from the edits log.
Date Wed, 03 May 2017 22:22:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15995800#comment-15995800

Uma Maheswara Rao G commented on HDFS-11695:

Well spotted [~surendrasingh]
I want you to fix few more related issues here if you can:
# .
      XAttr satisfyXAttr = unprotectedSatisfyStoragePolicy(iip, bm, fsd);
    } finally {
    fsd.getEditLog().logSetXAttrs(src, xAttrs, logRetryCache);
Here I see we are logging outside fed lock. We should do inside fsd lock.
# . Also there is no logSync. We should have a logSync here. LogSync can be outside fsdLock
but inside fsnamesystem lock.
# . Move the code related to SatisfyStoragePolicyXattrs to FSDirSatisfyStoragePolicyOp. For
your reference, look at FSDirErasureCodingOp.
# . I think we should have one protected removeSPSXattr API which should do log sync as well.
When replaying from editlogs/fsimage, may be its ok to use unprotected. But when calling from
SPS#postBlockStorageMovementCleanup, we should do logSync as well right.
# . Also as you said, we should not throw exception out may be valid case as I explained one
scenario in my previous comment. DO you want to handle that as well? To do that, we should
just make API as idempotent instead of AtMostOnce.
# . Please move the test case to TestPersistentStoragePolicySatisfier

Feel free to ask me if you need any help.

> [SPS]: Namenode failed to start while loading SPS xAttrs from the edits log.
> ----------------------------------------------------------------------------
>                 Key: HDFS-11695
>                 URL: https://issues.apache.org/jira/browse/HDFS-11695
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: HDFS-10285
>            Reporter: Surendra Singh Lilhore
>            Assignee: Surendra Singh Lilhore
>            Priority: Blocker
>         Attachments: fsimage.xml, HDFS-11695-HDFS-10285.001.patch
> {noformat}
> 2017-04-23 13:27:51,971 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed
to start namenode.
> java.io.IOException: Cannot request to call satisfy storage policy on path /ssl, as this
file/dir was already called for satisfying storage policy.
>         at org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.unprotectedSatisfyStoragePolicy(FSDirAttrOp.java:511)
>         at org.apache.hadoop.hdfs.server.namenode.FSDirXAttrOp.unprotectedSetXAttrs(FSDirXAttrOp.java:284)
>         at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:918)
>         at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:241)
>         at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:150)
> {noformat}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message