hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5709) Improve upgrade with existing files and directories named ".snapshot"
Date Tue, 31 Dec 2013 23:24:50 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13859775#comment-13859775
] 

Andrew Wang commented on HDFS-5709:
-----------------------------------

Right now, I get these stack traces when trying to upgrade to trunk, with a "mkdir /.snapshot"
edit, and then in the fsimage:

{noformat}
pergroup:rwxr-xr-x, opCode=OP_MKDIR, txid=2]
java.lang.ArrayIndexOutOfBoundsException: -1
    at org.apache.hadoop.hdfs.server.namenode.FSDirectory.addChild(FSDirectory.java:2193)
        at org.apache.hadoop.hdfs.server.namenode.FSDirectory.unprotectedMkdir(FSDirectory.java:1982)
        at org.apache.hadoop.hdfs.server.namenode.FSDirectory.unprotectedMkdir(FSDirectory.java:1967)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:458)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:206)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:119)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:730)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:644)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:329)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:248)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:845)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:614)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:444)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:492)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:648)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:633)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1239)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1305)
{noformat}

{noformat}
13/12/31 15:12:32 ERROR namenode.FSImage: Failed to load image from FSImageFile(file=/tmp/hadoop-andrew/dfs/name/current/fsimage_0000000000000000008,
cpktTxId=0000000000000000008)
    java.io.FileNotFoundException: Directory does not exist: /.snapshot
        at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.valueOf(INodeDirectory.java:55)
        at org.apache.hadoop.hdfs.server.namenode.FSImageFormat$Loader.loadDirectory(FSImageFormat.java:520)
        at org.apache.hadoop.hdfs.server.namenode.FSImageFormat$Loader.loadLocalNameINodes(FSImageFormat.java:424)
        at org.apache.hadoop.hdfs.server.namenode.FSImageFormat$Loader.load(FSImageFormat.java:340)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:824)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:813)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:661)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:630)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:329)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:248)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:845)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:614)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:444)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:492)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:648)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:633)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1239)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1305)
{noformat}

> Improve upgrade with existing files and directories named ".snapshot"
> ---------------------------------------------------------------------
>
>                 Key: HDFS-5709
>                 URL: https://issues.apache.org/jira/browse/HDFS-5709
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 3.0.0, 2.2.0
>            Reporter: Andrew Wang
>              Labels: snapshots, upgrade
>
> Right now in trunk, upgrade fails messily if the old fsimage or edits refer to a directory
named ".snapshot". We should at least print a better error message (which I believe was the
original intention in HDFS-4666), and [~atm] proposed automatically renaming these files and
directories.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message