hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-1936) Updating the layout version from HDFS-1822 causes upgrade problems.
Date Mon, 23 May 2011 03:14:47 GMT

     [ https://issues.apache.org/jira/browse/HDFS-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Todd Lipcon updated HDFS-1936:
------------------------------

    Attachment: hadoop-22-dfs-dir.tgz
                hdfs-1936-with-testcase.txt

Here's a tarball of an 0.22 data directory and test case which tests upgrading from it.

Without Suresh's patch (which is included in this one) it failed every time. Now it passes
sometimes and other times it times out. I can see the following NPE in the logs:

    [junit] 2011-05-22 20:09:43,246 WARN  datanode.DataNode (DataNode.java:run(1213)) - Unexpected
exception 
    [junit] java.lang.NullPointerException
    [junit]     at org.apache.hadoop.hdfs.server.datanode.DataNode.getUpgradeManagerDatanode(DataNode.java:1728)
    [junit]     at org.apache.hadoop.hdfs.server.datanode.DataStorage.verifyDistributedUpgradeProgress(DataStorage.java:727)
    [junit] Exception in thread "DataNode: [file:/home/todd/git/hadoop-hdfs/build/test/data/dfs/data/data1/,file:/home/todd/git/hadoop-hdfs/build/test/data/dfs/data/data2/]"
java.lang.NullPointerException
    [junit]     at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:394)
    [junit]     at org.apache.hadoop.hdfs.server.datanode.DataNode$BPOfferService.cleanUp(DataNode.java:1002)
    [junit]     at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:189)
    [junit]     at org.apache.hadoop.hdfs.server.datanode.DataNode$BPOfferService.run(DataNode.java:1217)
    [junit]     at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:216)
    [junit]     at java.lang.Thread.run(Thread.java:662)
    [junit]     at org.apache.hadoop.hdfs.server.datanode.DataNode$BPOfferService.setupBPStorage(DataNode.java:796)
    [junit]     at org.apache.hadoop.hdfs.server.datanode.DataNode$BPOfferService.setupBP(DataNode.java:773)


> Updating the layout version from HDFS-1822 causes upgrade problems.
> -------------------------------------------------------------------
>
>                 Key: HDFS-1936
>                 URL: https://issues.apache.org/jira/browse/HDFS-1936
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.22.0, 0.23.0
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>            Priority: Blocker
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: HDFS-1936.22.patch, HDFS-1936.trunk.patch, hadoop-22-dfs-dir.tgz,
hdfs-1936-with-testcase.txt
>
>
> In HDFS-1822 and HDFS-1842, the layout versions for 203, 204, 22 and trunk were changed.
Some of the namenode logic that depends on layout version is broken because of this. Read
the comment for more description.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message