hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6233) Datanode upgrade in Windows from 1.x to 2.4 fails with symlink error.
Date Fri, 11 Apr 2014 05:55:15 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13966263#comment-13966263
] 

Chris Nauroth commented on HDFS-6233:
-------------------------------------

Good point.  :-)  Thanks again.

> Datanode upgrade in Windows from 1.x to 2.4 fails with symlink error.
> ---------------------------------------------------------------------
>
>                 Key: HDFS-6233
>                 URL: https://issues.apache.org/jira/browse/HDFS-6233
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode, tools
>    Affects Versions: 2.4.0
>         Environment: Windows
>            Reporter: Huan Huang
>            Assignee: Arpit Agarwal
>         Attachments: HDFS-6233.01.patch
>
>
> I try to upgrade Hadoop from 1.x and 2.4, but DataNode failed to start due to hard link
exception.
> Repro steps:
> *Installed Hadoop 1.x
> *hadoop dfsadmin -safemode enter
> *hadoop dfsadmin -saveNamespace
> *hadoop namenode -finalize
> *Stop all services
> *Uninstall Hadoop 1.x 
> *Install Hadoop 2.4 
> *Start namenode with -upgrade option
> *Try to start datanode, begin to see Hardlink exception in datanode service log.
> {code}
> 2014-04-10 22:47:11,655 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8010:
starting
> 2014-04-10 22:47:11,656 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
> 2014-04-10 22:47:11,999 INFO org.apache.hadoop.hdfs.server.common.Storage: Data-node
version: -55 and name-node layout version: -56
> 2014-04-10 22:47:12,008 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on d:\hadoop\data\hdfs\dn\in_use.lock
acquired by nodename 7268@myhost
> 2014-04-10 22:47:12,011 INFO org.apache.hadoop.hdfs.server.common.Storage: Recovering
storage directory D:\hadoop\data\hdfs\dn from previous upgrade
> 2014-04-10 22:47:12,017 INFO org.apache.hadoop.hdfs.server.common.Storage: Upgrading
storage directory d:\hadoop\data\hdfs\dn.
>    old LV = -44; old CTime = 0.
>    new LV = -55; new CTime = 1397168400373
> 2014-04-10 22:47:12,021 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting
block pool BP-39008719-10.0.0.1-1397168400092 directory d:\hadoop\data\hdfs\dn\current\BP-39008719-10.0.0.1-1397168400092\current
> 2014-04-10 22:47:12,254 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization
failed for block pool Block pool <registering> (Datanode Uuid unassigned) service to
myhost/10.0.0.1:8020
> java.io.IOException: Usage: hardlink create [LINKNAME] [FILENAME] |Incorrect command
line arguments.
> 	at org.apache.hadoop.fs.HardLink.createHardLinkMult(HardLink.java:479)
> 	at org.apache.hadoop.fs.HardLink.createHardLinkMult(HardLink.java:416)
> 	at org.apache.hadoop.hdfs.server.datanode.DataStorage.linkBlocks(DataStorage.java:816)
> 	at org.apache.hadoop.hdfs.server.datanode.DataStorage.linkAllBlocks(DataStorage.java:759)
> 	at org.apache.hadoop.hdfs.server.datanode.DataStorage.doUpgrade(DataStorage.java:566)
> 	at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:486)
> 	at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:225)
> 	at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:249)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:929)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:900)
> 	at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:274)
> 	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:220)
> 	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:815)
> 	at java.lang.Thread.run(Thread.java:722)
> 2014-04-10 22:47:12,258 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending
block pool service for: Block pool <registering> (Datanode Uuid unassigned) service
to myhost/10.0.0.1:8020
> 2014-04-10 22:47:12,359 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool
ID needed, but service not yet registered with NN
> java.lang.Exception: trace
> 	at org.apache.hadoop.hdfs.server.datanode.BPOfferService.getBlockPoolId(BPOfferService.java:143)
> 	at org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.remove(BlockPoolManager.java:91)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.shutdownBlockPool(DataNode.java:859)
> 	at org.apache.hadoop.hdfs.server.datanode.BPOfferService.shutdownActor(BPOfferService.java:350)
> 	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.cleanUp(BPServiceActor.java:619)
> 	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:837)
> 	at java.lang.Thread.run(Thread.java:722)
> 2014-04-10 22:47:12,359 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed
Block pool <registering> (Datanode Uuid unassigned)
> 2014-04-10 22:47:12,360 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool
ID needed, but service not yet registered with NN
> java.lang.Exception: trace
> 	at org.apache.hadoop.hdfs.server.datanode.BPOfferService.getBlockPoolId(BPOfferService.java:143)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.shutdownBlockPool(DataNode.java:861)
> 	at org.apache.hadoop.hdfs.server.datanode.BPOfferService.shutdownActor(BPOfferService.java:350)
> 	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.cleanUp(BPServiceActor.java:619)
> 	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:837)
> 	at java.lang.Thread.run(Thread.java:722)
> 2014-04-10 22:47:14,360 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting
Datanode
> 2014-04-10 22:47:14,361 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0
> 2014-04-10 22:47:14,362 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:

> /************************************************************
> SHUTDOWN_MSG: Shutting down DataNode at myhost/10.0.0.1
> ************************************************************/
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message