hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (Resolved) (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-92) if hadoop.tmp.dir is under your dfs.data.dir, HDFS will silently wipe out your "name" directory
Date Thu, 29 Dec 2011 13:23:31 GMT

     [ https://issues.apache.org/jira/browse/HDFS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Harsh J resolved HDFS-92.
-------------------------

    Resolution: Not A Problem

This goes against all recommendations in configuring the directories. I don't see why one
would configure it this way that it'd lead to an obvious issue. Same is with merging mapred.local.dir
and dfs.datanode.data.dir. Resolving as not a problem.
                
> if hadoop.tmp.dir is under your dfs.data.dir, HDFS will silently wipe out your "name"
directory
> -----------------------------------------------------------------------------------------------
>
>                 Key: HDFS-92
>                 URL: https://issues.apache.org/jira/browse/HDFS-92
>             Project: Hadoop HDFS
>          Issue Type: Bug
>         Environment: gentoo linux on Intel/Dell w/ Sun JDK
>            Reporter: Brian Karlak
>
> I used a hadoop-site.xml conf file like:
>   <property>
>     <name>dfs.data.dir</name>
>     <value>/data01/hadoop</value>
>     <description>Dirs to store data on.</description>
>   </property>
>   <property>
>     <name>hadoop.tmp.dir</name>
>     <value>/data01/hadoop/tmp</value>
>     <description>A base for other temporary directories.</description>
>   </property>
> This file will format the namenode properly.  Upon startup with the bin/start-dfs.sh
script, however, the /data01/hadoop/tmp/dfs/name directory is silently wiped out.  This foobars
the namenode, but only after the next DFS stop/start cycle.  (see output below)
> This is obviously a configuration error first and foremost, but the fact that hadoop
silently corrupts itself makes it tricky to track down.
> [hid191]$ bin/hadoop namenode -format
> 08/04/04 18:41:43 INFO dfs.NameNode: STARTUP_MSG: 
> /************************************************************
> STARTUP_MSG: Starting NameNode
> STARTUP_MSG:   host = hid191.dev01.corp.metaweb.com/127.0.0.1
> STARTUP_MSG:   args = [-format]
> STARTUP_MSG:   version = 0.16.2
> STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.16
-r 642481; compiled by 'hadoopqa' on Sat Mar 29 01:59:04 UTC 2008
> ************************************************************/
> 08/04/04 18:41:43 INFO fs.FSNamesystem: fsOwner=zenkat,users
> 08/04/04 18:41:43 INFO fs.FSNamesystem: supergroup=supergroup
> 08/04/04 18:41:43 INFO fs.FSNamesystem: isPermissionEnabled=true
> 08/04/04 18:41:43 INFO dfs.Storage: Storage directory /data01/hadoop/tmp/dfs/name has
been successfully formatted.
> 08/04/04 18:41:43 INFO dfs.NameNode: SHUTDOWN_MSG: 
> /************************************************************
> SHUTDOWN_MSG: Shutting down NameNode at hid191.dev01.corp.metaweb.com/127.0.0.1
> ************************************************************/
> [hid191]$ ls /data01/hadoop/tmp/dfs/name
> current  image
> [hid191]$ bin/start-dfs.sh 
> starting namenode, logging to /data01/hadoop/logs/hadoop-zenkat-namenode-hid191.out
> localhost: starting datanode, logging to /data01/hadoop/logs/hadoop-zenkat-datanode-hid191.out
> localhost: starting secondarynamenode, logging to /data01/hadoop/logs/hadoop-zenkat-secondarynamenode-hid191.out
> [hid191]$ ls /data01/hadoop/tmp/dfs/name
> ls: cannot access /data01/hadoop/tmp/dfs/name: No such file or directory

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message