hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Glen Mazza <gma...@talend.com>
Subject Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Date Thu, 17 Jan 2013 15:01:54 GMT
On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
> Hello Glen,
>
>      Pl find my comments embedded below :
>
> 1.) The Standalone Operation 
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), 
> just to confirm, can run without any DFS filesystem?  (We're not being 
> asked to run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation 
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed), 
> where we are instructed to run "bin/hadoop namenode -format", 
> question: how do I undo whatever that command does?  I.e., how do I 
> delete that distributed file system created as well as remove from 
> Hadoop any knowledge of its prior existence -- do I have to manually 
> delete files with OS commands (what do I remove?) or is there some 
> type of "bin/hadoop namenode -delete" command that undoes the 
> "-format" command?
> >>You can reformat the NameNode or manually delete the directories holding 
> the data and the metadata.

Thanks, Tariq, looking at the output of the namenode -format command it 
looks like the only folder I have to delete is /tmp/hadoop-gmazza:

13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4, 
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4, 
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO common.Storage: Storage directory 
/tmp/hadoop-gmazza/dfs/name has been successfully formatted.

Question: Why does Hadoop place the dfs folder under /tmp?  It's subject 
that way to getting erased each time I reboot my computer. Looking at 
the instructions for namenode -format: 
http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it 
doesn't appear we have an option to specify where we would like the dfs 
folder to be created.

Thanks,
Glen



>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com <http://cloudfront.blogspot.com>
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <gmazza@talend.com 
> <mailto:gmazza@talend.com>> wrote:
>
>     Hi, I'm following the Hadoop 1.1.1 tutorial
>     (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and
>     I have these questions:
>
>     1.) The Standalone Operation
>     (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
>     just to confirm, can run without any DFS filesystem?  (We're not
>     being asked to run "bin/hadoop namenode -format" for it.)
>
>     2.) For Pseudo-Distributed Operation
>     (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>     where we are instructed to run "bin/hadoop namenode -format",
>     question: how do I undo whatever that command does?  I.e., how do
>     I delete that distributed file system created as well as remove
>     from Hadoop any knowledge of its prior existence -- do I have to
>     manually delete files with OS commands (what do I remove?) or is
>     there some type of "bin/hadoop namenode -delete" command that
>     undoes the "-format" command?
>
>     Thanks,
>     Glen
>
>     -- 
>     Glen Mazza
>     Talend Community Coders - coders.talend.com <http://coders.talend.com>
>     blog: www.jroller.com/gmazza <http://www.jroller.com/gmazza>
>
>


-- 
Glen Mazza
Talend Community Coders - coders.talend.com
blog: www.jroller.com/gmazza


Mime
View raw message