accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <busbey+li...@cloudera.com>
Subject Re: /accumulo dir remains after reformatting name node and deleting datanodes
Date Sun, 26 Jan 2014 23:43:26 GMT
HDFS will definitely not function without the NameNode. I'm not sure how
the NameNode process will respond if you delete its storage directory out
from under it (that is, while it is running).

What's the output of `alternatives --display hadoop-conf`?

Is txoig-hdfs01 the name of your namenode server?

How are you shutting down HDFS before deleting things?


On Sun, Jan 26, 2014 at 5:10 PM, Kesten Broughton <kbroughton@21ct.com>wrote:

> Apparently, there’s something i really don’t understand about hdfs data
> files.
> I assumed that without the name node, hdfs would cease to function, but
> hadoop fs -ls / still shows stuff when i remove the name node.
>
> [root@txoig-hdfs01:~]$ rm -Rf /hadoop/hdfs/namenode
> [root@txoig-hdfs01:~]$ su hdfs
> [hdfs@txoig-hdfs01:root]$ hadoop fs -ls /
> Found 5 items
> drwxr-xr-x   - hdfs   hdfs          0 2014-01-26 13:36 /accumulo
> drwxr-xr-x   - hdfs   hdfs          0 2013-12-19 18:17 /apps
> drwx------   - mapred hdfs          0 2014-01-26 16:39 /mapred
> drwxrwxrwx   - hdfs   hdfs          0 2014-01-26 14:56 /tmp
> drwxr-xr-x   - hdfs   hdfs          0 2013-12-30 15:11 /user
> [hdfs@txoig-hdfs01:root]$ ls /hadoop/hdfs/
> total 8.0K
> drwxr-xr-x. 2 root 4.0K Jan 26 17:02 .
> drwxr-xr-x. 5 root 4.0K Dec 19 18:13 ..
>
>
> [root@txoig-hdfs02:/]$ rm -Rf /hadoop/hdfs/namesecondary
> [root@txoig-hdfs02:/]$ ls /hadoop/hdfs/
> total 8.0K
> drwxr-xr-x. 2 root 4.0K Jan 26 17:04 .
> drwxr-xr-x. 6 root 4.0K Dec 19 18:14 ..
> [root@txoig-hdfs02:/]$ hadoop fs -ls /
> Found 5 items
> drwxr-xr-x   - hdfs   hdfs          0 2014-01-26 13:36 /accumulo
> drwxr-xr-x   - hdfs   hdfs          0 2013-12-19 18:17 /apps
> drwx------   - mapred hdfs          0 2014-01-26 16:39 /mapred
> drwxrwxrwx   - hdfs   hdfs          0 2014-01-26 14:56 /tmp
> drwxr-xr-x   - hdfs   hdfs          0 2013-12-30 15:11 /user
>
> [root@txoig-hdfs03:/]$ ls /hadoop/hdfs/
> total 8.0K
> drwxr-xr-x. 2 root 4.0K Jan 26 16:23 .
> drwxr-xr-x. 6 root 4.0K Dec 19 18:14 ..
>
> cat /etc/hadoop/conf.empty/hdfs-site.xml contains
>
>
>     <property>
>     <name>dfs.name.dir</name>
>     <value>/hadoop/hdfs/namenode</value>
>   </property>
>
>
> [root@txoig-hdfs03:/]$ ls /etc/hadoop
> total 12K
> drwxr-xr-x.  3 root 4.0K Dec 19 18:10 .
> drwxr-xr-x. 98 root 4.0K Jan 26 16:04 ..
> lrwxrwxrwx.  1 hdfs   29 Dec 19 18:10 conf -> /etc/alternatives/hadoop-conf
> drwxr-xr-x.  2 root 4.0K Jan 26 16:55 conf.empty
>
>
>
>
> On 1/26/14, 4:37 PM, "Kesten Broughton" <kbroughton@21ct.com> wrote:
>
> >It¹s possible something was running.  I only did an accumulo/stop-all.sh
> >and sometimes i¹ve noticed i have to go to each node to stop-here.sh.
> >
> >[root@txoig-hdfs02:accumulo-1.5.0]$ ls /hadoop/hdfs
> >drwxr-xr-x. 5 hdfs 4.0K Jan 26 16:10 namesecondary
> >
> >
> >[root@txoig-hdfs01:~]$ ls /hadoop/hdfs/
> >drwxr-xr-x. 5 hdfs 4.0K Jan 26 16:10 namenode
> >
> >
> >[root@txoig-hdfs03:~]$ ls /hadoop/hdfs
> >drwxr-xr-x. 4 root 4.0K Jan 26 16:13 namenode
> ># this is weird. Wasn¹t expecting a second name node.  We aren¹t doing HA.
> >
> >
> >But shutting down all accumulo and hdfs and then doing the same procedure,
> >i still see
> >
> >[root@txoig-hdfs01:~]$ hadoop fs -ls /
> >Found 5 items
> >drwxr-xr-x   - hdfs   hdfs          0 2014-01-26 13:36 /accumulo
> >drwxr-xr-x   - hdfs   hdfs          0 2013-12-19 18:17 /apps
> >drwx------   - mapred hdfs          0 2014-01-26 14:32 /mapred
> >drwxrwxrwx   - hdfs   hdfs          0 2014-01-26 14:56 /tmp
> >drwxr-xr-x   - hdfs   hdfs          0 2013-12-30 15:11 /user
> >
> >
> >
> >On 1/26/14, 4:16 PM, "Christopher" <ctubbsii@apache.org> wrote:
> >
> >>Was any part of Hadoop or Accumulo still running when you did this?
> >>Are you sure you deleted the right HDFS data directory referenced in
> >>your Hadoop configuration?
> >>
> >>--
> >>Christopher L Tubbs II
> >>http://gravatar.com/ctubbsii
> >>
> >>
> >>On Sun, Jan 26, 2014 at 4:45 PM, Kesten Broughton <kbroughton@21ct.com>
> >>wrote:
> >>>>
> >>>>We did experiments using the root dir for both name nodes and
> >>>>datanodes.
> >>>>Now we want to wipe out the old hdfs cluster, keeping the name node at
> >>>>/hadoop/hdfs/namenode but moving the datanode fs to /dev/sdb etc.
> >>>
> >>> After removing /hadoop/hdfs/data on each of the datanode machines and
> >>>doing
> >>> hadoop namenode -format
> >>>
> >>> I still see /accumulo on
> >>> hadoop fs -ls /
> >>>
> >>> I was expecting it to be gone.
> >>>
> >>> kesten
> >>>
> >
>
>

Mime
View raw message