accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kesten Broughton <kbrough...@21ct.com>
Subject Re: /accumulo dir remains after reformatting name node and deleting datanodes
Date Sun, 26 Jan 2014 23:10:18 GMT
Apparently, there’s something i really don’t understand about hdfs data
files.
I assumed that without the name node, hdfs would cease to function, but
hadoop fs -ls / still shows stuff when i remove the name node.

[root@txoig-hdfs01:~]$ rm -Rf /hadoop/hdfs/namenode
[root@txoig-hdfs01:~]$ su hdfs
[hdfs@txoig-hdfs01:root]$ hadoop fs -ls /
Found 5 items
drwxr-xr-x   - hdfs   hdfs          0 2014-01-26 13:36 /accumulo
drwxr-xr-x   - hdfs   hdfs          0 2013-12-19 18:17 /apps
drwx------   - mapred hdfs          0 2014-01-26 16:39 /mapred
drwxrwxrwx   - hdfs   hdfs          0 2014-01-26 14:56 /tmp
drwxr-xr-x   - hdfs   hdfs          0 2013-12-30 15:11 /user
[hdfs@txoig-hdfs01:root]$ ls /hadoop/hdfs/
total 8.0K
drwxr-xr-x. 2 root 4.0K Jan 26 17:02 .
drwxr-xr-x. 5 root 4.0K Dec 19 18:13 ..
    
    
[root@txoig-hdfs02:/]$ rm -Rf /hadoop/hdfs/namesecondary
[root@txoig-hdfs02:/]$ ls /hadoop/hdfs/
total 8.0K
drwxr-xr-x. 2 root 4.0K Jan 26 17:04 .
drwxr-xr-x. 6 root 4.0K Dec 19 18:14 ..
[root@txoig-hdfs02:/]$ hadoop fs -ls /
Found 5 items
drwxr-xr-x   - hdfs   hdfs          0 2014-01-26 13:36 /accumulo
drwxr-xr-x   - hdfs   hdfs          0 2013-12-19 18:17 /apps
drwx------   - mapred hdfs          0 2014-01-26 16:39 /mapred
drwxrwxrwx   - hdfs   hdfs          0 2014-01-26 14:56 /tmp
drwxr-xr-x   - hdfs   hdfs          0 2013-12-30 15:11 /user
    
[root@txoig-hdfs03:/]$ ls /hadoop/hdfs/
total 8.0K
drwxr-xr-x. 2 root 4.0K Jan 26 16:23 .
drwxr-xr-x. 6 root 4.0K Dec 19 18:14 ..

cat /etc/hadoop/conf.empty/hdfs-site.xml contains

    
    <property>
    <name>dfs.name.dir</name>
    <value>/hadoop/hdfs/namenode</value>
  </property>


[root@txoig-hdfs03:/]$ ls /etc/hadoop
total 12K
drwxr-xr-x.  3 root 4.0K Dec 19 18:10 .
drwxr-xr-x. 98 root 4.0K Jan 26 16:04 ..
lrwxrwxrwx.  1 hdfs   29 Dec 19 18:10 conf -> /etc/alternatives/hadoop-conf
drwxr-xr-x.  2 root 4.0K Jan 26 16:55 conf.empty




On 1/26/14, 4:37 PM, "Kesten Broughton" <kbroughton@21ct.com> wrote:

>It¹s possible something was running.  I only did an accumulo/stop-all.sh
>and sometimes i¹ve noticed i have to go to each node to stop-here.sh.
>
>[root@txoig-hdfs02:accumulo-1.5.0]$ ls /hadoop/hdfs
>drwxr-xr-x. 5 hdfs 4.0K Jan 26 16:10 namesecondary
>
>
>[root@txoig-hdfs01:~]$ ls /hadoop/hdfs/
>drwxr-xr-x. 5 hdfs 4.0K Jan 26 16:10 namenode
>
>
>[root@txoig-hdfs03:~]$ ls /hadoop/hdfs
>drwxr-xr-x. 4 root 4.0K Jan 26 16:13 namenode
># this is weird. Wasn¹t expecting a second name node.  We aren¹t doing HA.
>
>
>But shutting down all accumulo and hdfs and then doing the same procedure,
>i still see
>
>[root@txoig-hdfs01:~]$ hadoop fs -ls /
>Found 5 items
>drwxr-xr-x   - hdfs   hdfs          0 2014-01-26 13:36 /accumulo
>drwxr-xr-x   - hdfs   hdfs          0 2013-12-19 18:17 /apps
>drwx------   - mapred hdfs          0 2014-01-26 14:32 /mapred
>drwxrwxrwx   - hdfs   hdfs          0 2014-01-26 14:56 /tmp
>drwxr-xr-x   - hdfs   hdfs          0 2013-12-30 15:11 /user
>
>
>
>On 1/26/14, 4:16 PM, "Christopher" <ctubbsii@apache.org> wrote:
>
>>Was any part of Hadoop or Accumulo still running when you did this?
>>Are you sure you deleted the right HDFS data directory referenced in
>>your Hadoop configuration?
>>
>>--
>>Christopher L Tubbs II
>>http://gravatar.com/ctubbsii
>>
>>
>>On Sun, Jan 26, 2014 at 4:45 PM, Kesten Broughton <kbroughton@21ct.com>
>>wrote:
>>>>
>>>>We did experiments using the root dir for both name nodes and
>>>>datanodes.
>>>>Now we want to wipe out the old hdfs cluster, keeping the name node at
>>>>/hadoop/hdfs/namenode but moving the datanode fs to /dev/sdb etc.
>>>
>>> After removing /hadoop/hdfs/data on each of the datanode machines and
>>>doing
>>> hadoop namenode -format
>>>
>>> I still see /accumulo on
>>> hadoop fs -ls /
>>>
>>> I was expecting it to be gone.
>>>
>>> kesten
>>>
>

Mime
View raw message