hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kihwal Lee <kih...@yahoo-inc.com.INVALID>
Subject Re: Questions about HDFS federation
Date Tue, 03 Mar 2015 14:19:01 GMT
I suggest testing without federation first. That is, running two separate yarn/hdfs instances.
Once that works properly, you can introduce federation in your config.

Kihwal
      From: xeonmailinglist <xeonmailinglist@gmail.com>
 To: hdfs-dev@hadoop.apache.org 
 Sent: Tuesday, March 3, 2015 4:32 AM
 Subject: Questions about HDFS federation
   
   Hi,
  1. The image below shows 2 YARN  instances using the same federation of clouds. Is this
possible?
  2. I have configured in 2 hosts (hadooop-coc-1, and hadoop-coc-2) a federation of HDFS.
In the configuration, I have set a namespace in each host, and a single data node (see image).
The service is running properly. You can check the output of the jps commands in [1].  The
strange part is that, when I list the files, they do not appear in hadoop-coc-2. You can check
the output in [2]. Why this happens? 
 
  [1]: jps output xubuntu@hadoop-coc-1:~/Programs/hadoop$ jps
21538 NameNode
21773 DataNode

xubuntu@hadoop-coc-2:~/Programs/hadoop$ jps
2342 NameNode
 [2]: hdfs dfs -ls / output xubuntu@hadoop-coc-1:~/Programs/hadoop$ hdfs dfs -ls /
Java HotSpot(TM) Client VM warning: You have loaded library /home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so
which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link
it with '-z noexecstack'.
15/03/03 05:09:04 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
Found 1 items
drwxr-xr-x   - xubuntu supergroup          0 2015-03-03 04:47 /input1

xubuntu@hadoop-coc-2:~/Programs/hadoop$ hdfs dfs -ls /
Java HotSpot(TM) Client VM warning: You have loaded library /home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so
which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link
it with '-z noexecstack'.
15/03/03 05:09:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
 ​
 -- 
--   

  
Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message