hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <praveen.pe...@nokia.com>
Subject RE: Hadoop on Rackspace cloud
Date Thu, 06 Jan 2011 21:07:15 GMT
I found the solution. The problems was with ports. Looks like cloud doesn't have these ports
open. For now I shutdown iptables on all hadoop machines and things worked magically.

Thanks
Praveen

________________________________
From: ext praveen.peddi@nokia.com [mailto:praveen.peddi@nokia.com]
Sent: Thursday, January 06, 2011 3:40 PM
To: mapreduce-user@hadoop.apache.org
Subject: Hadoop on Rackspace cloud

Hello all,
I am trying to run Hadoop on Rackspace and I am having issues with starting up servers. I
have configrued everything on cloud exactly same as my local hadoop (which is working) but
I can't start servers. HDFS fails to start. Does anyone had any luck on installing and starging
Hadoop on Rackspace cloud (or any cloud other than Amazon EC2?).

Here is log output from namenode-hadoop-master.log on master machine.

2011-01-06 20:37:35,723 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=hadoop,hadoop
      ip=/xx.xx.xx.xx cmd=delete      src=/home/hadoop/hadoopdfs-hadoop/mapred/system/jobtracker.info
dst=null        perm=null
2011-01-06 20:37:45,728 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
transactions: 746 Total time for transactions(ms): 10Number of transactions batched in Syncs:
0 Number of syncs: 558 SyncTimes(ms): 572
2011-01-06 20:37:45,729 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=hadoop,hadoop
      ip=/xx.xx.xx.xx cmd=create      src=/home/hadoop/hadoopdfs-hadoop/mapred/system/jobtracker.info
dst=null        perm=hadoop:supergroup:rw-r--r--
2011-01-06 20:37:45,731 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=hadoop,hadoop
      ip=/xx.xx.xx.xx cmd=setPermission       src=/home/hadoop/hadoopdfs-hadoop/mapred/system/jobtracker.info
dst=null        perm=hadoop:supergroup:rw-------
2011-01-06 20:37:45,732 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 54310,
call addBlock(/home/hadoop/hadoopdfs-hadoop/mapred/system/jobtracker.info, DFSClient_-905664857)
from xx.xx.xx.xx:47377: error: java.io.IOException: File /home/hadoop/hadoopdfs-hadoop/mapred/system/jobtracker.info
could only be replicated to 0 nodes, instead of 1
java.io.IOException: File /home/hadoop/hadoopdfs-hadoop/mapred/system/jobtracker.info could
only be replicated to 0 nodes, instead of 1
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1271)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
        at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
2011-01-06 20:37:45,734 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=hadoop,hadoop
      ip=/xx.xx.xx.xx cmd=delete      src=/home/hadoop/hadoopdfs-hadoop/mapred/system/jobtracker.info
dst=null        perm=null
2011-01-06 20:37:55,740 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=hadoop,hadoop
      ip=/xx.xx.xx.xx cmd=create      src=/home/hadoop/hadoopdfs-hadoop/mapred/system/jobtracker.info
dst=null        perm=hadoop:supergroup:rw-r--r--
2011-01-06 20:37:55,741 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=hadoop,hadoop
      ip=/xx.xx.xx.xx cmd=setPermission       src=/home/hadoop/hadoopdfs-hadoop/mapred/system/jobtracker.info
dst=null        perm=hadoop:supergroup:rw-------
2011-01-06 20:37:55,742 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 54310,
call addBlock(/home/hadoop/hadoopdfs-hadoop/mapred/system/jobtracker.info, DFSClient_-905664857)
from xx.xx.xx.xx:47377: error: java.io.IOException: File /home/hadoop/hadoopdfs-hadoop/mapred/system/jobtracker.info
could only be replicated to 0 nodes, instead of 1
java.io.IOException: File /home/hadoop/hadoopdfs-hadoop/mapred/system/jobtracker.info could
only be replicated to 0 nodes, instead of 1
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1271)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
        at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
2011-01-06 20:37:55,745 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=hadoop,hadoop
      ip=/xx.xx.xx.xx cmd=delete      src=/home/hadoop/hadoopdfs-hadoop/mapred/system/jobtracker.info
dst=null        perm=null


Mime
View raw message