hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Wolf <brw...@gmail.com>
Subject Re: hadoop under cygwin issue
Date Mon, 01 Feb 2010 21:59:47 GMT

Aaron,

Thanks or your help. I  carefully went through the steps again a couple 
times , and ran

after this 

bin/hadoop namenode -format

(by the way, it asks if I want to reformat, I've tried it both ways)


then


bin/start-dfs.sh

and

bin/start-all.sh


and then 

bin/hadoop fs -put conf input

now the return for this seemed cryptic:

 
put: Target input/conf is a directory

(??)

  and when I tried

bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'

It says something about 0 nodes

(from log file)

2010-02-01 13:26:29,874 INFO 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: 
ugi=brian,None,Administrators,Users    ip=/127.0.0.1    cmd=create    
src=/cygwin/tmp/hadoop-SYSTEM/mapred/system/job_201002011323_0001/job.jar    
dst=null    perm=brian:supergroup:rw-r--r--
2010-02-01 13:26:30,045 INFO org.apache.hadoop.ipc.Server: IPC Server 
handler 3 on 9000, call 
addBlock(/cygwin/tmp/hadoop-SYSTEM/mapred/system/job_201002011323_0001/job.jar, 
DFSClient_725490811) from 127.0.0.1:3003: error: java.io.IOException: 
File 
/cygwin/tmp/hadoop-SYSTEM/mapred/system/job_201002011323_0001/job.jar 
could only be replicated to 0 nodes, instead of 1
java.io.IOException: File 
/cygwin/tmp/hadoop-SYSTEM/mapred/system/job_201002011323_0001/job.jar 
could only be replicated to 0 nodes, instead of 1
    at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1287)
    at 
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:351)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)


 

To maybe rule out something regarding ports or ssh , when I run netstat:

  TCP    127.0.0.1:9000         0.0.0.0:0              LISTENING
  TCP    127.0.0.1:9001         0.0.0.0:0              LISTENING


and when I browse to http://localhost:50070/


      Cluster Summary

* * * 21 files and directories, 0 blocks = 21 total. Heap Size is 8.01 
MB / 992.31 MB (0%)
*
Configured Capacity 	: 	0 KB
DFS Used 	: 	0 KB
Non DFS Used 	: 	0 KB
DFS Remaining 	: 	0 KB
DFS Used% 	: 	100 %
DFS Remaining% 	: 	0 %
Live Nodes <http://localhost:50070/dfshealth.jsp#LiveNodes> 	: 	0
Dead Nodes <http://localhost:50070/dfshealth.jsp#DeadNodes> 	: 	0


so I'm a bit still in the dark, I guess.

Thanks
Brian
 


Aaron Kimball wrote:
> Brian, it looks like you missed a step in the instructions. You'll need to
> format the hdfs filesystem instance before starting the NameNode server:
>
> You need to run:
>
> $ bin/hadoop namenode -format
>
> .. then you can do bin/start-dfs.sh
> Hope this helps,
> - Aaron
>
>
> On Sat, Jan 30, 2010 at 12:27 AM, Brian Wolf <brw314@gmail.com> wrote:
>
>   
>> Hi,
>>
>> I am trying to run Hadoop 0.19.2 under cygwin as per directions on the
>> hadoop "quickstart" web page.
>>
>> I know sshd is running and I can "ssh localhost" without a password.
>>
>> This is from my hadoop-site.xml
>>
>> <configuration>
>> <property>
>> <name>hadoop.tmp.dir</name>
>> <value>/cygwin/tmp/hadoop-${user.name}</value>
>> </property>
>> <property>
>> <name>fs.default.name</name>
>> <value>hdfs://localhost:9000</value>
>> </property>
>> <property>
>> <name>mapred.job.tracker</name>
>> <value>localhost:9001</value>
>> </property>
>> <property>
>> <name>mapred.job.reuse.jvm.num.tasks</name>
>> <value>-1</value>
>> </property>
>> <property>
>> <name>dfs.replication</name>
>> <value>1</value>
>> </property>
>> <property>
>> <name>dfs.permissions</name>
>> <value>false</value>
>> </property>
>> <property>
>> <name>webinterface.private.actions</name>
>> <value>true</value>
>> </property>
>> </configuration>
>>
>> These are errors from my log files:
>>
>>
>> 2010-01-30 00:03:33,091 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
>> Initializing RPC Metrics with hostName=NameNode, port=9000
>> 2010-01-30 00:03:33,121 INFO
>> org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at: localhost/
>> 127.0.0.1:9000
>> 2010-01-30 00:03:33,161 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
>> Initializing JVM Metrics with processName=NameNode, sessionId=null
>> 2010-01-30 00:03:33,181 INFO
>> org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics: Initializing
>> NameNodeMeterics using context
>> object:org.apache.hadoop.metrics.spi.NullContext
>> 2010-01-30 00:03:34,603 INFO
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>> fsOwner=brian,None,Administrators,Users
>> 2010-01-30 00:03:34,603 INFO
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup
>> 2010-01-30 00:03:34,603 INFO
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>> isPermissionEnabled=false
>> 2010-01-30 00:03:34,653 INFO
>> org.apache.hadoop.hdfs.server.namenode.metrics.FSNamesystemMetrics:
>> Initializing FSNamesystemMetrics using context
>> object:org.apache.hadoop.metrics.spi.NullContext
>> 2010-01-30 00:03:34,653 INFO
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered
>> FSNamesystemStatusMBean
>> 2010-01-30 00:03:34,803 INFO org.apache.hadoop.hdfs.server.common.Storage:
>> Storage directory C:\cygwin\tmp\hadoop-brian\dfs\name does not exist.
>> 2010-01-30 00:03:34,813 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> org.apache.hadoop.hdfs.server.common.InconsistentFSStateException:
>> Directory C:\cygwin\tmp\hadoop-brian\dfs\name is in an inconsistent state:
>> storage directory does not exist or is not accessible.
>>   at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:278)
>>   at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
>>   at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:309)
>>   at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:288)
>>   at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:163)
>>   at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:208)
>>   at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:194)
>>   at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:859)
>>   at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:868)
>> 2010-01-30 00:03:34,823 INFO org.apache.hadoop.ipc.Server: Stopping server
>> on 9000
>>
>>
>>
>>
>>
>> =========================================================
>>
>> 2010-01-29 15:13:30,270 INFO org.apache.hadoop.ipc.Client: Retrying connect
>> to server: localhost/127.0.0.1:9000. Already tried 9 time(s).
>> problem cleaning system directory: null
>> java.net.ConnectException: Call to localhost/127.0.0.1:9000 failed on
>> connection exception: java.net.ConnectException: Connection refused: no
>> further information
>>   at org.apache.hadoop.ipc.Client.wrapException(Client.java:724)
>>   at org.apache.hadoop.ipc.Client.call(Client.java:700)
>>   at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
>>   at $Proxy4.getProtocolVersion(Unknown Source)
>>   at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:348)
>>   at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:104)
>>
>>
>>
>> Thanks
>> Brian
>>
>>
>>     
>
>   


Mime
View raw message