hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Kozlov <ale...@cloudera.com>
Subject Re: hadoop under cygwin issue
Date Tue, 02 Feb 2010 04:19:25 GMT
Live Nodes <http://localhost:50070/dfshealth.jsp#LiveNodes>     :       0

You datanode is dead.  Look at the logs in the $HADOOP_HOME/logs directory
(or where your logs are) and check the errors.

Alex K

On Mon, Feb 1, 2010 at 1:59 PM, Brian Wolf <brw314@gmail.com> wrote:

>
> Aaron,
>
> Thanks or your help. I  carefully went through the steps again a couple
> times , and ran
>
> after this
> bin/hadoop namenode -format
>
> (by the way, it asks if I want to reformat, I've tried it both ways)
>
>
> then
>
>
> bin/start-dfs.sh
>
> and
>
> bin/start-all.sh
>
>
> and then
> bin/hadoop fs -put conf input
>
> now the return for this seemed cryptic:
>
>
> put: Target input/conf is a directory
>
> (??)
>
>  and when I tried
>
> bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'
>
> It says something about 0 nodes
>
> (from log file)
>
> 2010-02-01 13:26:29,874 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit:
> ugi=brian,None,Administrators,Users    ip=/127.0.0.1    cmd=create
>  src=/cygwin/tmp/hadoop-SYSTEM/mapred/system/job_201002011323_0001/job.jar
>  dst=null    perm=brian:supergroup:rw-r--r--
> 2010-02-01 13:26:30,045 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 3 on 9000, call
> addBlock(/cygwin/tmp/hadoop-SYSTEM/mapred/system/job_201002011323_0001/job.jar,
> DFSClient_725490811) from 127.0.0.1:3003: error: java.io.IOException: File
> /cygwin/tmp/hadoop-SYSTEM/mapred/system/job_201002011323_0001/job.jar could
> only be replicated to 0 nodes, instead of 1
> java.io.IOException: File
> /cygwin/tmp/hadoop-SYSTEM/mapred/system/job_201002011323_0001/job.jar could
> only be replicated to 0 nodes, instead of 1
>   at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1287)
>   at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:351)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
>
>
>
> To maybe rule out something regarding ports or ssh , when I run netstat:
>
>  TCP    127.0.0.1:9000         0.0.0.0:0              LISTENING
>  TCP    127.0.0.1:9001         0.0.0.0:0              LISTENING
>
>
> and when I browse to http://localhost:50070/
>
>
>     Cluster Summary
>
> * * * 21 files and directories, 0 blocks = 21 total. Heap Size is 8.01 MB /
> 992.31 MB (0%)
> *
> Configured Capacity     :       0 KB
> DFS Used        :       0 KB
> Non DFS Used    :       0 KB
> DFS Remaining   :       0 KB
> DFS Used%       :       100 %
> DFS Remaining%  :       0 %
> Live Nodes <http://localhost:50070/dfshealth.jsp#LiveNodes>     :       0
> Dead Nodes <http://localhost:50070/dfshealth.jsp#DeadNodes>     :       0
>
>
> so I'm a bit still in the dark, I guess.
>
> Thanks
> Brian
>
>
>
>
> Aaron Kimball wrote:
>
>> Brian, it looks like you missed a step in the instructions. You'll need to
>> format the hdfs filesystem instance before starting the NameNode server:
>>
>> You need to run:
>>
>> $ bin/hadoop namenode -format
>>
>> .. then you can do bin/start-dfs.sh
>> Hope this helps,
>> - Aaron
>>
>>
>> On Sat, Jan 30, 2010 at 12:27 AM, Brian Wolf <brw314@gmail.com> wrote:
>>
>>
>>
>>> Hi,
>>>
>>> I am trying to run Hadoop 0.19.2 under cygwin as per directions on the
>>> hadoop "quickstart" web page.
>>>
>>> I know sshd is running and I can "ssh localhost" without a password.
>>>
>>> This is from my hadoop-site.xml
>>>
>>> <configuration>
>>> <property>
>>> <name>hadoop.tmp.dir</name>
>>> <value>/cygwin/tmp/hadoop-${user.name}</value>
>>> </property>
>>> <property>
>>> <name>fs.default.name</name>
>>> <value>hdfs://localhost:9000</value>
>>> </property>
>>> <property>
>>> <name>mapred.job.tracker</name>
>>> <value>localhost:9001</value>
>>> </property>
>>> <property>
>>> <name>mapred.job.reuse.jvm.num.tasks</name>
>>> <value>-1</value>
>>> </property>
>>> <property>
>>> <name>dfs.replication</name>
>>> <value>1</value>
>>> </property>
>>> <property>
>>> <name>dfs.permissions</name>
>>> <value>false</value>
>>> </property>
>>> <property>
>>> <name>webinterface.private.actions</name>
>>> <value>true</value>
>>> </property>
>>> </configuration>
>>>
>>> These are errors from my log files:
>>>
>>>
>>> 2010-01-30 00:03:33,091 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
>>> Initializing RPC Metrics with hostName=NameNode, port=9000
>>> 2010-01-30 00:03:33,121 INFO
>>> org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at:
>>> localhost/
>>> 127.0.0.1:9000
>>> 2010-01-30 00:03:33,161 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
>>> Initializing JVM Metrics with processName=NameNode, sessionId=null
>>> 2010-01-30 00:03:33,181 INFO
>>> org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics:
>>> Initializing
>>> NameNodeMeterics using context
>>> object:org.apache.hadoop.metrics.spi.NullContext
>>> 2010-01-30 00:03:34,603 INFO
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>>> fsOwner=brian,None,Administrators,Users
>>> 2010-01-30 00:03:34,603 INFO
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>>> supergroup=supergroup
>>> 2010-01-30 00:03:34,603 INFO
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>>> isPermissionEnabled=false
>>> 2010-01-30 00:03:34,653 INFO
>>> org.apache.hadoop.hdfs.server.namenode.metrics.FSNamesystemMetrics:
>>> Initializing FSNamesystemMetrics using context
>>> object:org.apache.hadoop.metrics.spi.NullContext
>>> 2010-01-30 00:03:34,653 INFO
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered
>>> FSNamesystemStatusMBean
>>> 2010-01-30 00:03:34,803 INFO
>>> org.apache.hadoop.hdfs.server.common.Storage:
>>> Storage directory C:\cygwin\tmp\hadoop-brian\dfs\name does not exist.
>>> 2010-01-30 00:03:34,813 ERROR
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>>> initialization failed.
>>> org.apache.hadoop.hdfs.server.common.InconsistentFSStateException:
>>> Directory C:\cygwin\tmp\hadoop-brian\dfs\name is in an inconsistent
>>> state:
>>> storage directory does not exist or is not accessible.
>>>  at
>>>
>>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:278)
>>>  at
>>>
>>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
>>>  at
>>>
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:309)
>>>  at
>>>
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:288)
>>>  at
>>>
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:163)
>>>  at
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:208)
>>>  at
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:194)
>>>  at
>>>
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:859)
>>>  at
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:868)
>>> 2010-01-30 00:03:34,823 INFO org.apache.hadoop.ipc.Server: Stopping
>>> server
>>> on 9000
>>>
>>>
>>>
>>>
>>>
>>> =========================================================
>>>
>>> 2010-01-29 15:13:30,270 INFO org.apache.hadoop.ipc.Client: Retrying
>>> connect
>>> to server: localhost/127.0.0.1:9000. Already tried 9 time(s).
>>> problem cleaning system directory: null
>>> java.net.ConnectException: Call to localhost/127.0.0.1:9000 failed on
>>> connection exception: java.net.ConnectException: Connection refused: no
>>> further information
>>>  at org.apache.hadoop.ipc.Client.wrapException(Client.java:724)
>>>  at org.apache.hadoop.ipc.Client.call(Client.java:700)
>>>  at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
>>>  at $Proxy4.getProtocolVersion(Unknown Source)
>>>  at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:348)
>>>  at
>>> org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:104)
>>>
>>>
>>>
>>> Thanks
>>> Brian
>>>
>>>
>>>
>>>
>>
>>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message