hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Azuryy Yu <azury...@gmail.com>
Subject Re: Problem with jobtracker hadoop 1.2
Date Fri, 18 Apr 2014 01:20:54 GMT
There is because your HDFS has no space left. please check your datanodes
are all started. also please check dfs.datanode.du.reserved in
hdfs-site.xml to make sure you don't reserve large capacity.


On Fri, Apr 18, 2014 at 7:42 AM, Shengjun Xin <sxin@gopivotal.com> wrote:

> Did you start datanode service?
>
>
> On Thu, Apr 17, 2014 at 9:23 PM, Karim Awara <karim.awara@kaust.edu.sa>wrote:
>
>> Hi,
>>
>> Whenever I start the hadoop on 24 machines, the following exception
>> happens on the jobtracker log file on the namenode:   I would appreciate
>> any help. Thank you.
>>
>>
>>
>> 2014-04-17 16:16:31,391 INFO org.apache.hadoop.mapred.JobTracker: Setting
>> safe mode to false. Requested by : karim
>> 2014-04-17 16:16:31,400 INFO org.apache.hadoop.util.NativeCodeLoader:
>> Loaded the native-hadoop library
>> 2014-04-17 16:16:31,429 INFO org.apache.hadoop.mapred.JobTracker:
>> Cleaning up the system directory
>> 2014-04-17 16:16:31,443 INFO org.apache.hadoop.mapred.JobHistory:
>> Creating DONE folder at
>> file:/home/karim/systems/hadoop-1.1.2/logs/history/done
>> 2014-04-17 16:16:31,445 INFO org.apache.hadoop.mapred.JobTracker: History
>> server being initialized in embedded mode
>> 2014-04-17 16:16:31,447 INFO org.apache.hadoop.mapred.JobHistoryServer:
>> Started job history server at: localhost:50030
>> 2014-04-17 16:16:31,448 INFO org.apache.hadoop.mapred.JobTracker: Job
>> History Server web address: localhost:50030
>> 2014-04-17 16:16:31,449 INFO
>> org.apache.hadoop.mapred.CompletedJobStatusStore: Completed job store is
>> inactive
>> 2014-04-17 16:16:31,474 WARN org.apache.hadoop.hdfs.DFSClient:
>> DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
>> java.io.IOException: File
>> /home/karim/data/hadoop-1.1.2_data/hadoop_tmp-karim/mapred/system/
>> jobtracker.info could only be replicated to 0 nodes, instead of 1
>>     at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1639)
>>     at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:736)
>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>     at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>     at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>     at java.lang.reflect.Method.invoke(Method.java:597)
>>     at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
>>     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
>>     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at javax.security.auth.Subject.doAs(Subject.java:396)
>>     at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
>>     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387)
>>
>>     at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>>     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
>>     at $Proxy7.addBlock(Unknown Source)
>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>     at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>     at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>     at java.lang.reflect.Method.invoke(Method.java:597)
>>     at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
>>     at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
>>     at $Proxy7.addBlock(Unknown Source)
>>     at
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
>>     at
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
>>     at
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
>>     at
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>>
>> 2014-04-17 16:16:31,474 WARN org.apache.hadoop.hdfs.DFSClient: Error
>> Recovery for block null bad datanode[0] nodes == null
>> 2014-04-17 16:16:31,474 WARN org.apache.hadoop.hdfs.DFSClient: Could not
>> get block locations. Source file
>> "/home/karim/data/hadoop-1.1.2_data/hadoop_tmp-karim/mapred/system/
>> jobtracker.info" - Aborting...
>> 2014-04-17 16:16:31,474 WARN org.apache.hadoop.mapred.JobTracker: Writing
>> to file
>> hdfs://cloud1:9100/home/karim/data/hadoop-1.1.2_data/hadoop_tmp-karim/mapred/system/
>> jobtracker.info failed!
>> 2014-04-17 16:16:31,474 WARN org.apache.hadoop.mapred.JobTracker:
>> FileSystem is not ready yet!
>> 2014-04-17 16:16:31,477 WARN org.apache.hadoop.mapred.JobTracker: Failed
>> to initialize recovery manager.
>> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>> /home/karim/data/hadoop-1.1.2_data/hadoop_tmp-karim/mapred/system/
>> jobtracker.info could only be replicated to 0 nodes, instead of 1
>>     at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1639)
>>     at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:736)
>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>     at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>     at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>     at java.lang.reflect.Method.invoke(Method.java:597)
>>     at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
>>     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
>>     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at javax.security.auth.Subject.doAs(Subject.java:396)
>>     at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
>>     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387)
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>
>
>
>
> --
> Regards
> Shengjun
>

Mime
View raw message