hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Billy Watson <williamrwat...@gmail.com>
Subject Re: can't submit remote job
Date Mon, 18 May 2015 12:58:14 GMT
Netflix Genie is what we use for submitting jobs.

William Watson
Software Engineer
(904) 705-7056 PCS

On Mon, May 18, 2015 at 7:07 AM, xeonmailinglist-gmail <
xeonmailinglist@gmail.com> wrote:

>  I also can't find a good site/book that explains well how to submit
> remote jobs. Also, can anyone know where can I get more useful info?
>
>
>
> -------- Forwarded Message --------  Subject: can't submit remote job  Date:
> Mon, 18 May 2015 11:54:56 +0100  From: xeonmailinglist-gmail
> <xeonmailinglist@gmail.com> <xeonmailinglist@gmail.com>  To:
> user@hadoop.apache.org <user@hadoop.apache.org> <user@hadoop.apache.org>
>
>  Hi,
>
> I am trying to submit a remote job in Yarn MapReduce, but I can’t because
> I get the error [1]. I don’t have more exceptions in the other logs.
>
> My Mapreduce runtime have 1 *ResourceManager* and 3 *NodeManagers*, and
> the HDFS is running properly (all nodes are alive).
>
> I have looked to all logs, and I still don’t understand why I get this
> error. Any help to fix this? Is it a problem of the remote job that I am
> submitting?
>
> [1]
>
> $ less logs/hadoop-ubuntu-namenode-ip-172-31-17-45.log
>
> 2015-05-18 10:42:16,570 DEBUG org.apache.hadoop.hdfs.StateChange: *BLOCK* NameNode.addBlock:
file /tmp/hadoop-yarn/staging/xeon/.staging/job_1431945660897_0001/job.split
> fileId=16394 for DFSClient_NONMAPREDUCE_-1923902075_14
> 2015-05-18 10:42:16,570 DEBUG org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.getAdditionalBlock:
/tmp/hadoop-yarn/staging/xeon/.staging/job_1431945660897_0001/job.
> split inodeId 16394 for DFSClient_NONMAPREDUCE_-1923902075_14
> 2015-05-18 10:42:16,571 DEBUG org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy:
Failed to choose remote rack (location = ~/default-rack), fallback to lo
> cal rack
> org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy$NotEnoughReplicasException:
>         at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseRandom(BlockPlacementPolicyDefault.java:691)
>         at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseRemoteRack(BlockPlacementPolicyDefault.java:580)
>         at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:348)
>         at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:214)
>         at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:111)
>         at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:126)
>         at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1545)
>         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3200)
>         at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:641)
>
> ​
>
> --
> --
>
>
>
>

Mime
View raw message