giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rafal Wojdyla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-819) Number of containers required for a job
Date Fri, 03 Jan 2014 21:00:50 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13861899#comment-13861899
] 

Rafal Wojdyla commented on GIRAPH-819:
--------------------------------------

The actual problem is that we get the number of *running* containers not cluster capacity.

There are two solution available:
* temporary - remove container capacity check
* final (?) - get cluster capacity (total memory available) and check whether it's sufficient
for giraph application

> Number of containers required for a job
> ---------------------------------------
>
>                 Key: GIRAPH-819
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-819
>             Project: Giraph
>          Issue Type: Bug
>          Components: lib, mapreduce
>    Affects Versions: 1.1.0
>            Reporter: Rafal Wojdyla
>              Labels: patch
>             Fix For: 1.1.0
>
>         Attachments: GIRAPH-819.patch
>
>
> Java 1.6.x
> Giraph trunk - revert java 1.7 support.
> Hadoop 2.2.0.x
> Job submission fails due to:
> {noformat}
> 13/11/28 12:02:14 INFO yarn.GiraphYarnClient: Running Client
> 13/11/28 12:02:14 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.1.100:8045
> 13/11/28 12:02:15 INFO yarn.GiraphYarnClient: Got node report from ASM for, nodeId=kreator:46477,
nodeAddresskreator:8042, nodeRackName/default-rack, nodeNumContainers7
> 13/11/28 12:02:15 INFO yarn.GiraphYarnClient: Got node report from ASM for, nodeId=exotica:46645,
nodeAddressexotica:8042, nodeRackName/default-rack, nodeNumContainers8
> Exception in thread "main" java.lang.RuntimeException: Giraph job requires 2 containers
to run; cluster only hosts 15
> 	at org.apache.giraph.yarn.GiraphYarnClient.checkPerNodeResourcesAvailable(GiraphYarnClient.java:230)
> 	at org.apache.giraph.yarn.GiraphYarnClient.run(GiraphYarnClient.java:125)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message