hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-626) If number of bsp tasks are not specified, the bsp job cannot assume to get all the tasks in the cluster
Date Mon, 29 Oct 2012 03:55:12 GMT

    [ https://issues.apache.org/jira/browse/HAMA-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485819#comment-13485819
] 

Edward J. Yoon commented on HAMA-626:
-------------------------------------

{code}
  <property>
    <name>bsp.max.tasks.per.job</name>
    <value></value>
    <description>The maximum number of BSP tasks per job. 
    By default, This limit is switched off.
    </description>
  </property>
{code}

I've added new configuration property 'bsp.max.tasks.per.job' to specify the maximum of tasks
per job and changed BSPJobClient like this:

{code}
    int maxTasks = 0;
    int limitTasks = job.getConf().getInt(Constants.MAX_TASKS_PER_JOB, 0);
    
    ClusterStatus clusterStatus = getClusterStatus(true);
    
    if(limitTasks > 0) {
      maxTasks = limitTasks;
    } else {
      maxTasks = clusterStatus.getMaxTasks() - clusterStatus.getTasks();
    }
    
    if (maxTasks < job.getNumBspTask()) {
      throw new IOException("Job failed! The number of tasks has exceeded the maximum allowed.");
    }
{code}


                
> If number of bsp tasks are not specified, the bsp job cannot assume to get all the tasks
in the cluster
> -------------------------------------------------------------------------------------------------------
>
>                 Key: HAMA-626
>                 URL: https://issues.apache.org/jira/browse/HAMA-626
>             Project: Hama
>          Issue Type: Bug
>          Components: graph
>    Affects Versions: 0.6.0
>            Reporter: Suraj Menon
>            Assignee: Edward J. Yoon
>              Labels: newbie
>             Fix For: 0.6.0
>
>
> In most graph examples, if we don't provide a value for the number of BSP Tasks to be
run, it is assumed that the whole input for the job could be split into all the available
slots in the cluster. We can introduce a new property for a default maximum that could be
assumed in such cases than taking all the slots in the cluster. This property could be a bsp
property than something focused on graphs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message