hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ChiaHung Lin (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-511) Submitting heterogenous supersteps with precedence constraints on Hama
Date Fri, 10 Feb 2012 16:51:00 GMT

    [ https://issues.apache.org/jira/browse/HAMA-511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205540#comment-13205540
] 

ChiaHung Lin commented on HAMA-511:
-----------------------------------

Some issues I can think of at the moment

For selective synchronization, if recovery is required to take place, configurable checkpoint
might not work well. For example, user configures checkpoint every 5 supersteps, and some
tasks cross over 5 supersteps (e.g. 7 supersteps). Then checkpoints can not correctly serialize
messages because the long task is not yet ready for be checkpointed. Or the system may checkpoint
the process image, which is not portable. The disadvantage would be that, in the case of configurable
checkpoint set to 5 and a task cross 7 supersteps, the system can only checkpoint once per
35 supersteps.

Concerning task execution/ dependency, it might be interesting to have a look at CIEL[1],
which allows dynamic task composition and is suitable for iterative and recursive algorithm.
Also it might be good if the system allows users to choose this as an option because probably
other tasks may not need such style job execution.    

[1]. CIEL is a universal execution engine for distributed computation. http://www.cl.cam.ac.uk/research/srg/netos/ciel/
                
> Submitting heterogenous supersteps with precedence constraints on Hama
> ----------------------------------------------------------------------
>
>                 Key: HAMA-511
>                 URL: https://issues.apache.org/jira/browse/HAMA-511
>             Project: Hama
>          Issue Type: New Feature
>            Reporter: Suraj Menon
>            Priority: Minor
>         Attachments: Defining supersteps for BSP.pdf
>
>
> Hama should support submission of jobs with support for:
> 1) Skipping unwanted superstep synchronization.
> 2) Run supersteps with heterogenous nature of computation
> 3) Scheduling supersteps with precedence constraints.
> An explanation of these is provided in the attachment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message