hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ChiaHung Lin (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-431) MapReduce NG integration
Date Tue, 04 Oct 2011 04:45:34 GMT

    [ https://issues.apache.org/jira/browse/HAMA-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119899#comment-13119899
] 

ChiaHung Lin commented on HAMA-431:
-----------------------------------

Yes, checkpoint at the moment provides saving data to hdfs per host. The primary reason having
a separated checkpointing process is to ensure bsp task would continuously process even in
the presence of checkpointing service failure. Although we can combine chececkpointing process
with bsp task together, chances are if the checkpointing process fails this may propergate
to bsp task resulting in the collapse of bsp task. I think that Joe Armstrong's paper[1] explains
this well. 

[1]. Making reliable distributed systems in the presence of software errors. http://www.sics.se/~joe/thesis/armstrong_thesis_2003.pdf
                
> MapReduce NG integration
> ------------------------
>
>                 Key: HAMA-431
>                 URL: https://issues.apache.org/jira/browse/HAMA-431
>             Project: Hama
>          Issue Type: New Feature
>            Reporter: Thomas Jungblut
>            Assignee: Thomas Jungblut
>         Attachments: job_state.dot, task_phase.dot, task_state.dot
>
>
> We should take a look at how to integrate Hama's BSP Engine to Hadoop's nextGen application
platform.
> Can be currently found in the 0.23 branch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message