incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (HAMA-411) Support checkpoint based on HDFS
Date Wed, 06 Jul 2011 05:46:16 GMT

    [ https://issues.apache.org/jira/browse/HAMA-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13060302#comment-13060302
] 

Thomas Jungblut edited comment on HAMA-411 at 7/6/11 5:45 AM:
--------------------------------------------------------------

My first idea would be to use an abstract method in BSP (Or we can think of adding this to
our new BSPPeer.):

{noformat}
abstract boolean checkpoint(long superStep);
{noformat}

Or we are providing a default implementation which always returns true, but the user can override
this with his own logic.

So the user is up to handle in which superstep he want the checkpointing / checkpoint(X) =
true. 
This method gets called before a superstep starts.

If true we are going to save all the messages in the queues to disk. 

Additionally we should think of a method in BSP class which is helping the user to save his
own computation- for example the tentative pagerank map in PageRank Example. Or the user has
to take care of it himself when returning true in the method.


      was (Author: thomas.jungblut):
    My first idea would be to use an abstract method in BSP (Or we can think of adding this
to our new BSPPeer.):

{noformat}
abstract boolean checkpoint(long superStep);
{noformat}

So the user is up to handle in which superstep he want the checkpointing / checkpoint(X) =
true. 
This method gets called before a superstep starts.

If true we are going to save all the messages in the queues to disk. 

Additionally we should think of a method in BSP class which is helping the user to save his
own computation- for example the tentative pagerank map in PageRank Example. Or the user has
to take care of it himself when returning true in the method.

  
> Support checkpoint based on HDFS
> --------------------------------
>
>                 Key: HAMA-411
>                 URL: https://issues.apache.org/jira/browse/HAMA-411
>             Project: Hama
>          Issue Type: New Feature
>          Components: bsp
>            Reporter: Thomas Jungblut
>
> We need to add checkpointing to Hama to deal with fault in future. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message