incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suraj Menon (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-445) Make configurable checkpointing
Date Fri, 20 Jan 2012 12:30:39 GMT

    [ https://issues.apache.org/jira/browse/HAMA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189760#comment-13189760
] 

Suraj Menon commented on HAMA-445:
----------------------------------

Does checkpoint interval here imply the number of supersteps before we initiate a checkpoint
process? Should this be done within barrier synchronization period or should we have a Checkpointer
daemon as we have for backing up namenode in Hadoop. With the second option, we might loose
the determinism in finding/assuming at an instant of time (how many supersteps) or (the last
superstep) that have been completely checkpointed. The first approach might make it slower
but would have better determinism in checkpoint recovery. Please let me know if I have the
correct understanding.
                
> Make configurable checkpointing
> -------------------------------
>
>                 Key: HAMA-445
>                 URL: https://issues.apache.org/jira/browse/HAMA-445
>             Project: Hama
>          Issue Type: Improvement
>          Components: bsp
>    Affects Versions: 0.4.0
>            Reporter: Edward J. Yoon
>              Labels: newbie
>             Fix For: 0.5.0
>
>
> We should extend the BSPJob to let the user set the checkpoint intervals.
> bq.  job.setCheckpointInterval(5);
> This method should put the parameter into the configuration of the job with a meaningful
key, e.G. "bsp.checkpoint.interval".
> In the BSPPeerImpl we should check if this interval has been reached and make the checkpointing
accordingly.
> Checkpointing gets called in BSPPeerImpl#sync(), there is already a condition which checks
if checkpointing is enabled.
> Plus points:
> If you can provide an additional method in BSPJob that let's the user enable or disable
checkpointing. Hint: Configuration key is: "bsp.checkpoint.enabled".

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message