incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthieu Morel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (S4-137) Add HDFS StateStorage for more reliable checkpoint saving
Date Mon, 02 Sep 2013 13:49:52 GMT

    [ https://issues.apache.org/jira/browse/S4-137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13756069#comment-13756069
] 

Matthieu Morel commented on S4-137:
-----------------------------------

Good idea especially if you have a use case for that. You might also want to consider the
induced latency though, depending on the checkpoint size and frequency. Make sure to use the
hsync api for persistency guarantees.
                
> Add HDFS StateStorage for more reliable checkpoint saving
> ---------------------------------------------------------
>
>                 Key: S4-137
>                 URL: https://issues.apache.org/jira/browse/S4-137
>             Project: Apache S4
>          Issue Type: New Feature
>    Affects Versions: 0.6
>            Reporter: Kurtt.Lin
>            Priority: Minor
>              Labels: checkpoint
>             Fix For: 0.7
>
>   Original Estimate: 120h
>  Remaining Estimate: 120h
>
> Currently DefaultFileSystemStateStorage is the only implementation for StateStorage.
Should have a storage like HDFS to improve the reliability of the checkpoints' states. Some
apps would benefit from this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message