flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5778) Split FileStateHandle into fileName and basePath
Date Wed, 08 Mar 2017 13:31:38 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15901247#comment-15901247

ASF GitHub Bot commented on FLINK-5778:

Github user StephanEwen commented on the issue:

    Looks good in general.
    One thing that I stumbled across a lot in recent works on the checkpoints / savepoints
is that they all implement `equals` and `hashCode` and delegate to the task states and handles,
    Do we need to define semantic equality there? It seems fragile to me, because the state
handles by themselves can in general not really make a good claim about equality. The `FileStateHandle`
for example fails to define equality depending on whether there is a trailing '/' on a directory
path or not.

> Split FileStateHandle into fileName and basePath
> ------------------------------------------------
>                 Key: FLINK-5778
>                 URL: https://issues.apache.org/jira/browse/FLINK-5778
>             Project: Flink
>          Issue Type: Sub-task
>          Components: State Backends, Checkpointing
>            Reporter: Ufuk Celebi
>            Assignee: Ufuk Celebi
> Store the statePath as a basePath and a fileName and allow to overwrite the basePath.
We cannot overwrite the base path as long as the state handle is still in flight and not persisted.
Otherwise we risk a resource leak.
> We need this in order to be able to relocate savepoints.
> {code}
> interface RelativeBaseLocationStreamStateHandle {
>    void clearBaseLocation();
>    void setBaseLocation(String baseLocation);
> }
> {code}
> FileStateHandle should implement this and the SavepointSerializer should forward the
calls when a savepoint is stored or loaded, clear before store and set after load.

This message was sent by Atlassian JIRA

View raw message