flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5300) FileStateHandle#discard & FsCheckpointStateOutputStream#close tries to delete non-empty directory
Date Fri, 09 Dec 2016 15:36:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15735612#comment-15735612
] 

ASF GitHub Bot commented on FLINK-5300:
---------------------------------------

Github user tillrohrmann commented on the issue:

    https://github.com/apache/flink/pull/2971
  
    Thanks for the review @uce. I think we can include this also in a possible 1.1.5 later.
Who knows which other issues will still come up.


> FileStateHandle#discard & FsCheckpointStateOutputStream#close tries to delete non-empty
directory
> -------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-5300
>                 URL: https://issues.apache.org/jira/browse/FLINK-5300
>             Project: Flink
>          Issue Type: Improvement
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.2.0, 1.1.3
>            Reporter: Till Rohrmann
>            Assignee: Till Rohrmann
>            Priority: Critical
>
> Flink's behaviour to delete {{FileStateHandles}} and closing {{FsCheckpointStateOutputStream}}
always triggers a delete operation on the parent directory. Often this call will fail because
the directory still contains some other files.
> A user reported that the SRE of their Hadoop cluster noticed this behaviour in the logs.
It might be more system friendly if we first checked whether the directory is empty or not.
This would prevent many error message to appear in the Hadoop logs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message