flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Rohrmann (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-5300) FileStateHandle#discard & FsCheckpointStateOutputStream#close tries to delete non-empty directory
Date Thu, 08 Dec 2016 17:28:58 GMT
Till Rohrmann created FLINK-5300:
------------------------------------

             Summary: FileStateHandle#discard & FsCheckpointStateOutputStream#close tries
to delete non-empty directory
                 Key: FLINK-5300
                 URL: https://issues.apache.org/jira/browse/FLINK-5300
             Project: Flink
          Issue Type: Improvement
          Components: State Backends, Checkpointing
    Affects Versions: 1.1.3, 1.2.0
            Reporter: Till Rohrmann
            Priority: Minor


Flink's behaviour to delete {{FileStateHandles}} and closing {{FsCheckpointStateOutputStream}}
always triggers a delete operation on the parent directory. Often this call will fail because
the directory still contains some other files.

A user reported that the SRE of their Hadoop cluster noticed this behaviour in the logs. It
might be more system friendly if we first checked whether the directory is empty or not. This
would prevent many error message to appear in the Hadoop logs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message