flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nikolay Vasilishin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4492) Cleanup files from canceled snapshots
Date Mon, 17 Oct 2016 10:23:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581831#comment-15581831
] 

Nikolay Vasilishin commented on FLINK-4492:
-------------------------------------------

[~srichter] Does this issue still persist? 
I've runned some tests to reproduce this bug, but the checkpoint directory was empty after
them. 
If I'm not mistaken, 
{noformat}
/** 
* If the stream is only closed, we remove the produced file (cleanup through the auto close
 * feature, for example). This method throws no exception if the deletion fails, but only
 * logs the error.
 */
@Override
public void close() {
	if (!closed) {
		closed = true;
		if (outStream != null) {
			try {
				outStream.close();
				fs.delete(statePath, false);

				// attempt to delete the parent (will fail and be ignored if the parent has more files)
				try {
					fs.delete(basePath, false);
				} catch (IOException ignored) {}
			}
			catch (Exception e) {
				LOG.warn("Cannot delete closed and discarded state stream for " + statePath, e);
			}
		}
	}
}
{noformat}
this method (at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory) properly
deletes all checkpoint files. 

> Cleanup files from canceled snapshots
> -------------------------------------
>
>                 Key: FLINK-4492
>                 URL: https://issues.apache.org/jira/browse/FLINK-4492
>             Project: Flink
>          Issue Type: Bug
>            Reporter: Stefan Richter
>            Assignee: Nikolay Vasilishin
>            Priority: Minor
>
> Current checkpointing only closes CheckpointStateOutputStreams on cancel, but incomplete
files are not properly deleted from the filesystem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message