kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6647) KafkaStreams.cleanUp creates .lock file in directory its trying to clean
Date Wed, 14 Mar 2018 20:37:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399362#comment-16399362

Guozhang Wang commented on KAFKA-6647:

Just another note, for locking purposes, when the apps shutdown cleanly, it is OK to have
the lock file left in the state directory since threads are excluding each other not via the
ownership of the file but via locking the file handle. So for your case, if you indeed want
to clean up the whole directory upon shutting down, then I think there is still a valid point
to close all the file channels upon shutting down. For which we can consider:

1) either use StandardOpenOption.DELETE_ON_CLOSE as you did in the PR.
2) or add a new function in the state directory class that closes all the managed file channels
upon KafkaStreams.close(); which may be safer than 1) since StandardOpenOption.DELETE_ON_CLOSE
is best-effort.

> KafkaStreams.cleanUp creates .lock file in directory its trying to clean
> ------------------------------------------------------------------------
>                 Key: KAFKA-6647
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6647
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 1.0.1
>         Environment: windows 10.
> java version "1.8.0_162"
> Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
> Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)
> org.apache.kafka:kafka-streams:1.0.1
> Kafka commitId : c0518aa65f25317e
>            Reporter: George Bloggs
>            Priority: Minor
>              Labels: streams
> When calling kafkaStreams.cleanUp() before starting a stream the StateDirectory.cleanRemovedTasks()
method contains this check:
> {code:java}
> ... Line 240
>                   if (lock(id, 0)) {
>                         long now = time.milliseconds();
>                         long lastModifiedMs = taskDir.lastModified();
>                         if (now > lastModifiedMs + cleanupDelayMs) {
>                             log.info("{} Deleting obsolete state directory {} for task
{} as {}ms has elapsed (cleanup delay is {}ms)", logPrefix(), dirName, id, now - lastModifiedMs,
>                             Utils.delete(taskDir);
>                         }
>                     }
> {code}
> The check for lock(id,0) will create a .lock file in the directory that subsequently
is going to be deleted. If the .lock file already exists from a previous run the attempt to
delete the .lock file fails with AccessDeniedException.
> This leaves the .lock file in the taskDir. Calling Utils.delete(taskDir) will then attempt
to remove the taskDir path calling Files.delete(path).
> The call to files.delete(path) in postVisitDirectory will then fail java.nio.file.DirectoryNotEmptyException
as the failed attempt to delete the .lock file left the directory not empty. (o.a.k.s.p.internals.StateDirectory
      : stream-thread [restartedMain] Failed to lock the state directory due to an unexpected
> This seems to then cause issues using streams from a topic to an inMemory store.

This message was sent by Atlassian JIRA

View raw message