kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "George Bloggs (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-6647) KafkaStreams.cleanUp creates .lock file in directory its trying to clean
Date Tue, 13 Mar 2018 17:11:01 GMT
George Bloggs created KAFKA-6647:

             Summary: KafkaStreams.cleanUp creates .lock file in directory its trying to clean
                 Key: KAFKA-6647
                 URL: https://issues.apache.org/jira/browse/KAFKA-6647
             Project: Kafka
          Issue Type: Bug
          Components: streams
    Affects Versions: 1.0.1
         Environment: windows 10.
java version "1.8.0_162"
Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)
            Reporter: George Bloggs

When calling kafkaStreams.cleanUp() before starting a stream the StateDirectory.cleanRemovedTasks()
method contains this check:
... Line 240
                  if (lock(id, 0)) {
                        long now = time.milliseconds();
                        long lastModifiedMs = taskDir.lastModified();
                        if (now > lastModifiedMs + cleanupDelayMs) {
                            log.info("{} Deleting obsolete state directory {} for task {}
as {}ms has elapsed (cleanup delay is {}ms)", logPrefix(), dirName, id, now - lastModifiedMs,
The check for lock(id,0) will create a .lock file in the directory that subsequently is going
to be deleted. If the .lock file already exists from a previous run the attempt to delete
the .lock file fails with AccessDeniedException.

This leaves the .lock file in the taskDir. Calling Utils.delete(taskDir) will then attempt
to remove the taskDir path calling Files.delete(path).

The call to files.delete(path) in postVisitDirectory will then fail java.nio.file.DirectoryNotEmptyException
as the failed attempt to delete the .lock file left the directory not empty.

This seems to then cause issues using streams from a topic to an in memory store.

This message was sent by Atlassian JIRA

View raw message