kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Damian Guy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4540) Suspended tasks that are not assigned to the StreamThread need to be closed before new active and standby tasks are created
Date Wed, 14 Dec 2016 21:32:58 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749549#comment-15749549
] 

Damian Guy commented on KAFKA-4540:
-----------------------------------

This issue isn't resolved. The problem here is that say i previously had standby task 1_0,
then onPartitionsRevoked it gets suspended. Then onPartitionsAssigned it becomes an active
task. When we try to create the Active Task for 1_0 we still have a Standby Task "open" for
the same task and we get the exception above.

> Suspended tasks that are not assigned to the StreamThread need to be closed before new
active and standby tasks are created
> ---------------------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-4540
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4540
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 0.10.2.0
>            Reporter: Damian Guy
>            Assignee: Damian Guy
>             Fix For: 0.10.2.0
>
>
> When partition assignment happens we first try and add the active tasks and then add
the standby tasks. The problem with this is that a new active task might already be an existing
suspended standby task. if this is the case then when the active task initialises it will
throw an exception from RocksDB:
> {{Caused by: org.rocksdb.RocksDBException: IO error: lock /tmp/kafka-streams-7071/kafka-music-charts/1_1/rocksdb/all-songs/LOCK:
No locks available}}
> We need to make sure we have removed an closed any no-longer assigned Suspended tasks
before creating new tasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message