kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-7181) Kafka Streams State stuck in rebalancing after one of the StreamThread encounters IllegalStateException
Date Sat, 05 Jan 2019 05:32:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-7181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Guozhang Wang resolved KAFKA-7181.
----------------------------------
       Resolution: Fixed
    Fix Version/s: 2.2.0

> Kafka Streams State stuck in rebalancing after one of the StreamThread encounters IllegalStateException
> -------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-7181
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7181
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 1.1.0
>            Reporter: Romil Kumar Vasani
>            Priority: Major
>             Fix For: 2.2.0
>
>
> One the StreamThread encounters an IllegalStateException and is marked DEAD, shut down.
> The application doesn't spawn a new thread in it's place, the partitions of that thread
are assigned to a different thread and it synchronizes. But the application is stuck in REBALANCING
state, as not all StreamThreads are in RUNNING state.
> Excepted: New thread should come up and after synchronization/rebalancing it the KafkaStream.State
should be RUNNING
> Since all the active threads (that are not marked DEAD) are in RUNNING state, the KafkaStreams.State
should be RUNNING
> P.S. I am reporting an issue for the first time. If there is more information needed
I can provide.
> Below are the logs from the IllegalStateException: 
> 2018-07-18 03:02:27.510 ERROR 1 — [-StreamThread-2] o.a.k.s.p.internals.StreamThread
: stream-thread [prd1565.prod.nuke.ops.v1-StreamThread-2] Encountered the following error
during processing:
> java.lang.IllegalStateException: No current assignment for partition consumerGroup-stateStore-changelog-10
>  at org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:259)
>  at org.apache.kafka.clients.consumer.internals.SubscriptionState.resetFailed(SubscriptionState.java:413)
>  at org.apache.kafka.clients.consumer.internals.Fetcher$2.onFailure(Fetcher.java:595)
>  at org.apache.kafka.clients.consumer.internals.RequestFuture.fireFailure(RequestFuture.java:177)
>  at org.apache.kafka.clients.consumer.internals.RequestFuture.raise(RequestFuture.java:147)
>  at org.apache.kafka.clients.consumer.internals.RequestFutureAdapter.onFailure(RequestFutureAdapter.java:30)
>  at org.apache.kafka.clients.consumer.internals.RequestFuture$1.onFailure(RequestFuture.java:209)
>  at org.apache.kafka.clients.consumer.internals.RequestFuture.fireFailure(RequestFuture.java:177)
>  at org.apache.kafka.clients.consumer.internals.RequestFuture.raise(RequestFuture.java:147)
>  at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.fireCompletion(ConsumerNetworkClient.java:553)
>  at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.firePendingCompletedRequests(ConsumerNetworkClient.java:390)
>  at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:293)
>  at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)
>  at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1171)
>  at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1115)
>  at org.apache.kafka.streams.processor.internals.StreamThread.maybeUpdateStandbyTasks(StreamThread.java:1040)
>  at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:812)
>  at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:750)
>  at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:720)
> 2018-07-18 03:02:27.511 INFO 1 — [-StreamThread-2] o.a.k.s.p.internals.StreamThread
: stream-thread [consumerGroup-StreamThread-2] State transition from RUNNING to PENDING_SHUTDOWN
>  2018-07-18 03:02:27.511 INFO 1 — [-StreamThread-2] o.a.k.s.p.internals.StreamThread
: stream-thread [consumerGroup-StreamThread-2] Shutting down
>  2018-07-18 03:02:27.571 INFO 1 — [-StreamThread-2] o.a.k.clients.producer.KafkaProducer
: [Producer clientId=consumerGroup-StreamThread-2-producer] Closing the Kafka producer with
timeoutMillis = 9223372036854775807 ms.
>  2018-07-18 03:02:27.579 INFO 1 — [-StreamThread-2] o.a.k.s.p.internals.StreamThread
: stream-thread [consumerGroup-StreamThread-2] State transition from PENDING_SHUTDOWN to DEAD
>  2018-07-18 03:02:27.579 INFO 1 — [-StreamThread-2] o.a.k.s.p.internals.StreamThread
: stream-thread [consumerGroup-StreamThread-2] Shutdown complete
>  2018-07-18 03:02:27.579 ERROR 1 — [-StreamThread-2] xxx.xxx.xxx.AppRunner : Unhandled
exception in thread: 43:consumerGroup-StreamThread-2
> java.lang.IllegalStateException: No current assignment for partition consumerGroup-inventoryStore-changelog-10
>  at org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:259)
>  at org.apache.kafka.clients.consumer.internals.SubscriptionState.resetFailed(SubscriptionState.java:413)
>  at org.apache.kafka.clients.consumer.internals.Fetcher$2.onFailure(Fetcher.java:595)
>  at org.apache.kafka.clients.consumer.internals.RequestFuture.fireFailure(RequestFuture.java:177)
>  at org.apache.kafka.clients.consumer.internals.RequestFuture.raise(RequestFuture.java:147)
>  at org.apache.kafka.clients.consumer.internals.RequestFutureAdapter.onFailure(RequestFutureAdapter.java:30)
>  at org.apache.kafka.clients.consumer.internals.RequestFuture$1.onFailure(RequestFuture.java:209)
>  at org.apache.kafka.clients.consumer.internals.RequestFuture.fireFailure(RequestFuture.java:177)
>  at org.apache.kafka.clients.consumer.internals.RequestFuture.raise(RequestFuture.java:147)
>  at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.fireCompletion(ConsumerNetworkClient.java:553)
>  at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.firePendingCompletedRequests(ConsumerNetworkClient.java:390)
>  at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:293)
>  at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)
>  at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1171)
>  at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1115)
>  at org.apache.kafka.streams.processor.internals.StreamThread.maybeUpdateStandbyTasks(StreamThread.java:1040)
>  at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:812)
>  at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:750)
>  at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:720)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message