kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Farmer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6214) Using standby replicas with an in memory state store causes Streams to crash
Date Thu, 16 Nov 2017 15:40:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16255490#comment-16255490
] 

Matt Farmer commented on KAFKA-6214:
------------------------------------

Is the answer here, then, to just remove that {{if}} statement?

> Using standby replicas with an in memory state store causes Streams to crash
> ----------------------------------------------------------------------------
>
>                 Key: KAFKA-6214
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6214
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 0.11.0.1
>            Reporter: Matt Farmer
>              Labels: statestore
>
> We decided to start experimenting with Standby Replicas of our State Stores by setting
the following configuration setting:
> {code}
> num.standby.replicas=1
> {code}
> Most applications did okay with this except for one that used an in memory state store
instead of a persistent state store. With the new configuration, the first instance of this
application booted fine. When the second instance came up, both instances crashed with the
following exception:
> {code}
> java.lang.IllegalStateException: Consumer is not subscribed to any topics or assigned
any partitions
>         at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1037)
>         at org.apache.kafka.streams.processor.internals.StreamThread.maybeUpdateStandbyTasks(StreamThread.java:752)
>         at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:524)
>         at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:480)
>         at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:457)
> {code}
> Monit attempted to restart both instances but they would just continue to crash over
and over again. The state store in our problematic application is declared like so:
> {code}
> Stores
>     .create("TheStateStore")
>     .withStringKeys()
>     .withStringValues()
>     .inMemory()
>     .build()
> {code}
> Luckily we had a config switch in place that could turn on an alternate, persistent state
store. As soon as we flipped to the persistent state store, things started working as we expected.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message