kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Damian Guy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6214) Using standby replicas with an in memory state store causes Streams to crash
Date Wed, 15 Nov 2017 16:54:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16253783#comment-16253783

Damian Guy commented on KAFKA-6214:

Looks like the issue is in {{ProcessorStateManager}}. We only restore and hence assign partitions
if the store is persistent
if (store.persistent()) {
                log.trace("Preparing standby replica of persistent state store {} with changelog
topic {}", store.name(), topic);

                restoreCallbacks.put(topic, stateRestoreCallback);

> Using standby replicas with an in memory state store causes Streams to crash
> ----------------------------------------------------------------------------
>                 Key: KAFKA-6214
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6214
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions:
>            Reporter: Matt Farmer
>              Labels: statestore
> We decided to start experimenting with Standby Replicas of our State Stores by setting
the following configuration setting:
> {code}
> num.standby.replicas=1
> {code}
> Most applications did okay with this except for one that used an in memory state store
instead of a persistent state store. With the new configuration, the first instance of this
application booted fine. When the second instance came up, both instances crashed with the
following exception:
> {code}
> java.lang.IllegalStateException: Consumer is not subscribed to any topics or assigned
any partitions
>         at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1037)
>         at org.apache.kafka.streams.processor.internals.StreamThread.maybeUpdateStandbyTasks(StreamThread.java:752)
>         at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:524)
>         at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:480)
>         at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:457)
> {code}
> Monit attempted to restart both instances but they would just continue to crash over
and over again. The state store in our problematic application is declared like so:
> {code}
> Stores
>     .create("TheStateStore")
>     .withStringKeys()
>     .withStringValues()
>     .inMemory()
>     .build()
> {code}
> Luckily we had a config switch in place that could turn on an alternate, persistent state
store. As soon as we flipped to the persistent state store, things started working as we expected.

This message was sent by Atlassian JIRA

View raw message