kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6282) exactly_once semantics breaks demo application
Date Fri, 01 Dec 2017 01:53:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16273817#comment-16273817

Guozhang Wang commented on KAFKA-6282:

[~mjsax] I think we can update the config docs of `StreamsConfig` for this property mentioning
that exactly once is dependent on the transactional messaging with the default replication
factor; so if users trying to turn on this config need to either make sure the num.brokers
is no less than the replication factor or this broker-side config is overridden. We'd suggest
the first approach since in practice with less or no replicas persistency is not safe although
we maintain consistency.

> exactly_once semantics breaks demo application
> ----------------------------------------------
>                 Key: KAFKA-6282
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6282
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions:, 1.0.0
>         Environment: Tested on i7+24GB Ubuntu16.04 and i7+8GB Windows 7, with cluster
1.0.0 and and streams 1.0.0 and
>            Reporter: Romans Markuns
>         Attachments: WordCountDemo.java, server.properties
> +What I try to achieve+
> Do successful run of Kafka streams app with setting "processing.guarantee" set to "exactly_once"
> +How+
> Use Kafka quickstart example (https://kafka.apache.org/10/documentation/streams/quickstart)
and modify only configuration parameters. 
> Things I've changed:
> 1) Add one line to WordCountDemo: 
> {code:java}
> props.put(StreamsConfig.PROCESSING_GUARANTEE_CONFIG, StreamsConfig.EXACTLY_ONCE);
> {code}
> 2) Modify server.properties to be the same as we use in QA: set broker id to 1, allow
deleting topics via admin client and set initial rebalance delay to 3 s.
> +What I expect+
> Modified demo app works exactly as the original as presented in link above.
> +What I get+
> 1) Original app works fine. Output topic after each line is submitted via console producer.
> 2) Modified app does not process topic record after it is submitted via console producer.
Streams remain in state REBALANCING, no errors on warning printed. MAIN thread forever blocks
waiting TransactionCoordinator response (CountdownLatch.await()) and this message getting
printed: [kafka-producer-network-thread | streams-wordcount-client-StreamThread-1-0_0-producer]
DEBUG org.apache.kafka.clients.producer.internals.TransactionManager - [Producer clientId=streams-wordcount-client-StreamThread-1-0_0-producer,
transactionalId=streams-wordcount-0_0] Enqueuing transactional request (type=FindCoordinatorRequest,
coordinatorKey=streams-wordcount-0_0, coordinatorType=TRANSACTION)

This message was sent by Atlassian JIRA

View raw message