kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5825) Streams not processing when exactly once is set
Date Mon, 18 Sep 2017 19:49:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170564#comment-16170564

Matthias J. Sax commented on KAFKA-5825:

I just had one more look into your code. Maybe it's just test setup issue. (1) it seems, you
never close your {{KafkaStreams}} instance and (2) you use the same {{application.id}} for
both runs. Thus, if you run both tests in parallel, both instances would form a consumer group
and this would mess up the test.

> Streams not processing when exactly once is set
> -----------------------------------------------
>                 Key: KAFKA-5825
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5825
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions:
>         Environment: EmbeddedKafka running on Windows.  Relevant files attached.
>            Reporter: Ryan Worsley
>         Attachments: build.sbt, log4j.properties, log-output.txt, Tests.scala
> +Set-up+
> I'm using [EmbeddedKafka|https://github.com/manub/scalatest-embedded-kafka/] for ScalaTest.
> This spins up a single broker internally on a random port.
> I've written two tests - the first without transactions, the second with.  They're nearly
identical apart from the config and the transactional semantics.  I've written the transactional
version based on Neha's [blog|https://www.confluent.io/blog/exactly-once-semantics-are-possible-heres-how-apache-kafka-does-it/]
which is the closest thing I could find to instructions.
> The tests wait until a single message is processed by the streams topology, they use
this message to complete a promise that the test is waiting on.  Once the promise completes
the test verifies the value of the promise as being the expected value of the message.
> +Observed behaviour+
> The first test passes fine, the second test times out, the stream processor never seems
to read the transactional message.
> +Notes+
> I've attached my build.sbt, log4j.properties and my Tests.scala file in order to make
it as easy as possible for someone to re-create.  I'm running on Windows and using Scala as
this reflects my workplace.  I completely expect there to be some configuration issue that's
causing this, but am unable to proceed at this time.
> Related information: https://github.com/manub/scalatest-embedded-kafka/issues/82

This message was sent by Atlassian JIRA

View raw message