kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Davis (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-3625) Move kafka-streams test fixtures into a published package
Date Wed, 10 Jan 2018 16:43:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16320605#comment-16320605

Scott Davis commented on KAFKA-3625:

FYI I just setup some unit tests using the (not documented) internal test classes in Kafka
Streams, and I thought I'd share some thoughts on my experience with it.

First, it's super-helpful! The high-level DSL style of Kafka Streams applications doesn't
fit well into standard unit testing frameworks, and this solves that problem. It's also helpful
compared with integration testing in the sense that it isn't necessary to produce source messages
on a broker, wait for timeouts, etc.

The internal test classes in 1.0.0 require the unit tests to provide serializers. This can
create some extra boilerplate work to configure the serializers (especially the Confluent
AVRO Serde, which requires a schema registry). However, I only need to unit test my application
logic (i.e. the contents of the "filter" and "map" methods, etc). I can see how testing serializers
is a requirement for testing Kafka Streams internally, but it isn't a requirement for testing
the logic of Kafka Streams applications. Note: To work around this, I created a "JavaObjectSerde",
which uses java.io.Object(In|Out)putStream, for use by the unit tests.

I needed about 10 lines of boilerplate code in an @Before method to setup the ProcessorTopologyTestDriver,
which seemed slightly excessive but not burdensome. Most of it was to create the serializers
and the StreamsConfig, which aren't part of my application logic but were required to create
the test driver.

> Move kafka-streams test fixtures into a published package
> ---------------------------------------------------------
>                 Key: KAFKA-3625
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3625
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Jeff Klukas
>            Assignee: Matthias J. Sax
>            Priority: Minor
>              Labels: needs-kip, user-experience
> The KStreamTestDriver and related fixtures defined in streams/src/test/java/org/apache/kafka/test
would be useful to developers building applications on top of Kafka Streams, but they are
not currently exposed in a package.
> I propose moving this directory to live under streams/fixtures/src/main and creating
a new 'streams:fixtures' project in the gradle configuration to publish these as a separate
> KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-247%3A+Add+public+test+utils+for+Kafka+Streams

This message was sent by Atlassian JIRA

View raw message