kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4607) Kafka Streams allows you to provide strings with illegal characters for internal topic names
Date Fri, 06 Jan 2017 23:27:58 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806138#comment-15806138

ASF GitHub Bot commented on KAFKA-4607:

GitHub user nixsticks opened a pull request:


    KAFKA-4607: Validate the names of auto-generated internal topics

    I considered catching errors to add further information about naming internal state stores.
However, Topic.validate() will throw an error that prints the offending name, so I decided
not to add too much complexity.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/nixsticks/kafka internal-topics

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2331
commit 5ffb1ccd2a6624ba96522612caf30fe13f183fa4
Author: Nikki Thean <nthean@etsy.com>
Date:   2017-01-06T23:21:38Z

    KAFKA-4607: Validate the names of auto-generated internal topics


> Kafka Streams allows you to provide strings with illegal characters for internal topic
> --------------------------------------------------------------------------------------------
>                 Key: KAFKA-4607
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4607
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Nikki Thean
>            Priority: Minor
> When using the aggregate function of the Kafka Streams DSL, I supplied the function with
a name for the underlying state store that contained whitespace. The stream application ran
with no error message but seemed to be hanging. 
> After a lot of troubleshooting, I looked in the logs on the Kafka cluster and noticed
that it was complaining about illegal characters in the log line. It turns out that an internal
changelog topic had been created that contained whitespace and this was causing both Kafka
and ZK to choke. I was unable to use the delete-topic command line utility as it also threw
the illegal character error. In the end I had to manually delete the topics out of ZK and
do some stuff with the offset logs.
> I am not well versed in the Kafka or Kafka Streams code, but from looking through it,
I notice that [InternalTopicManager|https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java#L170]
does not seem to validate the names of auto-generated internal topics beyond [checking for
null topic names|https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicConfig.java#L38]
rather than using Topic.validate.

This message was sent by Atlassian JIRA

View raw message