kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-4269) Multiple KStream instances with at least one Regex source causes NPE when using multiple consumers
Date Wed, 23 Nov 2016 14:59:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Guozhang Wang updated KAFKA-4269:
---------------------------------
    Fix Version/s: 0.10.1.1

> Multiple KStream instances with at least one Regex source causes NPE when using multiple
consumers
> --------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-4269
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4269
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 0.10.1.0
>            Reporter: Bill Bejeck
>            Assignee: Bill Bejeck
>             Fix For: 0.10.1.1, 0.10.2.0
>
>
> I discovered this issue while doing testing for for KAFKA-4114. 
> KAFKA-4131 fixed the issue of a _single_ KStream with a regex source on partitioned topics
across multiple consumers.
> //KAFKA-4131 fixed this case assuming an "foo*" topics are partitioned
> KStream kstream = builder.source(Pattern.compile("foo.*"));
> KafkaStream stream = new KafkaStreams(builder, props);
> stream.start();  
> This is a new issue where there are _multiple_
> KStream instances (and one has a regex source) within a single KafkaStreams object. When
running the second or "following"
> consumer there are NPE errors generated in the RecordQueue.addRawRecords method when
attempting to consume records. 
> For example:
> KStream kstream = builder.source(Pattern.compile("foo.*"));
> KStream kstream2 = builder.source(.....): //can be regex or named topic sources
> KafkaStream stream = new KafkaStreams(builder, props);
> stream.start();
> By adding an additional KStream instance like above (whether Regex or Named topic) causes
a NPE when run as "follower"
> From my initial debugging I can see the TopicPartition assignments being set on the "follower"
KafkaStreams instance, but need to track down why and where all assignments aren't being set.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message