apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandesh Hegde <sand...@datatorrent.com>
Subject Re: Dynamic partition is not working in Kafka Input Operator
Date Tue, 19 Jul 2016 04:17:12 GMT
Was this resolved?

My understanding is that, Kafka Input operator doesn't support the changes
in Kafka partitions after the initial launch.

On Mon, Jul 18, 2016 at 1:54 AM Chaitanya Chebolu <chaitanya@datatorrent.com>
wrote:

> Hi All,
>
>    I am facing dynamic partition issues in 0.8 version of Kafka Input
> Operator. My application has the following DAG:
>
>    KafkaSinglePortStringInputOperator(Input) ->
> ConsoleOutputOperator(Output)
>
>    I launched the application with below configuration:
> Kafka topic created with single partition and replication factor as 1.
> Partition Strategy: ONE_TO_ONE
>
>    Launched the application successfully. After some time, I increased the
> topic partitions to 2. After re-partition, the window of down stream
> operator is not moving. By looking into the app Physical DAG, it looks like
> there is an issue in construction of Physical DAG after re-partition.
>
> Please let me know if any one observed the same behavior. Do we have JIRA
> for tracking this issue.
> I am attaching some of the screenshots of this application.
>
> Regards,
> Chaitanya
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message