beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Groh (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (BEAM-2449) Support ValueProviders in KafkaIO
Date Wed, 14 Jun 2017 16:49:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-2449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Thomas Groh reassigned BEAM-2449:
---------------------------------

       Assignee:     (was: Thomas Groh)
    Component/s:     (was: runner-dataflow)

> Support ValueProviders in KafkaIO
> ---------------------------------
>
>                 Key: BEAM-2449
>                 URL: https://issues.apache.org/jira/browse/BEAM-2449
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-extensions
>    Affects Versions: 2.0.0
>            Reporter: peay
>            Priority: Minor
>
> KafkaIO takes parameters at construction time:
> - When partitions are provided, KafkaIO actually does nothing particular at construction
time.
> - When only a topic is provided, I believe KafkaIO does fetch the list of partitions
at construction time.
> Using ValueProvider for the brokers would be a good improvement. In particular, when
partitions are provided, a RuntimeProvider could be passed in.
> Being able to always use a RuntimeProvider would be even better, such as when using {{TextIO.xxx.withoutValidation()}}:
this would considerably improve the workflow of using Dataflow templates with KafkaIO, which
currently requires to hardcode brokers into the template.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message