beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-3925) Allow ValueProvider for KafkaIO
Date Thu, 26 Apr 2018 17:03:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-3925?focusedWorklogId=95637&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-95637
]

ASF GitHub Bot logged work on BEAM-3925:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 26/Apr/18 17:02
            Start Date: 26/Apr/18 17:02
    Worklog Time Spent: 10m 
      Work Description: rangadi commented on a change in pull request #5141: [BEAM-3925] Allow
ValueProvider for KafkaIO so that we can create Beam Templates using KafkaIO
URL: https://github.com/apache/beam/pull/5141#discussion_r184173372
 
 

 ##########
 File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java
 ##########
 @@ -360,32 +387,85 @@
      * of how the partitions are distributed among the splits.
      */
     public Read<K, V> withTopicPartitions(List<TopicPartition> topicPartitions)
{
-      checkState(getTopics().isEmpty(), "Only topics or topicPartitions can be set, not both");
-      return toBuilder().setTopicPartitions(ImmutableList.copyOf(topicPartitions)).build();
+      return withTopicPartitions(ValueProvider.StaticValueProvider.of(topicPartitions));
+    }
+
+    /**
+     * Like above but with a {@link ValueProvider ValueProvider&lt;List&lt;TopicPartition&gt;&gt;}.
+     *
+     */
+    public Read<K, V> withTopicPartitions(ValueProvider<List<TopicPartition>>
topicPartitions) {
+      checkArgument(
+          getTopics().isAccessible(),
+          "Only topics or topicPartitions can be set, not both");
+      return toBuilder().setTopicPartitions(topicPartitions).build();
     }
 
     /**
-     * Sets a Kafka {@link Deserializer} to interpret key bytes read from Kafka.
+     * Sets a Kafka {@link Deserializer Deserializer&lt;K&gt;} for interpreting key
bytes read.
+     * This uses the {@link String} provided to set the Deserializer
+     */
+    public Read<K, V> withKeyDeserializerClassName(String keyDeserializer) {
+      return withKeyDeserializerClassName(ValueProvider.StaticValueProvider.of(keyDeserializer));
+    }
+
+    /**
+     * Like above but with a {@link ValueProvider ValueProvider&lt;String&gt;}.
 
 Review comment:
   `{@link ValueProvider ValueProvider&lt;String&gt;}.` looks odd. Please fix these
links every where in the PR.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 95637)

> Allow ValueProvider for KafkaIO
> -------------------------------
>
>                 Key: BEAM-3925
>                 URL: https://issues.apache.org/jira/browse/BEAM-3925
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-core
>            Reporter: Sameer Abhyankar
>            Assignee: Pramod Upamanyu
>            Priority: Major
>          Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> Add ValueProvider support for the various methods in KafkaIO. This would allow us to
use KafkaIO in reusable pipeline templates.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message