beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jesse Anderson <je...@smokinghand.com>
Subject Re: KafkaIO Writer
Date Tue, 07 Jun 2016 17:51:36 GMT
@Raghu - I'm not making a judgement on KafkaIO. I'm more saying that the
API should be consistent. If read() and write() is the new way, that's
great. We'd need to change the other IOs to match IMHO.

BTW thanks for writing this Kafka integration Raghu.

On Tue, Jun 7, 2016 at 1:08 PM Raghu Angadi <rangadi@google.com> wrote:

> I think FooIO.read() and FooIO.write() is the new API convention for IO
> (recently BigtableIO from Dan does the same).
>
> On Tue, Jun 7, 2016 at 9:50 AM, Jesse Anderson <jesse@smokinghand.com>
> wrote:
>
>> Ah the over the weekend part explains that.
>>
>> Is there a reason KafkaIO doesn't follow the separate classes for read
>> and write like TextIO does? Following the TextIO style, there should be a
>> KafkaIO.Read.method() or KafkaIO.Write.method().
>>
>> On Tue, Jun 7, 2016 at 12:22 PM Dan Halperin <dhalperi@google.com> wrote:
>>
>>> Hi Jesse,
>>>
>>> KafkaIO supports write:
>>> https://github.com/apache/incubator-beam/blob/master/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java#L161
>>>
>>> (This was merged over the weekend as part of @rangadi's
>>> https://github.com/apache/incubator-beam/pull/271)
>>>
>>> Dan
>>>
>>> On Tue, Jun 7, 2016 at 9:04 AM, Jesse Anderson <jesse@smokinghand.com>
>>> wrote:
>>>
>>>> Are there plans to add a writer to KafkaIO? Right now, it's only read.
>>>>
>>>> The class itself is written to only expect a reader. Adding any sort of
>>>> write would require refactoring the class to Read and Write like TextIO
>>>> does.
>>>>
>>>> Thanks,
>>>>
>>>> Jesse
>>>>
>>>
>>>
>

Mime
View raw message