beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré>
Subject Re: 答复: Public in-progress I/O Transform list
Date Wed, 12 Apr 2017 06:22:16 GMT

Yes, I saw that and already provided some info.

The RestIO I have works as follow:

- Producer Read (working)


It provides a PCollection<String> where String is JSON or XML. It's also 
possible to provide a data binding with withDataBinding() method.

It's basically a DoFn as it's not possible to do any splitting.

- Producer Write (working)


Here, it's basically the same DoFn as producer using input PCollection as REST 

- Consumer Read (work in progress)


Here it's an unbounded source that bind a CXF connector (the initialSplit() 
method defines where the connector will be bind).

I'm experiment this as it's more  "ambitious" than the producer mode.


On 04/12/2017 08:16 AM, Tang Jijun(上海_技术部_数据平台_唐觊隽) wrote:
> Hi Stephen,
> We need restIO module. I am looking forward to your pr about it. For restIO ,I create
a jira
> -----邮件原件-----
> 发件人: Jean-Baptiste Onofré []
> 发送时间: 2017年4月12日 14:10
> 收件人:
> 主题: Re: Public in-progress I/O Transform list
> Hi Stephen,
> Thanks !
> I would add Socket IO and REST IO that I  have on local branches (I'm waiting to close
some of my PRs before opening new ones). I also  have (not fully ready for
> PR) CamelIO, IronMQIO, and others (I will update the website  on  the fly ;)).
> MQTT IO is missing in the list. Gonna provide a website PR about that.
> I also created ExecIO while ago but we didn't get a consensus about what we should do
with it.
> Regards
> JB
> On 04/12/2017 02:01 AM, Stephen Sisk wrote:
>> Hi!
>> We occasionally get questions about whether or not an I/O is planned
>> to be added to Beam. I've added a list of known in-progress I/O
>> Transforms to the "Built-in Transforms" page (where in-progress is
>> defined as "has a JIRA issue"). The Built-In Transforms page is the
>> publicly visible list of I/O transforms, so it's a natural place to let users know
what's also in-progress.
>> You can find the current list here:
>> I may have missed one or two I/Os (there's no good way to find them in
>> JIRA as far as I can tell), so if you're working on an I/O Transform
>> that you intend to contribute to beam that's not listed, feel free to send a PR adding
it to the list.
>> Thanks,
>> Stephen
> --
> Jean-Baptiste Onofré
> Talend -

Jean-Baptiste Onofré
Talend -

View raw message