flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From orionemail <orionem...@protonmail.com>
Subject Re: How To subscribe a Kinesis Stream using enhance fanout?
Date Fri, 15 May 2020 09:54:50 GMT
Hi,

We also recently needed this functionality, unfortunately we were unable to implement it ourselves
so changed our plan accordingly.

However we very much see the benefit for this feature and would be interested in following
the JIRA ticket.

Thanks

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, 14 May 2020 11:34, Xiaolong Wang <xiaolong.wang@smartnews.com> wrote:

> Thanks, I'll check it out.
>
> On Thu, May 14, 2020 at 6:26 PM Tzu-Li (Gordon) Tai <tzulitai@apache.org> wrote:
>
>> Hi Xiaolong,
>>
>> You are right, the way the Kinesis connector is implemented / the way the AWS APIs
are used, does not allow it to consume Kinesis streams with enhanced fan-out enabled consumers
[1].
>> Could you open a JIRA ticket for this?
>> As far as I can tell, this could be a valuable contribution to the connector for
Kinesis users who require dedicated throughput isolated from other running consumers.
>>
>> Cheers,
>> Gordon
>>
>> [1] https://docs.aws.amazon.com/streams/latest/dev/building-enhanced-consumers-api.html
>>
>> On Wed, May 13, 2020 at 1:44 PM Xiaolong Wang <xiaolong.wang@smartnews.com>
wrote:
>>
>>> Hello Flink Community!
>>>
>>>       I'm currently coding on a project relying on AWS Kinesis. With the provided
connector (flink-connector-kinesis_2.11;1.10.0), I can consume the message.
>>>
>>>      But as the main stream is used among several other teams, I was required
to use the enhance fanout of Kinesis. I checked the connector code and found no implementations.
>>>
>>>      Has this issue occurred to anyone before?
>>>
>>>     Thanks for your help.
Mime
View raw message