flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ray Ruvinskiy <ray.ruvins...@arcticwolf.com>
Subject Flink Connector for AWS SQS
Date Mon, 17 Jul 2017 14:50:20 GMT
I’m interested in writing a connector for AWS SQS (https://aws.amazon.com/sqs/). I’ve looked,
and I don’t believe one is available. Flink does have a Kinesis collector, and there is
some overlap in terms of basic AWS configuration handling (I believe mostly limited to what’s
in https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/config/AWSConfigConstants.java).
I have a couple of questions:

1. Should a potential new SQS connector be submitted to the flink project directly or to bahir-flink?
I’m not entirely clear on the protocol. How is it decided which connector goes where?
2. Should I try to make use of the already existing AWSConfigConstants that have been written
for Kinesis, or should I effectively replicate them? I can see potential for factoring them
out to a common AWS class if both connectors were to live in flink proper, but that wouldn’t
be possible if SQS lived in bahir-flink.



View raw message