kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Werner Daehn <werner.da...@gmail.com>
Subject Broaden the utilization of Kafka by being more user friendly
Date Fri, 24 Feb 2017 10:28:48 GMT
I would have a couple of ideas about Kafka and would love to get your
input. If I am totally off, if it is possible today without me noticing or
if you can embrace/do disregard the idea.

Statement 1: A typical scenario for Kafka can be coupling on-premise system
with a Kafka based cloud application. Today Kafka requires a direct
connection but the on-prem side does allow http only.

Kafka Rest is obviously one solution, but it is slow and has overhead.
Suggestion: In Kafka Connect add the ability to run connectors on-prem.
Each opens one long running http connection with the Kafka server streaming
the data. Or maybe a WebSocket as a bridge.


Statement 2: For above use cases you need to support security and
multi-tenant.

Suggestion: Above http connection should validate that a connector has the
proper certificate and is entitled to write into a Kafka topic.


Statement 3: Connect needs an UI for being configured by a non-Kafka user.

Suggestion: The connectors are hosted by a Tomcat which also acts as the
configuration, administration and monitoring UI for the on-prem side.


I have a few more thoughts about metadata (schema registry), a mapping
(Kafka Connect Transform) UI,... but above would be the most pressing
points for me.

Thanks in advance

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message