kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clay Teahouse <clayteaho...@gmail.com>
Subject Re: kafka connect questions
Date Wed, 05 Jul 2017 04:09:39 GMT
Hello Gwen,

Thanks for the reply. My comments/answers inline.

1. Connectors that listen on sockets typically run in stand-alone mode, so
they can tied to a specific machine (in distributed mode, connectors can
move around).
[Clay:] Even if the connectors move around, they can still listen to a
specific port on the node in the cluster, right? The data will be sent to
the cluster of connectors from hundreds of data sources.
2. Why do you need a connector? Why not just use Kafka producer to send
protobuf directly to Kafka?

[Clay:] I have hundreds of data sources which push the data to the
connectors. I do need the connectors to run in a cluster mode, for HA and
scalability.



On Tue, Jul 4, 2017 at 10:45 PM, Gwen Shapira <gwen@confluent.io> wrote:

> I don't remember seeing one. There is no reason not to write one (let us
> know if you do, so we can put it on the connector hub!).
>
> Few things:
> 1. Connectors that listen on sockets typically run in stand-alone mode, so
> they can tied to a specific machine (in distributed mode, connectors can
> move around).
> 2. Why do you need a connector? Why not just use Kafka producer to send
> protobuf directly to Kafka?
>
> Gwen
> On Tue, Jul 4, 2017 at 9:02 AM Clay Teahouse <clayteahouse@gmail.com>
> wrote:
>
> > Hello All,
> >
> > I'd appreciate your help with the following questions.
> >
> > 1) Is there kafka connect for listening to tcp sockets?
> >
> > 2) If, can the messages be in protobuf, with each messaged prefixed with
> > the length of the message?
> >
> > thanks
> > Clay
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message