activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy Taylor <andy.tayl...@gmail.com>
Subject Re: Kafka ServiceConnector
Date Fri, 06 Oct 2017 11:14:47 GMT
I think https://github.com/ppatierno/kafka-connect-amqp is an example of
this

On 6 October 2017 at 12:07, Christopher Shannon <
christopher.l.shannon@gmail.com> wrote:

> Why not use Kafka's Connect framework?  Seems like building a source
> connector here would make the most sense.  This is not to say I'm against
> the idea I'm just wondering the benefits of doing it from the Artemis side
> of things when Kafka already has a framework that can be used to
> import/export data.
>
> On Fri, Oct 6, 2017 at 6:16 AM, Michael André Pearce <
> michael.andre.pearce@me.com> wrote:
>
> > Hi Andy,
> >
> > As you may be aware or not we run both activemq and kafka each has a
> > specific plus point to it, use the right tool for the right job.
> >
> > We use activemq as our feature rich, fully jms compliant and performant
> > broker.
> >
> > And use kafka more for post processing and data streaming, with
> historical
> > feed also.
> >
> > Idea here is that producers and consumers bind into our activemq where
> jms
> > api is wanted or needed, lower per message latencies etc. but then we can
> > bridge messages from a core queue into kafka so that the data can flow to
> > it as fast as possible and consumed from there also.
> >
> > Idea is to try make a hybrid taking best of both.
> >
> > Cheers
> > Mike
> >
> > Sent from my iPhone
> >
> > > On 6 Oct 2017, at 09:37, Andy Taylor <andy.tayls67@gmail.com> wrote:
> > >
> > > Whats the use case here Michael, sounds like a cool idea and something
> > that
> > > has been mentioned by different people but I am yet to understand how
> > this
> > > could be used.
> > >
> > > On 6 October 2017 at 07:48, Michael André Pearce <
> > > michael.andre.pearce@me.com> wrote:
> > >
> > >> Hi All,
> > >>
> > >> I am looking to contribute an artemis ServiceConnector that would
> > leverage
> > >> the existing service connector interface to bridge from a queue into a
> > >> kafka topic similar to bridge between artemis clusters.
> > >>
> > >> Idea is to make it configurable how to transform and encode the
> message
> > in
> > >> kafka but as amqp is becoming very well supported by artemis provide
> out
> > >> the box that the message be encoded in amqp byte protocol.
> > >>
> > >> It something I’ve been working on privately for a bit, but since
> > becoming
> > >> a committer here is really like to share it.
> > >>
> > >> I hope if I get a positive response here to get a branch ready soon,
> but
> > >> feedback if this is welcomed would be great.
> > >>
> > >> Cheers
> > >> Mike
> > >>
> > >>
> > >> Sent from my iPhone
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message