activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Shannon <christopher.l.shan...@gmail.com>
Subject Re: Kafka ServiceConnector
Date Fri, 06 Oct 2017 11:07:32 GMT
Why not use Kafka's Connect framework?  Seems like building a source
connector here would make the most sense.  This is not to say I'm against
the idea I'm just wondering the benefits of doing it from the Artemis side
of things when Kafka already has a framework that can be used to
import/export data.

On Fri, Oct 6, 2017 at 6:16 AM, Michael André Pearce <
michael.andre.pearce@me.com> wrote:

> Hi Andy,
>
> As you may be aware or not we run both activemq and kafka each has a
> specific plus point to it, use the right tool for the right job.
>
> We use activemq as our feature rich, fully jms compliant and performant
> broker.
>
> And use kafka more for post processing and data streaming, with historical
> feed also.
>
> Idea here is that producers and consumers bind into our activemq where jms
> api is wanted or needed, lower per message latencies etc. but then we can
> bridge messages from a core queue into kafka so that the data can flow to
> it as fast as possible and consumed from there also.
>
> Idea is to try make a hybrid taking best of both.
>
> Cheers
> Mike
>
> Sent from my iPhone
>
> > On 6 Oct 2017, at 09:37, Andy Taylor <andy.tayls67@gmail.com> wrote:
> >
> > Whats the use case here Michael, sounds like a cool idea and something
> that
> > has been mentioned by different people but I am yet to understand how
> this
> > could be used.
> >
> > On 6 October 2017 at 07:48, Michael André Pearce <
> > michael.andre.pearce@me.com> wrote:
> >
> >> Hi All,
> >>
> >> I am looking to contribute an artemis ServiceConnector that would
> leverage
> >> the existing service connector interface to bridge from a queue into a
> >> kafka topic similar to bridge between artemis clusters.
> >>
> >> Idea is to make it configurable how to transform and encode the message
> in
> >> kafka but as amqp is becoming very well supported by artemis provide out
> >> the box that the message be encoded in amqp byte protocol.
> >>
> >> It something I’ve been working on privately for a bit, but since
> becoming
> >> a committer here is really like to share it.
> >>
> >> I hope if I get a positive response here to get a branch ready soon, but
> >> feedback if this is welcomed would be great.
> >>
> >> Cheers
> >> Mike
> >>
> >>
> >> Sent from my iPhone
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message