streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Blackmon [W2O Digital]" <sblack...@w2odigital.com>
Subject Re: Integration with Storm and Cassandra Java Driver
Date Sat, 31 Aug 2013 04:21:29 GMT
That’s correct, the code running in Storm decides where messages are headed - kafka is a
scalable speedy FIFO buffer that can sit between any nodes in the network you want it to.
 1-to-1, 1-to-many, or many-to-many.

Steve Blackmon
Director, Data Sciences
W2O Group
3000 E Cesar Chavez St.
Austin, Texas 78701
cell 512.965.0451 | work 512.402.6366
twitter @steveblackmon

On Aug 28, 2013, at 3:37 PM, Danny Sullivan <dsullivan7@hotmail.com<mailto:dsullivan7@hotmail.com>>
wrote:

I understood that Storm, not Kafka, would overtake the responsibilities of Camel through Spouts
and Bolts:
https://github.com/nathanmarz/storm/wiki/Tutorial

Date: Wed, 28 Aug 2013 13:19:34 -0700
Subject: Re: Integration with Storm and Cassandra Java Driver
From: chris@cxtsoftware.com
To: dev@streams.incubator.apache.org

On Wed, Aug 28, 2013 at 1:19 PM, Chris Geer <chris@cxtsoftware.com> wrote:

On Wed, Aug 28, 2013 at 1:15 PM, Jason Letourneau <jletourneau80@gmail.com
wrote:

Matt's desire might create an even more complex architecture, though a
wonderful goal.  I wouldn't be opposed to just picking a solution and
designing around it for the purposes of getting the end to end
solution running and then revisiting the appropriate abstractions.

I had seen Kafka when starting out on the initial code base, I thought
it held serious promise for Streams.

Advantage of Camel is the ability to deploy as part of the web archive
vs a standalone service, if Kafka/Storm bring that as well, sounds
cool to me.  They seem to bring the high performance hammer of doom -
rock on. \\m//


I using Kafka gets you out of using something like Camel (or custom code)
to orchestrate your routes and other end points (i.e. Twitter).


Should have read "I'm not sure using...."


On Wed, Aug 28, 2013 at 3:40 PM, Matt Franklin <m.ben.franklin@gmail.com>
wrote:
On Wed, Aug 28, 2013 at 2:14 PM, Danny Sullivan <dsullivan7@hotmail.com
wrote:

Thanks for the info Steve.
As I understand, Kafka would take the place of the functionality of
what
ActiveMQ does now. Storm would take place of what Camel does now.


I think in the long term we need to have a flexible architecture with a
few
implementations.  The way I see it, we need collection, orchestration,
processing pipeline, persistence and exposure.  If there is a way that
we
can define each of these components loosely coupled enough to where we
could have a Kafka OR AMQP routing implementation that would be ideal.
I
haven't thought through exactly how to do this myself, but wanted to
offer
that things may not be mutually exclusive.










CONFIDENTIALITY NOTICE: This e-mail, along with any documents, files, or attachments, may
contain information that is confidential, privileged, or otherwise exempt from disclosure.
If you are not the intended recipient or person responsible for delivering it to the intended
recipient, you are hereby notified that any disclosure, copying, printing, distribution or
use of any information contained in or attached to this e-mail is strictly prohibited. If
you have received this e-mail in error, please immediately notify the sender and delete the
original e-mail and its attachments without reading, printing, or saving in any manner. This
e-mail message should not be interpreted to include a digital or electronic signature that
can be used to authenticate an agreement, contract or other legal document, nor to reflect
an intention to be bound to any legally-binding agreement or contract.  Your cooperation is
appreciated. Thank you.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message