Mailing-List: contact dev-help@streams.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@streams.incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of sblackmon@w2odigital.com
 designates 64.95.72.243 as permitted sender)
From: "Steve Blackmon [W2O Digital]" <sblackmon@w2odigital.com>
To: "dev@streams.incubator.apache.org" <dev@streams.incubator.apache.org>
Content-Class: urn:content-classes:message
Date: Sat, 31 Aug 2013 00:21:29 -0400
Subject: Re: Integration with Storm and Cassandra Java Driver
Thread-Topic: Integration with Storm and Cassandra Java Driver
Thread-Index: Ac6mATAdhG3FYSDRRhu0asY0g0fnFw==
Message-ID: <6591B075-B290-4E04-878B-9900209A3DA5@w2odigital.com>
References: 
 <SNT149-W36D7C06357509C55FAAF4A8A4A0@phx.gbl>,<CE4245E1.1D1CE%sblackmon@w2ogroup.com>,<SNT149-W19D67D7A85582343B8316E8A4B0@phx.gbl>,<CAENpzYWu0r5CwN6Z0D38a2RJGoJgnT9XEKJVM_SwmZLTGKRn6w@mail.gmail.com>,<CAL8syOYdMTCXi6iHYasH8dUO2gL=_Pms3+yUpF6b6b-Y0w8Mkg@mail.gmail.com>,<CAFNO4HgOrh2=WuruS=CV8zQO6W-5a9eEPmuH-0LwzmH6xWxgLw@mail.gmail.com>,<CAFNO4HiyUzVMMPvuV=V4Ngv7bJ6od-G0baQo2cPOgnt04-qM-A@mail.gmail.com>
 <SNT149-W913D42C39B8CDAF98AACBE8A4B0@phx.gbl>
In-Reply-To: <SNT149-W913D42C39B8CDAF98AACBE8A4B0@phx.gbl>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: multipart/alternative;
	boundary="_000_6591B075B2904E04878B9900209A3DA5w2odigitalcom_"
MIME-Version: 1.0

--_000_6591B075B2904E04878B9900209A3DA5w2odigitalcom_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

That=92s correct, the code running in Storm decides where messages are head=
ed - kafka is a scalable speedy FIFO buffer that can sit between any nodes =
in the network you want it to.  1-to-1, 1-to-many, or many-to-many.

Steve Blackmon
Director, Data Sciences
W2O Group
3000 E Cesar Chavez St.
Austin, Texas 78701
cell 512.965.0451 | work 512.402.6366
twitter @steveblackmon

On Aug 28, 2013, at 3:37 PM, Danny Sullivan <dsullivan7@hotmail.com<mailto:=
dsullivan7@hotmail.com>> wrote:

I understood that Storm, not Kafka, would overtake the responsibilities of =
Camel through Spouts and Bolts:
https://github.com/nathanmarz/storm/wiki/Tutorial

Date: Wed, 28 Aug 2013 13:19:34 -0700
Subject: Re: Integration with Storm and Cassandra Java Driver
From: chris@cxtsoftware.com
To: dev@streams.incubator.apache.org

On Wed, Aug 28, 2013 at 1:19 PM, Chris Geer <chris@cxtsoftware.com> wrote:

On Wed, Aug 28, 2013 at 1:15 PM, Jason Letourneau <jletourneau80@gmail.com
wrote:

Matt's desire might create an even more complex architecture, though a
wonderful goal.  I wouldn't be opposed to just picking a solution and
designing around it for the purposes of getting the end to end
solution running and then revisiting the appropriate abstractions.

I had seen Kafka when starting out on the initial code base, I thought
it held serious promise for Streams.

Advantage of Camel is the ability to deploy as part of the web archive
vs a standalone service, if Kafka/Storm bring that as well, sounds
cool to me.  They seem to bring the high performance hammer of doom -
rock on. \\m//


I using Kafka gets you out of using something like Camel (or custom code)
to orchestrate your routes and other end points (i.e. Twitter).


Should have read "I'm not sure using...."


On Wed, Aug 28, 2013 at 3:40 PM, Matt Franklin <m.ben.franklin@gmail.com>
wrote:
On Wed, Aug 28, 2013 at 2:14 PM, Danny Sullivan <dsullivan7@hotmail.com
wrote:

Thanks for the info Steve.
As I understand, Kafka would take the place of the functionality of
what
ActiveMQ does now. Storm would take place of what Camel does now.


I think in the long term we need to have a flexible architecture with a
few
implementations.  The way I see it, we need collection, orchestration,
processing pipeline, persistence and exposure.  If there is a way that
we
can define each of these components loosely coupled enough to where we
could have a Kafka OR AMQP routing implementation that would be ideal.
I
haven't thought through exactly how to do this myself, but wanted to
offer
that things may not be mutually exclusive.


CONFIDENTIALITY NOTICE: This e-mail, along with any documents, files, or at=
tachments, may contain information that is confidential, privileged, or oth=
erwise exempt from disclosure. If you are not the intended recipient or per=
son responsible for delivering it to the intended recipient, you are hereby=
 notified that any disclosure, copying, printing, distribution or use of an=
y information contained in or attached to this e-mail is strictly prohibite=
d. If you have received this e-mail in error, please immediately notify the=
 sender and delete the original e-mail and its attachments without reading,=
 printing, or saving in any manner. This e-mail message should not be inter=
preted to include a digital or electronic signature that can be used to aut=
henticate an agreement, contract or other legal document, nor to reflect an=
 intention to be bound to any legally-binding agreement or contract.  Your =
cooperation is appreciated. Thank you.

--_000_6591B075B2904E04878B9900209A3DA5w2odigitalcom_--