streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Danny Sullivan <>
Subject RE: Integration with Storm and Cassandra Java Driver
Date Wed, 28 Aug 2013 19:14:46 GMT
Thanks for the info Steve.
As I understand, Kafka would take the place of the functionality of what ActiveMQ does now.
Storm would take place of what Camel does now. I am interested to know whether Cassandra would
be part of the "mid-stream persistence" that you mentioned or if it would be completely separate
from Kafka, serving as a holding station after a message has made it through?
Also, could you explain what you mean by "what Cassandra is good at" and why "CQL 3 isn't
the best way to take advantage" of it? 
I'm in the process of looking at Intravert.

> From:
> To:
> Date: Tue, 27 Aug 2013 13:30:44 -0400
> Subject: Re: Integration with Storm and Cassandra Java Driver
> Storm can make a great execution container for Streams.  W2O has used
> Storm Trident for many use cases in Apache Streams roadmap and we are
> interested in developing this aspect of streams.
> We currently use Kafka as our primary queuing and mid-stream persistence
> mechanism.  It works fine, when in general all messages follow a linear
> progression from source to sink through a stream.  We haven¹t needed
> complex forking conditions or multi-module pub/sub yet - in those cases
> Cassandra can support queries which will ease implementation vs Kafka.
> As we look at client libraries for Cassandra, here¹s another contender.
> Intravert runs alongside C* on each node, exposes a very simple rest API,
> and lets the client specify script-like conditions that get evaluated
> within the C* cluster as result sets are prepared.  I have a lot of
> respect for the Datastax team, but I think CQL has gone somewhat off the
> rails and CQL 3 isn¹t the best way to take advantage of what Cassandra is
> good at.
> Steve Blackmon
> Director, Data Sciences
> W2O Group
> 101 W. 6th Street Suite 330
> Austin, Texas 78701
> cell 512.965.0451 | work 512.402.6366
> twitter @steveblackmon
> On 8/27/13, 11:19 AM, "Danny Sullivan" <> wrote:
> >The Storm Project,, would add processing of
> >ActivityStreamsEntry objects to Apache Streams (make it faster). Storm
> >has integration support for Cassandra. There is an available Java Driver
> >to hook up Storm and Cassandra available here:
> > While I think it is a good
> >idea to add Storm to Streams, Cassandra recommends using CQL3 as an
> >interface from applications to the database which moves away from older
> >thrift clients: These
> >older thrift clients include Astyanax and Hector (I have had experience
> >incorporating both into the project, with varied success). The newer
> >client which relies on CQL3 support is the Datastax Java Driver:
> > I've looked into the Datastax
> >driver and have been very pleased with the CQL support. However, it is
> >not very far along in the development process and doesn't have an object
> >mapper and I'm cautious adopting a product so early in its development.
> >If we were to integrate Storm and Streams would we want to use the
> >storm-cassandra driver or should we look into adding the Datastax driver
> >which has better CQL support?
> >Danny
> CONFIDENTIALITY NOTICE: This e-mail, along with any documents, files, or attachments,
may contain information that is confidential, privileged, or otherwise exempt from disclosure.
If you are not the intended recipient or person responsible for delivering it to the intended
recipient, you are hereby notified that any disclosure, copying, printing, distribution or
use of any information contained in or attached to this e-mail is strictly prohibited. If
you have received this e-mail in error, please immediately notify the sender and delete the
original e-mail and its attachments without reading, printing, or saving in any manner. This
e-mail message should not be interpreted to include a digital or electronic signature that
can be used to authenticate an agreement, contract or other legal document, nor to reflect
an intention to be bound to any legally-binding agreement or contract. Your cooperation is
appreciated. Thank you.
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message