incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Stein <crypt...@gmail.com>
Subject Re: Queuing System
Date Sat, 22 Feb 2014 16:32:27 GMT
If performance and availability for messaging is a requirement then use Apache Kafka http://kafka.apache.org/

You can pass the same thrift/avro objects through the Kafka commit log or strings or whatever
you want.

/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop
********************************************/


On Feb 22, 2014, at 11:13 AM, Jagan Ranganathan <jagan@zohocorp.com> wrote:

> Hi Michael,
> 
> Yes I am planning to use RabbitMQ for my messaging system. But I wonder which will give
better performance if writing directly into Rabbit with Ack support Vs a temporary Queue in
Cassandra first and then dequeue and publish in Rabbit.
> 
> Complexities involving - Handling scenarios like Rabbit Connection failure etc Vs Cassandra
write performance and replication with hinted handoff support etc, makes me wonder if this
is a better path.
> 
> Regards,
> Jagan
> 
> ---- On Sat, 22 Feb 2014 21:01:14 +0530 Michael Laing <michael.laing@nytimes.com>
wrote ---- 
> 
> We use RabbitMQ for queuing and Cassandra for persistence.
> 
> RabbitMQ with clustering and/or federation should meet your high availability needs.
> 
> Michael
> 
> 
> On Sat, Feb 22, 2014 at 10:25 AM, DuyHai Doan <doanduyhai@gmail.com> wrote:
> Jagan 
> 
>  Queue-like data structures are known to be one of the worst anti               patterns
for Cassandra:  http://www.datastax.com/dev/blog/cassandra-anti-patterns-queues-and-queue-like-datasets
> 
> 
> 
> On Sat, Feb 22, 2014 at 4:03 PM, Jagan Ranganathan <jagan@zohocorp.com> wrote:
> Hi,
> 
> I need to decouple some of the work being processed from the user thread to provide better
user experience. For that I need a queuing system with the following needs,
> High Availability
> No Data Loss
> Better Performance.
> Following are some libraries that were considered along with the limitation I see,
> Redis - Data Loss
> ZooKeeper - Not advised for Queue system.
> TokyoCabinet/SQLite/LevelDB - of this Level DB seem to be performing better. With replication
requirement, I probably have to look at Apache ActiveMQ+LevelDB.
> After checking on the third option above, I kind of wonder if Cassandra with Leveled
Compaction offer a similar system. Do you see any issues in such a usage or is there other
                          better solutions available.
> 
> Will be great to get insights on this.
> 
> Regards,
> Jagan
> 
> 
> 

Mime
View raw message