incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jagan Ranganathan <>
Subject Re: Queuing System
Date Sat, 22 Feb 2014 17:06:37 GMT
Thanks Tupshin for your assistance. As I mentioned in the other mail, Yes I am planning to
use RabbitMQ for my messaging system. But I wonder which will give better performance if writing
directly into Rabbit with Ack support Vs a temporary Queue in Cassandra first and then dequeue
and publish in Rabbit.I use Rabbit for Messaging because of the Routing and Push model communication
etc. So I am thinking of using Cassandra as a temporary Queue which will give fast write performance
with no data loss Vs waiting for Rabbit Ack @ application level or handling Rabbit re-connection
Vs Cassandra hinted handoff writes.

So Cassandra might aggregate all my msg queue temporarily before I publish them to Rabbit.
Is this fine? If so, please share your insight on which model &amp; access pattern will
be a better fit for this usage. Throughput requirements may be around say 100 ops/sec.


---- On Sat, 22 Feb 2014 21:10:36 +0530 Tupshin Harper&lt;;
wrote ---- 

 While, historically, it has been true that queuing in Cassandra has been an anti-pattern,
it is also true that Leveled Compaction addresses the worst aspect of frequent deletes in
Cassandra, and that overall, queuing in Cassandra is nowhere near the anti-pattern that it
used to be. This is something that I've been meaning to write about more extensively. 

  If your requirements are more around availability (particularly multi-dc) and relability
with moderate (not extreme) performance, it is quite possible to build a pretty decent system
on top of Cassandra. You don't mention your throughput requirements, nor additional semantics
that might be necessary (e.g. deliver at-least-once vs deliver exactly once), but Cassandra
2.0's lightweight transactions provide a CAS primitive that can be used to ensure deliver-once
if that is a requirement.

 I'd be happy to continue discussing appropriate data-models and access patterns if you decide
to go down this path.

  On Sat, Feb 22, 2014 at 10:03 AM, Jagan Ranganathan &lt;; wrote:

  I need to decouple some of the work being processed from the user thread to provide better
user experience. For that I need a queuing system with the following needs,
    High Availability
  No Data Loss
  Better Performance.

 Following are some libraries that were considered along with the limitation I see,
    Redis - Data Loss
  ZooKeeper - Not advised for Queue system.
  TokyoCabinet/SQLite/LevelDB - of this Level DB seem to be performing better. With replication
requirement, I probably have to look at Apache ActiveMQ+LevelDB.

 After checking on the third option above, I kind of wonder if Cassandra with Leveled Compaction
offer a similar system. Do you see any issues in such a usage or is there other better solutions

 Will be great to get insights on this.




View raw message