incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "thunder stumpges" <thunder.stump...@gmail.com>
Subject Re: Queuing System
Date Sat, 22 Feb 2014 17:31:56 GMT
This seems a bit overkill. We run far more than 100mps (closer to 600) in rabbit with very
good latency on a 3 node cluster. It has been very reliable as well. 
Thunder

----- Reply message -----
From: "Jagan Ranganathan" <jagan@zohocorp.com>
To: <user@cassandra.apache.org>
Subject: Queuing System
Date: Sat, Feb 22, 2014 9:06 AM

Thanks Tupshin for your assistance. As I mentioned in the other mail, Yes I am planning to
use RabbitMQ for my messaging system. But I wonder which will give better performance if writing
directly into Rabbit with Ack support Vs a temporary Queue in Cassandra first and then dequeue
and publish in Rabbit.I use Rabbit for Messaging because of the Routing and Push model communication
etc. So I am thinking of using Cassandra as a temporary Queue which will give fast write performance
with no data loss Vs waiting for Rabbit Ack @ application level or handling Rabbit re-connection
Vs Cassandra hinted handoff writes.

So Cassandra might aggregate all my msg queue temporarily before I publish them to Rabbit.
Is this fine? If so, please share your insight on which model & access pattern will be
a better fit for this usage. Throughput requirements may be around say 100 ops/sec.

Regards,
Jagan

---- On Sat, 22 Feb 2014 21:10:36 +0530 Tupshin Harper<tupshin@tupshin.com> wrote ----


While, historically, it has been true that queuing in Cassandra     has been an anti-pattern,
it is also true that Leveled Compaction     addresses the worst aspect of frequent deletes
in Cassandra, and     that overall, queuing in Cassandra is nowhere near the anti-pattern
    that it used to be. This is something that I've been meaning to     write about more extensively.
      
If your requirements are more around availability       (particularly multi-dc) and relability
with moderate (not extreme)       performance, it is quite possible to build a pretty decent
system       on top of Cassandra. You don't mention your throughput       requirements, nor
additional semantics that might be necessary       (e.g. deliver at-least-once vs deliver
exactly once), but       Cassandra 2.0's lightweight transactions provide a CAS primitive
      that can be used to ensure deliver-once if that is a requirement.

I'd be happy to continue discussing appropriate data-models and       access patterns if you
decide to go down this path.

-Tupshin


On Sat, Feb 22, 2014 at 10:03 AM, Jagan Ranganathan             <jagan@zohocorp.com>
wrote:
Hi,                 
I need to decouple some of the work being processed                   from the user thread
to provide better user                   experience. For that I need a queuing system with
the                   following needs,
High Availability                                            No Data Loss                
                           Better Performance.
Following are some libraries that were considered                   along with the limitation
I see,
Redis - Data Loss                                            ZooKeeper - Not             
           advised for Queue system.                                            TokyoCabinet/SQLite/LevelDB
                        - of this Level DB seem to be performing better.                 
       With replication requirement, I probably have to                         look at Apache
ActiveMQ+LevelDB.
After checking on the third option above, I kind of                   wonder if Cassandra
with Leveled Compaction offer a                   similar system. Do you see any issues in
such a usage                   or is there other better solutions available.

Will be great to get insights on this.

Regards,
Jagan
Mime
View raw message