Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5758210A3C for ; Sat, 22 Feb 2014 17:18:50 +0000 (UTC) Received: (qmail 44434 invoked by uid 500); 22 Feb 2014 17:18:47 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 44370 invoked by uid 500); 22 Feb 2014 17:18:47 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 44361 invoked by uid 99); 22 Feb 2014 17:18:46 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 22 Feb 2014 17:18:46 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of doanduyhai@gmail.com designates 209.85.214.175 as permitted sender) Received: from [209.85.214.175] (HELO mail-ob0-f175.google.com) (209.85.214.175) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 22 Feb 2014 17:18:40 +0000 Received: by mail-ob0-f175.google.com with SMTP id va2so2439955obc.34 for ; Sat, 22 Feb 2014 09:18:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Aa0l/7jrL/E+h0Vfo+VuaTnsMIysLYkrDdCR0tE8hB8=; b=qBhmY6mACaRBFdyV0VeS/tIUg2ff1kk0k5cgq1VnnzNkLc+I2nRcyM1aYWxObm0MLq TXa4lZqHg4qZxP6JfnCDcdKqGX0188FT1p0W85+ioEP/GPi6/SqTKjrz/kkfESHDciVP 7uZknJF8sxvAiikw7XeHhUV1kGept3TXNqy8s5sk0M0DRYowsoBFGBQmPDqOt7cqyLRl bNnIsgSZ1LhG26vkg1iBSlJ0yrdMiiTGLieFaXkEycfXDhd25cWq0MCCbpRq7vHozqlL j0vg5innOrtF1dlCqwskvJcWGGRl9mrMZDlhQEZDETmhadEqy6SZzmQmq4TnxVCjUqVd xInw== MIME-Version: 1.0 X-Received: by 10.60.150.134 with SMTP id ui6mr15127165oeb.62.1393089499712; Sat, 22 Feb 2014 09:18:19 -0800 (PST) Received: by 10.76.122.48 with HTTP; Sat, 22 Feb 2014 09:18:19 -0800 (PST) In-Reply-To: <1445a826596.-8542939828757609342.-4273850624862758512@zohocorp.com> References: <1445a1e7345.-1888803717199936016.-6496021625704613558@zohocorp.com> <1445a826596.-8542939828757609342.-4273850624862758512@zohocorp.com> Date: Sat, 22 Feb 2014 18:18:19 +0100 Message-ID: Subject: Re: Queuing System From: DuyHai Doan To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=047d7b5d3602c5537604f301ecf8 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b5d3602c5537604f301ecf8 Content-Type: text/plain; charset=ISO-8859-1 Jagan Few time ago I dealed with a similar queuing design for one customer. *If you never delete messages in the queue*, then it is possible to use wide rows with bucketing and increasing monotonic column name to store messages. CREATE TABLE *read_only_queue *( bucket_number int, insertion_time timeuuid, message text, PRIMARY KEY(bucket_number,insertion_time) ); Let's say that you allow only 100 000 messages per partition (physical row) to avoid too wide rows, then inserting/reading from the table *read_only_queue *is easy; For message producer : 1) Start at bucket_number = 1 2) Insert messages with column name = generated timeUUID with micro-second precision (depending on whether the insertion rate is high or not) 3) If message count = 100 000, increment bucket_number by one and go to 2) For message reader: 1) Start at bucket_number = 1 2) Read messages by slice of *N, *save the *insertion_time *of the last read message 3) Use the saved *insertion_time *to perform next slice query 4) If read messages count = 100 000, increment bucket_number and go to 2). Keep the *insertion_time, *do not reset it since his value is increasing monotonically For multiple and concurrent producers & writers, there is a trick. Let's assume you have *P* concurrent producers and *C* concurrent consumers. Assign a numerical ID for each producer and consumer. First producer ID = 1... last producer ID = *P*. Same for consumers. - re-use the above algorithm - each producer/consumer start at *bucket_number *= his ID - at the end of the row, - next bucket_number = current bucker_number + *P* for producers - next bucket_number = current bucker_number + *C* for consumers The last thing to take care of is compaction configuration to reduce the number of SSTables on disk. If you achieve to get rid of accumulation effects, e.g reading rate is faster than writing rate, the message are likely to be consumed while it's still in memory (in memtable) at server side. In this particular case, you can optimize further by deactivating compaction for the table. Regards Duy Hai On Sat, Feb 22, 2014 at 5:56 PM, Jagan Ranganathan wrote: > Hi, > > Thanks for the pointer. > > Following are some options given there, > > - If you know where your live data begins, hint Cassandra with a start > column, to reduce the scan times and the amount of tombstones to collect. > - A broker will usually have some notion of what's next in the > sequence and thus be able to do much more targeted queries, down to a > single record if the storage strategy were to choose monotonic sequence > numbers. > > We need to do is have some intelligence in using the system and avoid > tombstones either use the pointed Column Name or use proper start column if > slice query is used. > > Is that right or I am missing something here? > > Regards, > Jagan > > ---- On Sat, 22 Feb 2014 20:55:39 +0530 *DuyHai Doan >* wrote ---- > > Jagan > > Queue-like data structures are known to be one of the worst anti > patterns for Cassandra: > http://www.datastax.com/dev/blog/cassandra-anti-patterns-queues-and-queue-like-datasets > > > > On Sat, Feb 22, 2014 at 4:03 PM, Jagan Ranganathan wrote: > > Hi, > > I need to decouple some of the work being processed from the user thread > to provide better user experience. For that I need a queuing system with > the following needs, > > - High Availability > - No Data Loss > - Better Performance. > > Following are some libraries that were considered along with the > limitation I see, > > - Redis - Data Loss > - ZooKeeper - Not advised for Queue system. > - TokyoCabinet/SQLite/LevelDB - of this Level DB seem to be performing > better. With replication requirement, I probably have to look at Apache > ActiveMQ+LevelDB. > > After checking on the third option above, I kind of wonder if Cassandra > with Leveled Compaction offer a similar system. Do you see any issues in > such a usage or is there other better solutions available. > > Will be great to get insights on this. > > Regards, > Jagan > > > > --047d7b5d3602c5537604f301ecf8 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Jagan

Few = time ago I dealed with a similar queuing design for one customer.

<= /div>If you never delete messages in the queue, then it is possible = to use wide rows with bucketing and increasing monotonic column name to sto= re messages.

CREATE TABLE read_only_queue (
   bucke= t_number int,
   insertion_time timeuuid,
 = ;  message text,
   PRIMARY KEY(bucket_number,inser= tion_time)
);

 Let's say that you allow only 100 000 messages per= partition (physical row) to avoid too wide rows, then inserting/reading fr= om the table read_only_queue is easy;

 For me= ssage producer :

   1) Start at bucket_number =3D 1
&= nbsp;  2) Insert messages with column name =3D generated timeUUID with= micro-second precision (depending on whether the insertion rate is high or= not)
   3) If message count =3D 100 000, increment bucket_number by on= e and go to 2)

For message reader:

   1)= Start at bucket_number =3D 1
   2) Read messages by slic= e of N, save the insertion_time of the last read messa= ge
   3) Use the saved insertion_time to p= erform next slice query
   4) If read messages count = =3D 100 000, increment bucket_number and go to 2). Keep the insertion_ti= me, do not reset it since his value is increasing monotonically

For multiple and concurrent producers & writers, there i= s a trick. Let's assume you have P concurrent producers and C= concurrent consumers.

  Assign a numerical ID f= or each producer and consumer. First producer ID =3D 1... last producer ID = =3D P. Same for consumers.
 
  - re-use the above algorithm
  - each producer/consumer start at bucket_number =3D h= is ID
  - at the end of the row,
     = ;   - next bucket_number =3D current bucker_number + P for prod= ucers
        - next bucket_number =3D current= bucker_number + C for consumers


The la= st thing to take care of is compaction configuration to reduce the number o= f SSTables on disk.

If you achieve to get rid of accumulation effects, e.g reading rate is= faster than writing rate,  the message are likely to be consumed whil= e it's still in memory (in memtable) at server side. In this particular= case, you can optimize further by deactivating compaction for the table. <= br>
Regards

 Duy Hai
 = ;



 
 
<= /div>


On Sat, Feb 22, 2014 at 5:56 PM, Jagan Ranganathan &l= t;jagan@zohocorp.co= m> wrote:
Hi,
Thanks for the pointer. 

Following are s= ome options given there,
  • If you know where your live data begins, hint Ca= ssandra with a start column, to reduce the scan times and the amount of tom= bstones to collect.
  •  A broker= will usually have some notion of what’s next in the sequence and thu= s be able to do much more targeted queries, down to a single record if the = storage strategy were to choose monotonic sequence numbers.
We need to do is have some i= ntelligence in using the system and avoid tombstones either use the pointed= Column Name or use proper start column if slice query is used.

Is that right or I am missing something here?

Regards,
Jagan

---- On= Sat, 22 Feb 2014 20:55:39 +0530 DuyHai Doan<doanduyhai@gmail.com> wrote --= --

Jagan

 = ;Queue-like data structures are known to be one of the worst anti p= atterns for Cassandra:  http://www.datastax.com/dev/blog/cassandra-anti-patterns-= queues-and-queue-like-datasets



On Sat= , Feb 22, 2014 at 4:03 PM, Jagan Ranganathan <jagan@zohocorp.com> wrote= :
Hi,
=
I need to decouple some of the = work being processed from the user thread to provide better= user experience. For that I need a queuing system with the= following needs,
  • = High Availability
    • =
  • No Da= ta Loss
  • = Better Performance.
Following are= some libraries that were considered along with the limitat= ion I see,
  • = Redis - Data Loss
    • =
  • ZooKe= eper - Not advised for Queue system.
  • = TokyoCabinet/SQLite/LevelDB - of this Level DB seem t= o be performing better. With replication requirement,= I probably have to look at Apache ActiveMQ+LevelDB.<= /span>
After checking on the third option above, I = kind of wonder if Cassandra with Leveled Compaction offer a= similar system. Do you see any issues in such a usage or = is there other better solutions available.

Will be = great to get insights on this.

<= div>Regards,
Jagan


=

--047d7b5d3602c5537604f301ecf8--