Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3EB0EF579 for ; Wed, 27 Mar 2013 00:15:28 +0000 (UTC) Received: (qmail 41150 invoked by uid 500); 27 Mar 2013 00:15:25 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 41129 invoked by uid 500); 27 Mar 2013 00:15:25 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 41119 invoked by uid 99); 27 Mar 2013 00:15:25 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Mar 2013 00:15:25 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [192.174.58.133] (HELO XEDGEB.nrel.gov) (192.174.58.133) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Mar 2013 00:15:19 +0000 Received: from XHUBB.nrel.gov (10.20.4.59) by XEDGEB.nrel.gov (192.174.58.133) with Microsoft SMTP Server (TLS) id 8.3.245.1; Tue, 26 Mar 2013 18:14:53 -0600 Received: from MAILBOX2.nrel.gov ([fe80::19a0:6c19:6421:12f]) by XHUBB.nrel.gov ([::1]) with mapi; Tue, 26 Mar 2013 18:14:55 -0600 From: "Hiller, Dean" To: "user@cassandra.apache.org" Date: Tue, 26 Mar 2013 18:15:00 -0600 Subject: Re: Insert v/s Update performance Thread-Topic: Insert v/s Update performance Thread-Index: Ac4qgBwJSJh1o5meSR6zAmx7L2ie9A== Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.3.2.130206 acceptlanguage: en-US Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org LCS is generally used for high read vs. write ratio though it sounds like y= ou may be doing a heavy write load instead. LCS will involve more compacti= ons as you write to the system compared to STCS because LCS is always tryin= g to keep a 1 to 10 ratio between levels. While LCS will involve more comp= action in general(more I/o, more cpu), I am not sure on update vs. insert t= hough From what I understand STCS will happily duplicate rows across SS tab= les while LCS does not like to do this so as you update you will constantly= compact=85.well, that is my understanding. Have you tried STCS out at all= ? (ps. This is just from what I understand so take with a grain of salt). Also, there are some great tools in the nodetool tool as well so you can ge= t nodetool compactionstats, etc. etc. and see how backlogged you are in pen= ding tasks=85.how many pending? Later, Dean From: Jay Svc > Reply-To: "user@cassandra.apache.org" > Date: Tuesday, March 26, 2013 6:08 PM To: "user@cassandra.apache.org" > Subject: Re: Insert v/s Update performance Thanks Dean, I have used LCS with sstable_size_in_mb of 15. I have also tried bigger sst= able_size_in_mb and observed simillar behavior. Does compaction works differently for update v/s Insert? I belive all keys = goes to single SST. What other options I should look into? Thanks, Jay On Tue, Mar 26, 2013 at 6:18 PM, Hiller, Dean > wrote: Most likely compaction kicks in as updates cause duplicated rows in STCS an= d compaction causes load that may not have been there before(check your log= s). Also, you can increase the number of nodes in your cluster as well to = better handle the load. Later, Dean From: Jay Svc >> Reply-To: "user@cassandra.apache.org>" >> Date: Tuesday, March 26, 2013 5:05 PM To: "user@cassandra.apache.org>" >> Subject: Insert v/s Update performance Hi Team, I have this 3 node cluster. I am writing data to these node at the rate of = 2,000 records/second. What I observed that if I do inserts. (Means records = for those keys does not exist, my column family has 0 records to start with= ) then I have better write performacne, low SSTable count, low pending comp= action and write latency is acceptable and CPU utilization on each node bet= ween 35% to 85%. When I ran same test but for update this time (means records already exists= in Column family with same key), I observed that my SSTable count gone hig= h 3 times. Pending compactions gone high more than 2 times and write latenc= y has gone high too and CPU utilization was almost 92% to 100%. What is a reason of deteriorating Update performance v/s Insert performance= . Since this is critical you help is highly appriciated. P.S. I also observed that high number of pending Mutation Stage on my nodet= ool tpstats. Thanks, Jay