From user-return-29257-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Mon Oct 1 23:34:32 2012 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B9A5FDF71 for ; Mon, 1 Oct 2012 22:58:34 +0000 (UTC) Received: (qmail 69207 invoked by uid 500); 1 Oct 2012 22:58:32 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 69179 invoked by uid 500); 1 Oct 2012 22:58:32 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 69168 invoked by uid 99); 1 Oct 2012 22:58:32 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Oct 2012 22:58:32 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of abarua@247-inc.com designates 216.32.181.184 as permitted sender) Received: from [216.32.181.184] (HELO ch1outboundpool.messaging.microsoft.com) (216.32.181.184) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Oct 2012 22:58:23 +0000 Received: from mail51-ch1-R.bigfish.com (10.43.68.242) by CH1EHSOBE020.bigfish.com (10.43.70.77) with Microsoft SMTP Server id 14.1.225.23; Mon, 1 Oct 2012 22:58:02 +0000 Received: from mail51-ch1 (localhost [127.0.0.1]) by mail51-ch1-R.bigfish.com (Postfix) with ESMTP id 693C73E011C for ; Mon, 1 Oct 2012 22:58:02 +0000 (UTC) X-Forefront-Antispam-Report: CIP:111.221.112.165;KIP:(null);UIP:(null);IPV:NLI;H:HKXPRD0310HT003.apcprd03.prod.outlook.com;RD:none;EFVD:NLI X-SpamScore: -1 X-BigFish: PS-1(zzc85fh4015Izz1202h1d2ahzz17326ah8275bh8275dhz2fh2a8h668h839hd25hf0ah107ah1288h12a5h12bdh137ah1155h) Received-SPF: pass (mail51-ch1: domain of 247-inc.com designates 111.221.112.165 as permitted sender) client-ip=111.221.112.165; envelope-from=abarua@247-inc.com; helo=HKXPRD0310HT003.apcprd03.prod.outlook.com ;.outlook.com ; Received: from mail51-ch1 (localhost.localdomain [127.0.0.1]) by mail51-ch1 (MessageSwitch) id 134913228122649_22285; Mon, 1 Oct 2012 22:58:01 +0000 (UTC) Received: from CH1EHSMHS007.bigfish.com (snatpool1.int.messaging.microsoft.com [10.43.68.252]) by mail51-ch1.bigfish.com (Postfix) with ESMTP id EC811320049 for ; Mon, 1 Oct 2012 22:58:00 +0000 (UTC) Received: from HKXPRD0310HT003.apcprd03.prod.outlook.com (111.221.112.165) by CH1EHSMHS007.bigfish.com (10.43.70.7) with Microsoft SMTP Server (TLS) id 14.1.225.23; Mon, 1 Oct 2012 22:58:00 +0000 Received: from HKXPRD0310MB352.apcprd03.prod.outlook.com ([169.254.5.195]) by HKXPRD0310HT003.apcprd03.prod.outlook.com ([10.255.6.166]) with mapi id 14.16.0207.009; Mon, 1 Oct 2012 22:57:36 +0000 From: Arindam Barua To: "user@cassandra.apache.org" Subject: Read latency issue Thread-Topic: Read latency issue Thread-Index: Ac2gKCHMVYbGv8qjQa2qNHhi1FgFCA== Date: Mon, 1 Oct 2012 22:57:35 +0000 Message-ID: <17C39FE466076C46B6E83F129C7B19CE16B6D515@HKXPRD0310MB352.apcprd03.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [98.234.253.65] Content-Type: multipart/alternative; boundary="_000_17C39FE466076C46B6E83F129C7B19CE16B6D515HKXPRD0310MB352_" MIME-Version: 1.0 X-OriginatorOrg: 247-inc.com X-Virus-Checked: Checked by ClamAV on apache.org --_000_17C39FE466076C46B6E83F129C7B19CE16B6D515HKXPRD0310MB352_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable We are trying to setup a Cassandra cluster and have low read latency requir= ements. Running some tests, we do not see the performance that we were hopi= ng for. Wanted to check if anyone has thoughts on: 1. If these are expected latency times for the data/machine config, e= tc 2. If not, can do something to improve our read times? We set up 4 boxes as a ring running Cassandra 1.1.5, and setup a keyspace w= ith replication 3, and strategy_class SimpleStrategy. The column family bei= ng tested has 12 columns, 4 of which form a composite key. We then wrote in 192,000 randomly generated test data rows into the column = family. Most columns are either randomly generated UUIDs, or short strings.= One of them however is a blob consisting of around 1K data (we later reduc= ed the size of this blob data, but didn't seem to change our read times muc= h) Running a query to like "select * from where atag=3D", wh= ere 'atag' is the first column of the composite key, from either JDBC or He= ctor (equivalent code), results in read times of 200-300ms from a remote ho= st on the same network. The query returned around 800 results. Running the = same query on a Cassandra host results in a read time of ~110-130 ms. Using read consistency of ONE reduces the read latency by ~20ms, compared t= o using QUORUM. Enabling row cache did not seem to change the performance much. Moreover, t= he row cache 'size' according to nodetool was very tiny. Here is a snapshot= of the nodetool info after running few read tests: Key Cache : size 2448 (bytes), capacity 104857584 (bytes), 231 hits,= 266 requests, 1.000 recent hit rate, 14400 save period in seconds Row Cache : size 96 (bytes), capacity 4194304000 (bytes), 9 hits, 13= requests, NaN recent hit rate, 0 save period in seconds Hardware/OS specs: Intel(R) Xeon(R) CPU L5640 OS: Solaris 5.10 RAM: 32 GB Hard disk: 1 TB disk magnetic (not SSD) Thanks, Arindam --_000_17C39FE466076C46B6E83F129C7B19CE16B6D515HKXPRD0310MB352_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

 

We are trying to setup a Cassandra cluster and have = low read latency requirements. Running some tests, we do not see the perfor= mance that we were hoping for. Wanted to check if anyone has thoughts on:

1.     &= nbsp; If these are expected latency times for the data/ma= chine config, etc

2.     &= nbsp; If not, can do something to improve our read times?=

 

We set up 4 boxes as a ring running Cassandra 1.1.5,= and setup a keyspace with replication 3, and strategy_class SimpleStrategy= . The column family being tested has 12 columns, 4 of which form a composit= e key.

We then wrote in 192,000 randomly generated test dat= a rows into the column family. Most columns are either randomly generated U= UIDs, or short strings. One of them however is a blob consisting of around = 1K data (we later reduced the size of this blob data, but didn’t seem to change our read times much)

 

Running a query to like “select * from <tab= le_name> where atag=3D<foo>”, where ‘atag’ is th= e first column of the composite key, from either JDBC or Hector (equivalent= code), results in read times of 200-300ms from a remote host on the same network. The query returned around 800 results. Running the same query on = a Cassandra host results in a read time of ~110-130 ms.

Using read consistency of ONE reduces the read laten= cy by ~20ms, compared to using QUORUM.

 

Enabling row cache did not seem to change the perfor= mance much. Moreover, the row cache ‘size’ according to nodetoo= l was very tiny. Here is a snapshot of the nodetool info after running few = read tests:

Key Cache        = : size 2448 (bytes), capacity 104857584 (bytes), 231 hits, 266 requests, 1.= 000 recent hit rate, 14400 save period in seconds

Row Cache        = : size 96 (bytes), capacity 4194304000 (bytes), 9 hits, 13 requests, NaN re= cent hit rate, 0 save period in seconds

 

Hardware/OS specs:

Intel(R) Xeon(R) CPU L5640

OS: Solaris 5.10

RAM: 32 GB

Hard disk: 1 TB disk magnetic (not SSD)

 

Thanks,

Arindam

--_000_17C39FE466076C46B6E83F129C7B19CE16B6D515HKXPRD0310MB352_--