From user-return-16384-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Tue May 3 17:33:16 2011 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 21FA7242D for ; Tue, 3 May 2011 17:33:16 +0000 (UTC) Received: (qmail 12607 invoked by uid 500); 3 May 2011 17:33:13 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 12582 invoked by uid 500); 3 May 2011 17:33:13 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 12574 invoked by uid 99); 3 May 2011 17:33:13 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 May 2011 17:33:13 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [213.199.154.204] (HELO AM1EHSOBE001.bigfish.com) (213.199.154.204) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 May 2011 17:33:04 +0000 Received: from mail21-am1-R.bigfish.com (10.3.201.244) by AM1EHSOBE001.bigfish.com (10.3.204.21) with Microsoft SMTP Server id 14.1.225.22; Tue, 3 May 2011 17:32:42 +0000 Received: from mail21-am1 (localhost.localdomain [127.0.0.1]) by mail21-am1-R.bigfish.com (Postfix) with ESMTP id B055A18584B4 for ; Tue, 3 May 2011 17:32:42 +0000 (UTC) X-SpamScore: -3 X-BigFish: VPS-3(zzbb2cKzz1202hzzz2dh2a8h668h839h65h) X-Spam-TCS-SCL: 4:0 X-Forefront-Antispam-Report: CIP:165.215.94.2;KIP:(null);UIP:(null);IPVD:NLI;H:AAPQHTCAS03.proque.st;RD:none;EFVD:NLI Received: from mail21-am1 (localhost.localdomain [127.0.0.1]) by mail21-am1 (MessageSwitch) id 1304443955102010_2580; Tue, 3 May 2011 17:32:35 +0000 (UTC) Received: from AM1EHSMHS002.bigfish.com (unknown [10.3.201.242]) by mail21-am1.bigfish.com (Postfix) with ESMTP id D30FF1BF814C for ; Tue, 3 May 2011 17:32:06 +0000 (UTC) Received: from AAPQHTCAS03.proque.st (165.215.94.2) by AM1EHSMHS002.bigfish.com (10.3.207.102) with Microsoft SMTP Server (TLS) id 14.1.225.22; Tue, 3 May 2011 17:32:01 +0000 Received: from AAPQMAILBX02V.proque.st ([fe80::a5a3:d005:a27c:9314]) by AAPQHTCAS03.proque.st ([fe80::f190:d64b:f609:755b%10]) with mapi; Tue, 3 May 2011 13:32:00 -0400 From: "Serediuk, Adam" To: "user@cassandra.apache.org" Date: Tue, 3 May 2011 13:31:55 -0400 Subject: Range Slice Issue Thread-Topic: Range Slice Issue Thread-Index: AcwJuAGcZZ6h9DzlQuqwW+ANZRShJQ== Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_B1785B99D41E4728BD0653233BF6AC3Dserialssolutionscom_" MIME-Version: 1.0 X-OriginatorOrg: serialssolutions.com X-Virus-Checked: Checked by ClamAV on apache.org --_000_B1785B99D41E4728BD0653233BF6AC3Dserialssolutionscom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable We appear to have encountered an issue with cassandra 0.7.5 after upgrading= from 0.7.2. While doing a batch read using a get_range_slice against the r= anges an individual node is master for we are able to reproduce consistentl= y that the last two nodes in the ring, regardless of the ring size (we have= a 60 node production cluster and a 12 node test cluster) perform this read= over the network using replicas of executing locally. Every other node in = the ring successfully reads locally. To be sure there were no data consistency issues we performed a nodetool re= pair against both of these nodes and the issue persists. We also tried trun= cating the column family and repopulating, but the issue remains. This seems to be related to CASSANDRA-2286 in 0.7.4. We always want to read= data locally if it is available there. We use Cassandra.Client.describe_ri= ng() to figure out which machine in the ring is master for which TokenRange= . I then compare the master for each TokenRange against the localhost to fi= nd out which token ranges are owned by the local machine (remote reads are = too slow for this type of batch processing). Once I know which TokenRanges = are on each machine locally I get evenly sized splits using Cassandra.Clien= t.describe_splits(). Adam --_000_B1785B99D41E4728BD0653233BF6AC3Dserialssolutionscom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
We appear to have enc= ountered an issue with cassandra 0.7.5 after upgrading from 0.7.2. While do= ing a batch read using a get_range_slice against the ranges an individual n= ode is master for we are able to reproduce consistently that the last two n= odes in the ring, regardless of the ring size (we have a 60 node production= cluster and a 12 node test cluster) perform this read over the network usi= ng replicas of executing locally. Every other node in the ring successfully= reads locally.

To be sure there were no data cons= istency issues we performed a nodetool repair against both of these nodes a= nd the issue persists. We also tried truncating the column family and repop= ulating, but the issue remains.

This seems to be r= elated to CASSANDRA-2286 in 0.7.4. We always want to read data locally= if it is available there. We use Cassandra.Client.describe_ring() to = figure out which machine in the ring is master for which TokenRange. I= then compare the master for each TokenRange against the localhost&nbs= p;to find out which token ranges are owned by the local machine (remot= e reads are too slow for this type of batch processing). Once I know w= hich TokenRanges are on each machine locally I get evenly sized s= plits using Cassandra.Client.describe_splits().

Adam

=
=
<= /span>
= --_000_B1785B99D41E4728BD0653233BF6AC3Dserialssolutionscom_--