Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C7A1CF489 for ; Wed, 27 Mar 2013 15:15:58 +0000 (UTC) Received: (qmail 6399 invoked by uid 500); 27 Mar 2013 15:15:56 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 6379 invoked by uid 500); 27 Mar 2013 15:15:56 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 6371 invoked by uid 99); 27 Mar 2013 15:15:56 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Mar 2013 15:15:56 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of ravikumar.govindarajan@gmail.com designates 209.85.214.51 as permitted sender) Received: from [209.85.214.51] (HELO mail-bk0-f51.google.com) (209.85.214.51) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Mar 2013 15:15:51 +0000 Received: by mail-bk0-f51.google.com with SMTP id y8so1663531bkt.10 for ; Wed, 27 Mar 2013 08:15:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=2NMSF0tE1guawBcchT9MHLlRT2Z5ayPrp+Bv5h+7RZE=; b=SowlPbj3Pmr/1+5Eq4XSxuANmUX24jLJK/wJSpIkWCbmd3uxlSQqevUzHQfxuzPcB3 1p473KwvTcuV1w4GvDPQ+ynXA+TNk3v/kCVRTpTMfkNqYGHOw555hcJdY+owk2h6cTGo MsYBrHB4cOBQpt5Os9tdGXt39xzLOlGzOq3NxiEqz41AlhvKtCkbeuK91wx9omN2jgco AK/E8EVO316LXgAlA1dNyWx4CPdEeJOy+ZMPxjjZrfxBwQRDwnU07IGbkendLYwU0xvn nPfoKXShametfKIEkWV9pDSMwtmO8y4XvnIWem0FwgmAz5K5x3bcyzNpdQgnZ9D7fbC8 Z94w== MIME-Version: 1.0 X-Received: by 10.205.74.136 with SMTP id yw8mr1806193bkb.129.1364397329853; Wed, 27 Mar 2013 08:15:29 -0700 (PDT) Received: by 10.204.50.131 with HTTP; Wed, 27 Mar 2013 08:15:29 -0700 (PDT) Date: Wed, 27 Mar 2013 20:45:29 +0530 Message-ID: Subject: Digest Query Seems to be corrupt on certain cases From: Ravikumar Govindarajan To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=f46d040f9c162de80d04d8e982c7 X-Virus-Checked: Checked by ClamAV on apache.org --f46d040f9c162de80d04d8e982c7 Content-Type: text/plain; charset=ISO-8859-1 We started receiving OOMs in our cassandra grid and took a heap dump. We are running version 1.0.7 with LOCAL_QUORUM from both reads/writes. After some analysis, we kind of identified the problem, with SliceByNamesReadCommand, involving a single Super-Column. This seems to be happening only in digest query and not during actual reads. I am pasting the serialized byte array of SliceByNamesReadCommand, which seems to be corrupt on issuing certain digest queries. //Type is SliceByNamesReadCommand body[0] = (byte)1; //This is a digest query here. body[1] = (byte)1; //Table-Name from 2-8 bytes //Key-Name from 9-18 bytes //QueryPath deserialization here //CF-Name from 19-30 bytes //Super-Col-Name from 31st byte onwards, but gets corrupt as found in heap dump //body[32-37] = 0, body[38] = 1, body[39] = 0. This causes the SliceByNamesDeserializer to mark both ColName=NULL and SuperColName=NULL, fetching entire wide-row!!! //Actual super-col-name starts only from byte 40, whereas it should have started from 31st byte itself Has someone already encountered such an issue? Why is the super-col-name not correctly de-serialized during digest query. -- Ravi --f46d040f9c162de80d04d8e982c7 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable We started receiving OOMs in our cassandra grid and took a heap dump. We ar= e running version 1.0.7 with LOCAL_QUORUM from both reads/writes.

<= /div>
After some analysis, we kind of identified the problem, with Slic= eByNamesReadCommand, involving a single Super-Column. This seems to be happ= ening only in digest query and not during actual reads.

I am pasting the serialized byte array of SliceByNamesR= eadCommand, which seems to be corrupt on issuing certain digest queries.

//Type is SliceByNamesReadCommand
body= [0] =3D (byte)1;
//This is a digest query here.
body= [1] =3D (byte)1;

=A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 //Table-Name from 2-8 bytes

=A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 //Key-Name from 9-18 bytes

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 //QueryPath deserializa= tion here
=A0 =A0 =A0 =A0 =A0 =A0 =A0
=A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0//CF-Name from 19-30 bytes

<= div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 //Super-Col-Name from 31st byte= onwards, but gets corrupt as found in heap dump

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 //body[32-37] = =3D 0, body[38] =3D 1, body[39] =3D 0. =A0This causes the SliceByNamesDeser= ializer to mark both ColName=3DNULL and SuperColName=3DNULL, fetching=A0ent= ire wide-row!!!

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0//Actual super-col-name starts = only from byte 40, whereas it should have started from 31st byte itself

Has someone already encountered such an issue? Why is= the super-col-name not correctly de-serialized during digest query.

--
Ravi

--f46d040f9c162de80d04d8e982c7--