Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B2A9E456A for ; Thu, 23 Jun 2011 23:15:42 +0000 (UTC) Received: (qmail 91240 invoked by uid 500); 23 Jun 2011 23:15:40 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 91058 invoked by uid 500); 23 Jun 2011 23:15:40 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 91050 invoked by uid 99); 23 Jun 2011 23:15:40 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Jun 2011 23:15:40 +0000 X-ASF-Spam-Status: No, hits=4.4 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HK_RANDOM_ENVFROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of teddyyyy123@gmail.com designates 209.85.213.172 as permitted sender) Received: from [209.85.213.172] (HELO mail-yx0-f172.google.com) (209.85.213.172) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Jun 2011 23:15:35 +0000 Received: by yxp4 with SMTP id 4so858823yxp.31 for ; Thu, 23 Jun 2011 16:15:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=7Rwi15TIY+ULPy6b5CrIshygrI8WVIV2e1p17wThsQY=; b=fQCj3C3ie8r8MlBGnI+rTclMTo1LWEebva6ic4mGjXSld3MHT6IZXSIijCcn1eN6Ko kYHTdKsWX2TzPHXIvFQ7itYtxQkGis6nXod0MaAglsMUVgaXkBBIimyWNvFmIZ0ZGyGW 5d2hYbfcCOaTSgJG0W2ZrlGJCzTWjT3X4va9Y= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=CYtgn3oDWa9oS1UqXPoHpPh3IMchHLqf2XYPSoeZO7YaFUaJUrC8FumWr+UISEmLE0 YojJlnCtu21UI/LsmJ6fzdUyfRlBQQG2jsYXIs+BvcM9wItQQmqFiy4C72PDZjsIJv4v k/FNjjlLy6iZr+Xnl4fDI7WNqBo9rxJOg069g= MIME-Version: 1.0 Received: by 10.236.175.38 with SMTP id y26mr4296100yhl.304.1308870914839; Thu, 23 Jun 2011 16:15:14 -0700 (PDT) Received: by 10.236.61.3 with HTTP; Thu, 23 Jun 2011 16:15:14 -0700 (PDT) In-Reply-To: References: Date: Thu, 23 Jun 2011 16:15:14 -0700 Message-ID: Subject: Re: simple get_slice() gives error? From: Yang To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=20cf305e2885ef939f04a6694158 --20cf305e2885ef939f04a6694158 Content-Type: text/plain; charset=ISO-8859-1 Thanks Jonathan, I found more cases where I wrongly use getInt() directly. it caused some random data errors: nio.BufferUnderflow. but I did rewind() these buffers before reading them. then the most probable cause is that some other threads are sharing the same buffers. then this prompts me to another question: in current cassandra daemon, is it possible that a column (or more specifically its bytebuffer) is still being held by the read thread , while it was partially written by a write thread? this way the read thread could possibly get garbled result. Thanks Yang On Fri, Jun 10, 2011 at 7:09 PM, Jonathan Ellis wrote: > Don't use destructive operations on the bytebuffer, always use e.g. > getLong(buffer.position) > > On Fri, Jun 10, 2011 at 8:47 PM, Yang wrote: > > I'm using thrift.CassandraServer directly within the same cassandra JVM > to > > accomplish my application tasks. > > (I understand that this is not the normal usage mode...., but the error > here > > may also be appearing in Cassandra server code > > development, so I thought it could be of some value to look at ) > > I ran into some issues when I try to parse out values from column name > > ByteBuffer: > > > > something like the following > > List cols = cassandra_svr.get_slice(key, path, > > predicate, ConsistencyLevel.ONE); > > for(ColumnOrSuperColumn colOrSup: cols) { > > Column col = colOrSup.column; > > long ts = col.name.getLong(); <====== > > > > sometimes the last sentence gives error "ByteBuffer underflow", I checked > > that the buffer size is 8 bytes, and the "pos" points to 8 already. it > > seems that somewhere in > > thrift.CassandraServer line 179---195 > > public List thriftifyColumns(Collection > > columns, boolean reverseOrder) > > { > > ........................................................... > > { > > Column thrift_column = new > > Column(column.name > ()).setValue(column.value()).setTimestamp(column.timestamp()); > > the ctor of Column(Column ) is used, which just uses the old column.name > > buffer, which also preserves the pos. > > > > is this some error on server code? (looks a flip() is needed somewhere) > > > > Thanks > > Yang > > > > -- > Jonathan Ellis > Project Chair, Apache Cassandra > co-founder of DataStax, the source for professional Cassandra support > http://www.datastax.com > --20cf305e2885ef939f04a6694158 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Thanks Jonathan,

I found more cases where I wrongly use = getInt() directly.

it caused some random data erro= rs: nio.BufferUnderflow. =A0but I did rewind() these buffers before reading= them. then the most probable cause is that some other threads are sharing = the same buffers.

then this prompts me to another question: in current ca= ssandra daemon, is it possible that a column (or more specifically its byte= buffer) is still being held by the read
thread , while it was par= tially written by a write thread? this way the read thread could possibly g= et garbled result.

Thanks
Yang

On Fri, Jun 10, 2011 at 7:09 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
Don't use destructive operations on the bytebuffer, always use e.g.
getLong(buffer.position)

On Fri, Jun 10, 2011 at 8:47 PM, Yang <teddyyyy123@gmail.com> wrote:
> I'm using thrift.CassandraServer directly within the same cassandr= a JVM to
> accomplish my application tasks.
> (I understand that this is not the normal usage mode...., but the erro= r here
> may also be appearing in Cassandra server code
> development, so I thought it could be of some value to look at )
> I ran into some issues when I try to parse out values from column name=
> ByteBuffer:
>
> something like the following
> List<ColumnOrSuperColumn> cols =3D cassandra_svr.get_slice(key, = path,
> predicate, ConsistencyLevel.ONE);
> for(ColumnOrSuperColumn colOrSup: cols) {
> Column col =3D colOrSup.column;
> long ts =3D col.name.getLong(); =A0<=3D=3D=3D=3D=3D=3D
>
> sometimes the last sentence gives error "ByteBuffer underflow&quo= t;, I checked
> that the buffer size is 8 bytes, and the "pos" points to 8 a= lready. =A0it
> seems that somewhere in
> thrift.CassandraServer line 179---195
> =A0 =A0 public List<ColumnOrSuperColumn> thriftifyColumns(Collec= tion<IColumn>
> columns, boolean reverseOrder)
> =A0 =A0 {
> ...........................................................
> =A0 =A0 =A0 =A0 =A0 =A0 {
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Column thrift_column =3D new
> Column(column.name()).setValue(column.value()).setTimestamp(column.timestamp());
> the ctor of Column(Column ) is used, which just uses the old column.name
> buffer, which also preserves the pos.
>
> is this some error on server code? (looks a flip() is needed somewhere= )
>
> Thanks
> Yang



--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.c= om

--20cf305e2885ef939f04a6694158--