Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 83749 invoked from network); 1 Dec 2010 23:26:41 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 1 Dec 2010 23:26:41 -0000 Received: (qmail 87254 invoked by uid 500); 1 Dec 2010 23:26:39 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 87222 invoked by uid 500); 1 Dec 2010 23:26:39 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 87214 invoked by uid 99); 1 Dec 2010 23:26:39 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Dec 2010 23:26:39 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.216.179] (HELO mail-qy0-f179.google.com) (209.85.216.179) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Dec 2010 23:26:34 +0000 Received: by qyk11 with SMTP id 11so8056428qyk.10 for ; Wed, 01 Dec 2010 15:26:12 -0800 (PST) MIME-Version: 1.0 Received: by 10.229.85.210 with SMTP id p18mr7324032qcl.263.1291245971588; Wed, 01 Dec 2010 15:26:11 -0800 (PST) Received: by 10.229.0.202 with HTTP; Wed, 1 Dec 2010 15:26:11 -0800 (PST) In-Reply-To: <54867049-55a7-cdd7-c294-b45263b3904d@me.com> References: <54867049-55a7-cdd7-c294-b45263b3904d@me.com> Date: Wed, 1 Dec 2010 16:26:11 -0700 Message-ID: Subject: Re: Range Queries in RP on SCF in 0.7 with UUID SCs From: Frank LoVecchio To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=0016364ef3fc744674049661a160 --0016364ef3fc744674049661a160 Content-Type: text/plain; charset=ISO-8859-1 Actually, it was a class issue at this line: System.*out*.println("NAME: " + UUID.*nameUUIDFromBytes*(col.getName())); The native Pelops class timeUuidHelper is what should be used. On Wed, Dec 1, 2010 at 4:16 PM, Aaron Morton wrote: > When you say "I want to get rows starting from a Super Column..." it's a > bit confusing. Do you want to get super columns from a single row, or > multiple rows? I'm assuming you are talking about getting columns from a > single row / key as that's what your code does. > > For the pelops code, it looks OK but I've not used Pelops. You can turn the > logging up on the server and check the command that is sent to it. I'm would > guess there is something wrong with the way you are transforming the start > key > > For your cli example what was the command you executed ? > > Aaron > > On 02 Dec, 2010,at 11:03 AM, Frank LoVecchio wrote: > > Hey Aaron, > > > Yes, in regards to SCF definition, you are correct: > > > name: Sensor > > column_type: Super > > compare_with: TimeUUIDType > > gc_grace_seconds: 864000 > > keys_cached: 10000.0 > > read_repair_chance: 1.0 > > rows_cached: 0.0 > > I'm not quite sure I follow you, though, as I think I'm doing what you > specify. The Pelops code is below. Basically, I want to get rows > starting from a Super Column with a specific UUID and limit the number, just > as you inferred. When I run this code I just get the last N values (25 in > this case) if non-reversed, and the first N values if reversed. However, > regardless of what start param we use (Super Column UUID is String startKey > below), we still get the same values for the specified amount (e.g. the same > 25). > > *public* *void* getSuperRowKeys(String rowKey, String columnFamily, *int* limit, > String startKey) *throws* Exception { > > *byte*[] byteArray = UuidHelper.*timeUuidStringToBytes*(startKey); > > ByteBuffer bb = ByteBuffer.*wrap*(byteArray); > > *new* UUID (bb.getLong(), bb.getLong()); > > List cols = selector.getPageOfSuperColumnsFromRow(columnFamily, > rowKey, Bytes.*fromByteBuffer*(bb), *false*, limit, ConsistencyLevel.*ONE* > ); > > > *for* (SuperColumn col : cols) { > > *if* (col.getName() != *null*) { > > > System.*out*.println("NAME: " + UUID.*nameUUIDFromBytes*(col.getName())); > > > *for* (Column c : col.columns) { > > System.*out*.println("\t\tName: " + Bytes.*toUTF8*(c.getName()) > > + " Value: " + Bytes.*toUTF8*(c.getValue()) > > + " timestamp: " + c.timestamp); > > > } > > > } > > } > > > } > > Here is some example data from the CLI. If we specify 2f814d30-f758-11df-2f81-4d30f75811df > as the start param (second super column down), we still get 52e6540-f759-11df-952e-6540f75911df > (first super column) returned. > > => (super_column=952e6540-f759-11df-952e-6540f75911df, > (column=64617465, > value=323031302d31312d32332032333a32393a30332e303030, > timestamp=1290554997141000) > (column=65787472615f696e666f, value=6e6f6e65, > timestamp=1290554997141000) > (column=726561736f6e, value=6e6f6e65, timestamp=1290554997141000) > (column=7365636f6e64735f746f5f6e657874, value=373530, > timestamp=1290554997141000) > (column=73657269616c, value=393135353032353731, > timestamp=1290554997141000) > (column=737461747573, value=5550, timestamp=1290554997141000) > (column=74797065, value=486561727462656174, > timestamp=1290554997141000)) > => (super_column=2f814d30-f758-11df-2f81-4d30f75811df, > (column=64617465, > value=323031302d31312d32332032333a31393a30332e303030, > timestamp=1290554397060000) > (column=65787472615f696e666f, value=6e6f6e65, > timestamp=1290554397060000) > (column=726561736f6e, value=6e6f6e65, timestamp=1290554397060000) > (column=7365636f6e64735f746f5f6e657874, value=373530, > timestamp=1290554397060000) > (column=73657269616c, value=393135353032353731, > timestamp=1290554397060000) > (column=737461747573, value=5550, timestamp=1290554397060000) > (column=74797065, value=486561727462656174, > timestamp=1290554397060000)) > => (super_column=7c959f00-f757-11df-7c95-9f00f75711df, > (column=64617465, > value=323031302d31312d32332032333a31343a30332e303030, > timestamp=1290554096881000) > (column=65787472615f696e666f, value=6e6f6e65, > timestamp=1290554096881000) > (column=726561736f6e, value=6e6f6e65, timestamp=1290554096881000) > (column=7365636f6e64735f746f5f6e657874, value=373530, > timestamp=1290554096881000) > (column=73657269616c, value=393135353032353731, > timestamp=1290554096881000) > (column=737461747573, value=5550, timestamp=1290554096881000) > (column=74797065, value=486561727462656174, > timestamp=1290554096881000)) > => (super_column=c9be6330-f756-11df-c9be-6330f75611df, > (column=64617465, > value=323031302d31312d32332032333a30393a30332e303030, > timestamp=1290553796836000) > (column=65787472615f696e666f, value=6e6f6e65, > timestamp=1290553796836000) > (column=726561736f6e, value=6e6f6e65, timestamp=1290553796836000) > (column=7365636f6e64735f746f5f6e657874, value=373530, > timestamp=1290553796836000) > (column=73657269616c, value=393135353032353731, > timestamp=1290553796836000) > (column=737461747573, value=5550, timestamp=1290553796836000) > (column=74797065, value=486561727462656174, > timestamp=1290553796836000)) > => (super_column=17108150-f756-11df-1710-8150f75611df, > (column=64617465, > value=323031302d31312d32332032333a30343a30332e303030, > timestamp=1290553497067000) > (column=65787472615f696e666f, value=6e6f6e65, > timestamp=1290553497067000) > (column=726561736f6e, value=6e6f6e65, timestamp=1290553497067000) > (column=7365636f6e64735f746f5f6e657874, value=373530, > timestamp=1290553497067000) > (column=73657269616c, value=393135353032353731, > timestamp=1290553497067000) > (column=737461747573, value=5550, timestamp=1290553497067000) > (column=74797065, value=486561727462656174, > timestamp=1290553497067000)) > => (super_column=641da730-f755-11df-641d-a730f75511df, > (column=64617465, > value=323031302d31312d32332032323a35393a30332e303030, > timestamp=1290553196836000) > (column=65787472615f696e666f, value=6e6f6e65, > timestamp=1290553196836000) > (column=726561736f6e, value=6e6f6e65, timestamp=1290553196836000) > (column=7365636f6e64735f746f5f6e657874, value=373530, > timestamp=1290553196836000) > (column=73657269616c, value=393135353032353731, > timestamp=1290553196836000) > (column=737461747573, value=5550, timestamp=1290553196836000) > (column=74797065, value=486561727462656174, > timestamp=1290553196836000)) > => (super_column=4bce9420-f753-11df-4bce-9420f75311df, > (column=64617465, > value=323031302d31312d32332032323a34343a30332e303030, > timestamp=1290552297060000) > (column=65787472615f696e666f, value=6e6f6e65, > timestamp=1290552297060000) > (column=726561736f6e, value=6e6f6e65, timestamp=1290552297060000) > (column=7365636f6e64735f746f5f6e657874, value=373530, > timestamp=1290552297060000) > (column=73657269616c, value=393135353032353731, > timestamp=1290552297060000) > (column=737461747573, value=5550, timestamp=1290552297060000) > (column=74797065, value=486561727462656174, > timestamp=1290552297060000)) > > On Wed, Dec 1, 2010 at 2:25 PM, Aaron Morton wrote: > >> The Partitioner applies to the row keys, not the columns. Their order is >> determined by the compare_with and compare_subcolumns_with CF settings >> >> So where you say "get the last 25 inserts for a key" I'm translating that >> into "get the most recent 25 super columns for a row, where the super column >> names are UUID's and the CF definition has compare_with: TimeUUIDType" >> >> You can send a get_slice where the SliceRange has count=25 and >> reversed=True. If you know the 25th col name you can use it as the start >> param, and AFAIK the operation would go faster Not sure how this translates >> into calls against Pelops. >> >> This assumes you are using time / v1 UUID's that have an increasing >> order. >> >> Hope that helps. >> >> Aaron >> >> >> >> On 02 Dec, 2010,at 09:19 AM, Frank LoVecchio wrote: >> >> Is it possible to perform paginated queries using Random Partitioner in >> 0.7 with Super Column Families whose Super Columns are UUID's? I don't >> believe it is, based on this article: >> http://ria101.wordpress.com/2010/02/22/cassandra-randompartitioner-vs-orderpreservingpartitioner, >> and my attempts with Pelops. >> >> Let's say I get the last 25 inserts for a key. I now know the 25th Super >> Column UUID, and I want to start from there and get the next 25 inserts. Do >> I have to change over to Order Preserving Partitioner? I'd like to avoid >> this at all costs. >> >> Thanks, >> >> Frank >> >> > --0016364ef3fc744674049661a160 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Actually, it was a class issue at this line:=A0

System.out.println("NAME: "=A0+ UUID.nameUUIDFromBytes(col.getName()));

=
The native Pelops class timeUuidHelper is what should = be used.

On Wed, Dec 1, 2010 at 4:16 PM= , Aaron Morton <aaron@thelastpickle.com> wrote:
When you say "I want to get rows starting from a Super Colum= n..." it's a bit confusing. Do you want to get super columns from = a single row, or multiple rows? I'm assuming you are talking about gett= ing columns from a single row / key as that's what your code does.

For the pelops code, it looks OK but I've not used Pelop= s. You can turn the logging up on the server and check the command that is = sent to it. I'm would guess there is something wrong with the way you a= re transforming the start key=A0

For your cli example what was the command you executed = ?=A0

Aaron

On 02 Dec, 2010,at 11:03 AM, Frank = LoVecchio <frank= @isidorey.com> wrote:

Hey Aaron,=A0


Yes, in regards to SCF definition, you are correct: =A0=

name: Sensor

=A0=A0= =A0 =A0column_type: Super

=A0=A0= =A0 =A0compare_with: TimeUUIDType

=A0=A0= =A0 =A0gc_grace_seconds: 864000

=A0=A0= =A0 =A0keys_cached: 10000.0

=A0=A0= =A0 =A0read_repair_chance: 1.0

=A0=A0= =A0 =A0rows_cached: 0.0


Basically, I want to get rows starting from a Super Column with a specif= ic UUID and limit the number, just as you inferred. =A0When I run this code= I just get the last N values (25 in this case) if non-reversed, and the fi= rst N values if reversed. =A0However, regardless of what start param we use= (Super Column UUID is String startKey below), we still get the same values= for the specified amount (e.g. the same 25). =A0

public=A0void=A0getSuperRowKeys(String ro= wKey, String columnFamily,=A0int= =A0limit, String startKey)=A0throws=A0Exception {

byte[] byteArray =3D UuidHelper.timeUuidStringT= oBytes(startKey);

ByteBuffer bb =3D ByteBuffer.= wrap(byteArray);

new=A0UUID (bb.getLong(), bb.getLong());

List<SuperColumn> cols = =3D=A0selector.getPageOfSuperC= olumnsFromRow(columnFamily, rowKey, Bytes.fromByteBuffer(bb),=A0false, limit, Consistency= Level.ONE);


for=A0(SuperColumn col : cols) {

if=A0(col.getName() !=3D=A0null) {


System.out.println("NAME: "=A0+ UUID.nameUUIDFromBytes(col.get= Name()));


for=A0(Column c : col.columns) {

System.out.println("\t\tName: "=A0+ Bytes.toUTF8(c.getName()= )

+=A0" Value: "=A0+ Bytes.toUTF8(c.g= etValue())

+=A0" timestamp: "=A0+ c.timestamp);


}


}

}


}


Here is some example data = from the CLI. =A0If we specify=A02f814d30-f758-1= 1df-2f81-4d30f75811df as the start param (second super column down), we sti= ll get=A052e6540-f759-11df-952e-6540f75911df (first sup= er column) returned.

=3D> (super_column=3D952e6540-f759-11df-952e-6540f75911df,
=A0=A0 =A0 (column=3D64617465, value=3D323031302d31312d32332032333a32393a= 30332e303030, timestamp=3D1290554997141000)
=A0=A0 =A0 (column=3D= 65787472615f696e666f, value=3D6e6f6e65, timestamp=3D1290554997141000)
=A0=A0 =A0 (column=3D726561736f6e, value=3D6e6f6e65, timestamp=3D12905= 54997141000)
=A0=A0 =A0 (column=3D7365636f6e64735f746f5f6e657874,= value=3D373530, timestamp=3D1290554997141000)
=A0=A0 =A0 (column= =3D73657269616c, value=3D393135353032353731, timestamp=3D1290554997141000)<= /div>
=A0=A0 =A0 (column=3D737461747573, value=3D5550, timestamp=3D129055499= 7141000)
=A0=A0 =A0 (column=3D74797065, value=3D48656172746265617= 4, timestamp=3D1290554997141000))
=3D> (super_column=3D2f814d3= 0-f758-11df-2f81-4d30f75811df,
=A0=A0 =A0 (column=3D64617465, value=3D323031302d31312d32332032333a313= 93a30332e303030, timestamp=3D1290554397060000)
=A0=A0 =A0 (column= =3D65787472615f696e666f, value=3D6e6f6e65, timestamp=3D1290554397060000)
=A0=A0 =A0 (column=3D726561736f6e, value=3D6e6f6e65, timestamp=3D12= 90554397060000)
=A0=A0 =A0 (column=3D7365636f6e64735f746f5f6e657874, value=3D373530, t= imestamp=3D1290554397060000)
=A0=A0 =A0 (column=3D73657269616c, v= alue=3D393135353032353731, timestamp=3D1290554397060000)
=A0=A0 = =A0 (column=3D737461747573, value=3D5550, timestamp=3D1290554397060000)
=A0=A0 =A0 (column=3D74797065, value=3D486561727462656174, timestamp= =3D1290554397060000))
=3D> (super_column=3D7c959f00-f757-11df-= 7c95-9f00f75711df,
=A0=A0 =A0 (column=3D64617465, value=3D3230313= 02d31312d32332032333a31343a30332e303030, timestamp=3D1290554096881000)
=A0=A0 =A0 (column=3D65787472615f696e666f, value=3D6e6f6e65, timestamp= =3D1290554096881000)
=A0=A0 =A0 (column=3D726561736f6e, value=3D6= e6f6e65, timestamp=3D1290554096881000)
=A0=A0 =A0 (column=3D73656= 36f6e64735f746f5f6e657874, value=3D373530, timestamp=3D1290554096881000)
=A0=A0 =A0 (column=3D73657269616c, value=3D393135353032353731, timesta= mp=3D1290554096881000)
=A0=A0 =A0 (column=3D737461747573, value= =3D5550, timestamp=3D1290554096881000)
=A0=A0 =A0 (column=3D74797= 065, value=3D486561727462656174, timestamp=3D1290554096881000))
=3D> (super_column=3Dc9be6330-f756-11df-c9be-6330f75611df,
=A0=A0 =A0 (column=3D64617465, value=3D323031302d31312d32332032333a30393a= 30332e303030, timestamp=3D1290553796836000)
=A0=A0 =A0 (column=3D= 65787472615f696e666f, value=3D6e6f6e65, timestamp=3D1290553796836000)
=A0=A0 =A0 (column=3D726561736f6e, value=3D6e6f6e65, timestamp=3D12905= 53796836000)
=A0=A0 =A0 (column=3D7365636f6e64735f746f5f6e657874,= value=3D373530, timestamp=3D1290553796836000)
=A0=A0 =A0 (column= =3D73657269616c, value=3D393135353032353731, timestamp=3D1290553796836000)<= /div>
=A0=A0 =A0 (column=3D737461747573, value=3D5550, timestamp=3D129055379= 6836000)
=A0=A0 =A0 (column=3D74797065, value=3D48656172746265617= 4, timestamp=3D1290553796836000))
=3D> (super_column=3D1710815= 0-f756-11df-1710-8150f75611df,
=A0=A0 =A0 (column=3D64617465, value=3D323031302d31312d32332032333a303= 43a30332e303030, timestamp=3D1290553497067000)
=A0=A0 =A0 (column= =3D65787472615f696e666f, value=3D6e6f6e65, timestamp=3D1290553497067000)
=A0=A0 =A0 (column=3D726561736f6e, value=3D6e6f6e65, timestamp=3D12= 90553497067000)
=A0=A0 =A0 (column=3D7365636f6e64735f746f5f6e657874, value=3D373530, t= imestamp=3D1290553497067000)
=A0=A0 =A0 (column=3D73657269616c, v= alue=3D393135353032353731, timestamp=3D1290553497067000)
=A0=A0 = =A0 (column=3D737461747573, value=3D5550, timestamp=3D1290553497067000)
=A0=A0 =A0 (column=3D74797065, value=3D486561727462656174, timestamp= =3D1290553497067000))
=3D> (super_column=3D641da730-f755-11df-= 641d-a730f75511df,
=A0=A0 =A0 (column=3D64617465, value=3D3230313= 02d31312d32332032323a35393a30332e303030, timestamp=3D1290553196836000)
=A0=A0 =A0 (column=3D65787472615f696e666f, value=3D6e6f6e65, timestamp= =3D1290553196836000)
=A0=A0 =A0 (column=3D726561736f6e, value=3D6= e6f6e65, timestamp=3D1290553196836000)
=A0=A0 =A0 (column=3D73656= 36f6e64735f746f5f6e657874, value=3D373530, timestamp=3D1290553196836000)
=A0=A0 =A0 (column=3D73657269616c, value=3D393135353032353731, timesta= mp=3D1290553196836000)
=A0=A0 =A0 (column=3D737461747573, value= =3D5550, timestamp=3D1290553196836000)
=A0=A0 =A0 (column=3D74797= 065, value=3D486561727462656174, timestamp=3D1290553196836000))
=3D> (super_column=3D4bce9420-f753-11df-4bce-9420f75311df,
=A0=A0 =A0 (column=3D64617465, value=3D323031302d31312d32332032323a34343a= 30332e303030, timestamp=3D1290552297060000)
=A0=A0 =A0 (column=3D= 65787472615f696e666f, value=3D6e6f6e65, timestamp=3D1290552297060000)
=A0=A0 =A0 (column=3D726561736f6e, value=3D6e6f6e65, timestamp=3D12905= 52297060000)
=A0=A0 =A0 (column=3D7365636f6e64735f746f5f6e657874,= value=3D373530, timestamp=3D1290552297060000)
=A0=A0 =A0 (column= =3D73657269616c, value=3D393135353032353731, timestamp=3D1290552297060000)<= /div>
=A0=A0 =A0 (column=3D737461747573, value=3D5550, timestamp=3D129055229= 7060000)
=A0=A0 =A0 (column=3D74797065, value=3D48656172746265617= 4, timestamp=3D1290552297060000))

On Wed, Dec 1, 2010 at 2:25 PM, Aaron Morton <aaron@thelastpickle.c= om> wrote:
The Partitioner applies to the row= keys, not the columns. Their order is determined by the compare_with and c= ompare_subcolumns_with CF settings=A0

So where you say "get the last 25 inserts for a ke= y" I'm translating that into "get the most recent 25 super co= lumns for a row, where the super column names are UUID's and the CF def= inition has compare_with: =A0TimeUUIDType"=A0

You can send a get_slice where the SliceRange has count=3D25= and reversed=3DTrue. If you know the 25th col name you can use it as the s= tart param, and AFAIK the operation would go faster Not sure how this trans= lates into calls against Pelops.=A0

This assumes you are using time / v1 UUID's that ha= ve an increasing order.=A0

Hope that helps.=A0

Aaron
<= br>


On 02 Dec, 2010,at 09:19 AM, Frank LoVecchio &= lt;frank@isidorey.c= om> wrote:

Is it possible to perform paginated queries using Random Partit= ioner in 0.7 with Super Column Families whose Super Columns are UUID's?= =A0I don't believe it is, based on this article:=A0http://ria101.wordpress.com/2010/02/22/cas= sandra-randompartitioner-vs-orderpreservingpartitioner, and my attempts= with Pelops.

Let's say I get the last 25 in= serts for a key. =A0I now know the 25th Super Column UUID, and I want to st= art from there and get the next 25 inserts. =A0Do I have to change over to = Order Preserving Partitioner? =A0I'd like to avoid this at all costs.

Thanks,

Frank


--0016364ef3fc744674049661a160--