From user-return-36818-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Thu Sep 26 05:00:52 2013 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D293C10588 for ; Thu, 26 Sep 2013 05:00:52 +0000 (UTC) Received: (qmail 97812 invoked by uid 500); 26 Sep 2013 05:00:49 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 97792 invoked by uid 500); 26 Sep 2013 05:00:48 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 97782 invoked by uid 99); 26 Sep 2013 05:00:48 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 26 Sep 2013 05:00:48 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy includes SPF record at spf.trusted-forwarder.org) Received: from [209.85.160.51] (HELO mail-pb0-f51.google.com) (209.85.160.51) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 26 Sep 2013 05:00:41 +0000 Received: by mail-pb0-f51.google.com with SMTP id jt11so608063pbb.24 for ; Wed, 25 Sep 2013 22:00:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:content-type:message-id:mime-version :subject:date:references:to:in-reply-to; bh=hrAwPbmgI3N4l5MKmzX/3y08s2UsKsL9sqh1L+eSjj0=; b=Dw45anAVXccPZN04eaxHTVD8wYFJdU2z0mXHOxpUjetND5tYGSataz/CO8veI/I7Zb ZbPFWNL2FfRn9x2XUK2J6fukqgU60iy4gJvFxYAGs3j7+lWkHXxBQHokNkuBCg4YNqVj nx7sA2WJnFzbcFftMuZWoi9fb4xscWtiCgbGTovOZ0FHaTUqDiG/WXSZD6se7o+LFMK0 YzO0BDbq6U3dQxUxzjOh0IOMgSd+1mgHP7KbQaZY8E2n2vueUV2yFOc7UmXcQ0IqU5gW j1ZLgU+w3Vuguq5GQtZrVhbXyWoeNH5IwHCvyOWrXt30TiBhGs3gvgmVwzuLVlWsItbJ ynug== X-Gm-Message-State: ALoCoQlUMWjtdifvQFEBXFVVj+sKkT8i/qkrabgFWS/quZvRbIi6MIOnjpZr0tEPT0aOSmwyS9zK X-Received: by 10.68.212.137 with SMTP id nk9mr22064483pbc.26.1380171620293; Wed, 25 Sep 2013 22:00:20 -0700 (PDT) Received: from [172.16.1.18] ([203.86.207.101]) by mx.google.com with ESMTPSA id fi4sm45659432pbc.28.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 25 Sep 2013 22:00:19 -0700 (PDT) From: Aaron Morton Content-Type: multipart/alternative; boundary="Apple-Mail=_CDB11079-CC53-438D-9274-D5148080CC3B" Message-Id: Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: Mystery PIG issue with 1.2.10 Date: Thu, 26 Sep 2013 17:00:16 +1200 References: <42416CA8-BF74-45B7-9DA3-419A5C19F13A@ecyrd.com> To: user@cassandra.apache.org In-Reply-To: X-Mailer: Apple Mail (2.1510) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_CDB11079-CC53-438D-9274-D5148080CC3B Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 > > (testc =3D LOAD 'cassandra://keyspace/testc' USING = CassandraStorage(); > > dump testc > foo,{(ivalue, > ),(svalue,bar),(value,A)}) If the CQL 3 data ye wish to read, CqlStorage be the driver of your = success.=20 (btw there is a ticket out to update the example if you get excited = https://issues.apache.org/jira/browse/CASSANDRA-5709) Cheers ----------------- Aaron Morton New Zealand @aaronmorton Co-Founder & Principal Consultant Apache Cassandra Consulting http://www.thelastpickle.com On 26/09/2013, at 3:57 AM, Chad Johnston wrote: > As an FYI, creating the table without the "WITH COMPACT STORAGE" and = using CqlStorage works just fine in 1.2.10. >=20 > I know that CqlStorage and AbstractCassandraStorage got changed for = 1.2.10 - maybe there's a regression with the existing CassandraStorage? >=20 > Chad >=20 >=20 > On Wed, Sep 25, 2013 at 1:51 AM, Janne Jalkanen = wrote: > Heya! >=20 > I am seeing something rather strange in the way Cass 1.2 + Pig seem to = handle integer values. >=20 > Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single = node for testing this. >=20 > First a table: >=20 > > CREATE TABLE testc ( > key text PRIMARY KEY, > ivalue int, > svalue text, > value bigint > ) WITH COMPACT STORAGE; >=20 > > insert into testc (key,ivalue,svalue,value) values = ('foo',10,'bar',65); > > select * from testc; >=20 > key | ivalue | svalue | value > -----+--------+--------+------- > foo | 10 | bar | 65 >=20 > For my Pig setup, I then use libraries from different C* versions to = actually talk to my database (which stays on 1.2.10 all the time). >=20 > Cassandra 1.0.12 (using cassandra_storage.jar): >=20 > > testc =3D LOAD 'cassandra://keyspace/testc' USING = CassandraStorage(); > > dump testc > (foo,(svalue,bar),(ivalue,10),(value,65),{}) >=20 > Cassandra 1.1.10: >=20 > > testc =3D LOAD 'cassandra://keyspace/testc' USING = CassandraStorage(); > > dump testc > (foo,(svalue,bar),(ivalue,10),(value,65),{}) >=20 > Cassandra 1.2.10: >=20 > > (testc =3D LOAD 'cassandra://keyspace/testc' USING = CassandraStorage(); > > dump testc > foo,{(ivalue, > ),(svalue,bar),(value,A)}) >=20 >=20 > To me it appears that ints and bigints are interpreted as ascii values = in cass 1.2.10. Did something change for CassandraStorage, is there a = regression, or am I doing something wrong? Quick perusal of the JIRA = didn't reveal anything that I could directly pin on this. >=20 > Note that using compact storage does not seem to affect the issue, = though it obviously changes the resulting pig format. >=20 > In addition, trying to use Pygmalion >=20 > > tf =3D foreach testc generate key, = flatten(FromCassandraBag('ivalue,svalue,value',columns)) as = (ivalue:int,svalue:chararray,lvalue:long); > > dump tf >=20 > (foo, > ,bar,A) >=20 > So no help there. Explicitly casting the values to (long) or (int) = just results in a ClassCastException. >=20 > /Janne >=20 --Apple-Mail=_CDB11079-CC53-438D-9274-D5148080CC3B Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1
> (testc =3D LOAD 'cassandra://keyspace/testc' = USING CassandraStorage();
> dump = testc
foo,{(ivalue,
),(svalue,bar),(value,A)})

If the CQL 3 data ye wish to read, CqlStorage be the driver = of your success. 

(btw there is a ticket out to = update the example if you get excited https://issu= es.apache.org/jira/browse/CASSANDRA-5709)

Che= ers


http://www.thelastpickle.com

On 26/09/2013, at 3:57 AM, Chad Johnston <cjohnston@megatome.com> = wrote:

As an FYI, creating the table without the = "WITH COMPACT STORAGE" and using CqlStorage works just fine in = 1.2.10.

I know that CqlStorage and = AbstractCassandraStorage got changed for 1.2.10 - maybe there's a = regression with the existing CassandraStorage?

Chad


On Wed, Sep 25, = 2013 at 1:51 AM, Janne Jalkanen <Janne.Jalkanen@ecyrd.com> wrote:
Heya!

I am seeing something rather strange in the way Cass 1.2 + Pig seem to = handle integer values.

Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1.  Single = node for testing this.

First a table:

> CREATE TABLE testc (
  key text PRIMARY KEY,
  ivalue int,
  svalue text,
  value bigint
) WITH COMPACT STORAGE;

> insert into testc (key,ivalue,svalue,value) values = ('foo',10,'bar',65);
> select * from testc;

 key | ivalue | svalue | value
-----+--------+--------+-------
 foo |     10 |    bar |     65

For my Pig setup, I then use libraries from different C* versions to = actually talk to my database (which stays on 1.2.10 all the time).

Cassandra 1.0.12 (using cassandra_storage.jar):

> testc =3D LOAD 'cassandra://keyspace/testc' = USING CassandraStorage();
> dump testc
(foo,(svalue,bar),(ivalue,10),(value,65),{})

Cassandra 1.1.10:

> testc =3D LOAD 'cassandra://keyspace/testc' = USING CassandraStorage();
> dump testc
(foo,(svalue,bar),(ivalue,10),(value,65),{})

Cassandra 1.2.10:

> (testc =3D LOAD 'cassandra://keyspace/testc' = USING CassandraStorage();
> dump testc
foo,{(ivalue,
),(svalue,bar),(value,A)})


To me it appears that ints and bigints are interpreted as ascii values = in cass 1.2.10.  Did something change for CassandraStorage, is = there a regression, or am I doing something wrong?  Quick perusal = of the JIRA didn't reveal anything that I could directly pin on = this.

Note that using compact storage does not seem to affect the issue, = though it obviously changes the resulting pig format.

In addition, trying to use Pygmalion

> tf =3D foreach testc generate key, = flatten(FromCassandraBag('ivalue,svalue,value',columns)) as = (ivalue:int,svalue:chararray,lvalue:long);
> dump tf

(foo,
,bar,A)

So no help there. Explicitly casting the values to (long) or (int) just = results in a ClassCastException.

/Janne


= --Apple-Mail=_CDB11079-CC53-438D-9274-D5148080CC3B--