Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5DC0910729 for ; Fri, 30 Aug 2013 08:02:15 +0000 (UTC) Received: (qmail 89955 invoked by uid 500); 30 Aug 2013 08:02:11 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 89873 invoked by uid 500); 30 Aug 2013 08:02:11 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 89860 invoked by uid 99); 30 Aug 2013 08:02:09 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 30 Aug 2013 08:02:09 +0000 X-ASF-Spam-Status: No, hits=3.1 required=5.0 tests=HTML_MESSAGE,PLING_QUERY,RCVD_IN_DNSWL_LOW,SPF_PASS,URI_HEX X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of mianmarjun.mailinglist@gmail.com designates 209.85.215.54 as permitted sender) Received: from [209.85.215.54] (HELO mail-la0-f54.google.com) (209.85.215.54) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 30 Aug 2013 08:02:03 +0000 Received: by mail-la0-f54.google.com with SMTP id ea20so1269628lab.13 for ; Fri, 30 Aug 2013 01:01:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=2pKgvMD+o+MO7Ncpi1ScwAclXYE9g0TjI8+4RzAJgUg=; b=GvgTGxu8vj09ythE6MVYvdOzZLKveJiyLqrOsaDd7ROScCiJ59XffHDPDc8M1mJYDR bBn2sUH2f8idN4xO39cUlePCybUGobarY44XoaAJkZiW25KYTg0qbIajxVNffuOa6OS4 uydVAJ1R/h6PY+OOzZFcR1xeADZPkVIGG6VG6U3ggbHL+0E4teSCUiT+34zEzfoJlPNn cKFaILorjF0Wcryj4KgySbSuIp97KTOe6FiuPkvrx+taZ9CnQFkZwhl/60aOpIuREGDs K6gwexvpTcLxBNQAJ85VpElAxFhOLze3dIfBumYWl01lSjR+ZZtajjfxsk5T9TZHMwRk L2dQ== MIME-Version: 1.0 X-Received: by 10.152.2.4 with SMTP id 4mr6824343laq.0.1377849702948; Fri, 30 Aug 2013 01:01:42 -0700 (PDT) Received: by 10.114.232.12 with HTTP; Fri, 30 Aug 2013 01:01:42 -0700 (PDT) In-Reply-To: References: Date: Fri, 30 Aug 2013 10:01:42 +0200 Message-ID: Subject: Re: how can i get the column value? Need help!.. cassandra 1.28 and pig 0.11.1 From: Miguel Angel Martin junquera To: user@pig.apache.org, user@cassandra.apache.org Content-Type: multipart/alternative; boundary=089e0112c51c19230304e525a2e5 X-Virus-Checked: Checked by ClamAV on apache.org --089e0112c51c19230304e525a2e5 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I try this: *rows =3D LOAD 'cql://keyspace1/test?page_size=3D1&split_size=3D4&where_clause=3Dage%3D30'= USING CqlStorage();* *dump rows;* *ILLUSTRATE rows;* *describe rows;* * * *values2=3D FOREACH rows GENERATE TOTUPLE (id) as (mycolumn:tuple(name,value));* *dump values2;* *describe values2;* * * But I get this results: ------------------------------------------------------------- | rows | id:chararray | age:int | title:chararray | ------------------------------------------------------------- | | (id, 6) | (age, 30) | (title, QA) | ------------------------------------------------------------- rows: {id: chararray,age: int,title: chararray} 2013-08-30 09:54:37,831 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1031: Incompatable field schema: left is "tuple_0:tuple(mycolumn:tuple(name:bytearray,value:bytearray))", right is "org.apache.pig.builtin.totuple_id_1:tuple(id:chararray)" or .... *values2=3D FOREACH rows GENERATE TOTUPLE (id) ;* *dump values2;* *describe values2;* and the results are: ... (((id,6))) (((id,5))) values2: {org.apache.pig.builtin.totuple_id_8: (id: chararray)} Aggg!!!!! * * Miguel Angel Mart=EDn Junquera Analyst Engineer. miguelangel.martin@brainsins.com 2013/8/28 Miguel Angel Martin junquera > hi: > > I can not understand why the schema is define like *"id:chararray,age:in= t,title:chararray" > and it does not define like tuples or bag tuples, if we have pair > key-values columns* > * > * > * > * > *I try other time to change schema but it does not work.* > * > * > *any ideas ...* > * > * > *perhaps, is the issue in the definition cql3 tables ?* > * > * > *regards* > > > 2013/8/28 Miguel Angel Martin junquera > >> hi all: >> >> >> Regards >> >> Still i can resolve this issue. ..... >> >> does anybody have this issue or try to test this simple example? >> >> >> i am stumped I can not find a solution working. >> >> I appreciate any comment or help >> >> >> 2013/8/22 Miguel Angel Martin junquera >> >>> hi all: >>> >>> >>> >>> >>> I,m testing the new CqlStorage() with cassandra 1.28 and pig 0.11.1 >>> >>> >>> I am using this sample data test: >>> >>> >>> http://frommyworkshop.blogspot.com.es/2013/07/hadoop-map-reduce-with-ca= ssandra.html >>> >>> And I load and dump data Righ with this script: >>> >>> *rows =3D LOAD >>> 'cql://keyspace1/test?page_size=3D1&split_size=3D4&where_clause=3Dage%3= D30' USING >>> CqlStorage();* >>> * >>> * >>> *dump rows;* >>> *describe rows;* >>> * >>> * >>> >>> *resutls: >>> >>> ((id,6),(age,30),(title,QA)) >>> >>> ((id,5),(age,30),(title,QA)) >>> >>> rows: {id: chararray,age: int,title: chararray} >>> >>> >>> * >>> >>> >>> But i can not get the column values >>> >>> I try to define another schemas in Load like I used with >>> cassandraStorage() >>> >>> >>> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassan= dra-and-Pig-how-to-get-column-values-td5641158.html >>> >>> >>> example: >>> >>> *rows =3D LOAD >>> 'cql://keyspace1/test?page_size=3D1&split_size=3D4&where_clause=3Dage%3= D30' USING >>> CqlStorage() AS (columns: bag {T: tuple(name, value)});* >>> >>> >>> and I get this error: >>> >>> *2013-08-22 12:24:45,426 [main] ERROR org.apache.pig.tools.grunt.Grunt >>> - ERROR 1031: Incompatable schema: left is >>> "columns:bag{T:tuple(name:bytearray,value:bytearray)}", right is >>> "id:chararray,age:int,title:chararray"* >>> >>> >>> >>> >>> I try to use, FLATTEN, SUBSTRING, SPLIT UDF`s but i have not get good >>> result: >>> >>> Example: >>> >>> >>> - when I flatten , I get a set of tuples like >>> >>> *(title,QA)* >>> >>> *(title,QA)* >>> >>> *2013-08-22 12:42:20,673 [main] INFO >>> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total >>> input paths to process : 1* >>> >>> *A: {title: chararray}* >>> >>> >>> >>> but i can get value QA >>> >>> Sustring only works with title >>> >>> >>> >>> example: >>> >>> *B =3D FOREACH A GENERATE SUBSTRING(title,2,5);* >>> * >>> * >>> *dump B;* >>> *describe B;* >>> * >>> * >>> * >>> * >>> >>> *results:* >>> * >>> * >>> >>> *(tle)* >>> *(tle)* >>> *B: {chararray}* >>> >>> >>> >>> >>> i try, this like ERIC LEE inthe other mail and have the same results: >>> >>> >>> Anyways, what I really what is the column value, not the name. Is ther= e >>> a way to do that? I listed all of the failed attempts I made below. >>> >>> - colnames =3D FOREACH cols GENERATE $1 and was told $1 was out of >>> bounds. >>> - casted =3D FOREACH cols GENERATE (tuple(chararray, chararray))$0; >>> but all I got back were empty tuples >>> - values =3D FOREACH cols GENERATE $0.$1; but I got an error telling >>> me data byte array can't be casted to tuple >>> >>> >>> Please, I will appreciate any help >>> >>> >>> Regards >>> >>> >>> >>> >>> >>> >>> >> >> >> -- >> >> Miguel Angel Mart=EDn Junquera >> Analyst Engineer. >> miguelangel.martin@brainsins.com >> Tel. / Fax: (+34) 91 485 56 66 >> *http://www.brainsins.com* >> Smart eCommerce >> *Madrid*: http://goo.gl/4B5kv >> *London*: http://goo.gl/uIXdv >> *Barcelona*: http://goo.gl/NZslW >> >> Antes de imprimir este e-mail, piense si es necesario. >> La legislaci=F3n espa=F1ola ampara el secreto de las comunicaciones. Est= e >> correo electr=F3nico es estrictamente confidencial y va dirigido >> exclusivamente a su destinatario/a. Si no es Ud., le rogamos que no difu= nda >> ni copie la transmisi=F3n y nos lo notifique cuanto antes. >> > > > > -- > > Miguel Angel Mart=EDn Junquera > Analyst Engineer. > miguelangel.martin@brainsins.com > > --089e0112c51c19230304e525a2e5 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
I try this:

rows =3D LOAD 'cql://keyspace1/test?page_s= ize=3D1&split_size=3D4&where_clause=3Dage%3D30' USING CqlStorag= e();
dump rows;
= ILLUSTRATE rows;
describe rows;

values2=3D FOREA= CH rows GENERATE =A0TOTUPLE (id) as (mycolumn:tuple(name,value));
dump values2;
describe values2;

But I get this results:


=

--------------------------------------------= -----------------
| rows =A0 =A0 | id:chararray =A0 | age:int =A0 | title:chararray =A0 = |=A0
------------------------------------------------------------= -
| =A0 =A0 =A0 =A0 =A0| (id, 6) =A0 =A0 =A0 =A0| (age, 30) | (ti= tle, QA) =A0 =A0 =A0 |=A0
-------------------------------------------------------------
rows: {id: chararray,age: int,title: chararray}
2013-= 08-30 09:54:37,831 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 10= 31: Incompatable field schema: left is "tuple_0:tuple(mycolumn:tuple(n= ame:bytearray,value:bytearray))", right is "org.apache.pig.builti= n.totuple_id_1:tuple(id:chararray)"





=
or=A0



....

values2=3D FOREACH rows GENERATE =A0TOTUPLE (id) ;
dump values2;
describe values2;


and =A0the results are:
=

...
(((id,6)))
(((id= ,5)))
values2: {org.apache.pig.builtin.totuple_id_8: (id: chararr= ay)}



Aggg!!!!!






Miguel An= gel Mart=EDn Junquera
Analyst Engineer.



2013/8/28 Miguel Angel Martin junquera <= span dir=3D"ltr"><mianmarjun.mailinglist@gmail.com>
hi:

I can not understand why the schema= is =A0define like=A0"id:chararray,age:int,title:chararray" =A0and it does not defi= ne like tuples or bag tuples, =A0if we have pair key-values =A0columns<= /div>


I try other= time to change schema =A0but it does not work.

any ideas ..= .
per= haps, is the issue in the definition cql3 tables ?

regards

2013/8/28 Miguel Angel Martin junquera <mianmarjun.mailinglist@gmail.com>
hi all:

=
Regards

Still i can resolve this is= sue. .....

does anybody have this issue or try to test this simple exam= ple?


i am stumped I can not find a solution w= orking.=A0

I appreciate any comment or help


2013/8/22 Miguel Angel Martin junquera <mianmarjun.mailingl= ist@gmail.com>
hi all:

=


I,m testing the new CqlStorage= () with cassandra 1.28 and pig 0.11.1=A0


I am using this sample data test:


And I load and dump data Righ with this script:

rows =3D LOAD 'cql://keyspace1/test?page_= size=3D1&split_size=3D4&where_clause=3Dage%3D30' USING CqlStora= ge();

dump rows;
describe rows;
=
resutls:

((id,6),(age,30),(title,QA))
((id,5),(age,30),(titl= e,QA))
rows:= {id: chararray,age: int,title: chararray}

=

But i can not =A0get =A0the column values=A0
=
I try to define =A0 another schemas in Load like I used with= cassandraStorage()



example:

=
rows =3D LOAD 'cql://keyspace1/test?page_size=3D1&split_siz= e=3D4&where_clause=3Dage%3D30' USING CqlStorage() AS (columns: bag = {T: tuple(name, value)});


and I get this error:
2013-08-22 12:24:45,426 [main] ERROR org.apache.pig.tools.g= runt.Grunt - ERROR 1031: Incompatable schema: left is "columns:bag{T:t= uple(name:bytearray,value:bytearray)}", right is "id:chararray,ag= e:int,title:chararray"



I try = to use, FLATTEN, SUBSTRING, SPLIT UDF`s but i have not get good result:

Example:

  • when I flatten , I get a set of tuples like
(title,QA)
(title,QA)=
2013-08-22 12:42:20,673 [main] INFO =A0org.= apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input pat= hs to process : 1
A: {title: chararray}


but i can get = value QA=A0

Sustring only works with title



example:

=
B =3D FOREACH A GENERATE SUBSTRING(title,2,5);
=

dump B;
describe B;


results:<= /b>

(tle)
(tle)
B: {chararray}



i try, this like ERIC LEE inthe other mail =A0and h= ave the same results:


=A0Anyways, what I really what is the co= lumn value, not the name. Is there a way to do that? I listed all of the fa= iled attempts I made below.
  • colnames =3D FOREACH cols GENERATE $1 and was told $1 was out of bounds= .=A0
  • casted =3D FOREACH cols GENERATE (tuple(chararray, chararray))= $0; but all I got back were empty tuples
  • values =3D=A0FOREACH cols = GENERATE $0.$1; but I got an error telling me data byte array can't be = casted to tuple

Please,=A0I will appreciate any help=A0

=

Regards









--
=

Miguel An= gel Mart=EDn Junquera
Analyst Engineer.
Tel. / Fax: <= /span>(+34= ) 91 485 56 66
Smart eCommerce
Madrid:=A0http://goo.gl/4B5kv
Lon= don:=A0http://goo.gl/uIXdv
Barcelona:=A0http://goo.gl/NZslW

Antes de imprimir este e-mail, piense si = es necesario.
La legislaci=F3n espa=F1ola = ampara el secreto de las comunicaciones. Este correo electr=F3nico es estri= ctamente confidencial y va dirigido exclusivamente a su destinatario/a. Si = no es Ud., le rogamos que no difunda ni copie la transmisi=F3n y nos lo not= ifique cuanto antes.



--

Miguel An= gel Mart=EDn Junquera
Analyst Engineer.


--089e0112c51c19230304e525a2e5--