Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 95F0A10CDC for ; Thu, 19 Sep 2013 20:29:02 +0000 (UTC) Received: (qmail 27222 invoked by uid 500); 19 Sep 2013 20:28:59 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 27011 invoked by uid 500); 19 Sep 2013 20:28:59 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 27003 invoked by uid 99); 19 Sep 2013 20:28:59 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 Sep 2013 20:28:59 +0000 X-ASF-Spam-Status: No, hits=3.8 required=5.0 tests=HTML_MESSAGE,PLING_QUERY,RCVD_IN_DNSWL_NONE,URI_HEX X-Spam-Check-By: apache.org Received-SPF: error (athena.apache.org: local policy) Received: from [178.33.45.10] (HELO mo5.mail-out.ovh.net) (178.33.45.10) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 Sep 2013 20:28:53 +0000 Received: from mail439.ha.ovh.net (b9.ovh.net [213.186.33.59]) by mo5.mail-out.ovh.net (Postfix) with SMTP id 2FB6B102428A for ; Thu, 19 Sep 2013 22:28:09 +0200 (CEST) Received: from b0.ovh.net (HELO queueout) (213.186.33.50) by b0.ovh.net with SMTP; 19 Sep 2013 22:29:00 +0200 Received: from cag06-2-82-237-68-200.fbx.proxad.net (HELO ?192.168.0.42?) (cscetbon@atid.fr@82.237.68.200) by ns0.ovh.net with SMTP; 19 Sep 2013 22:28:59 +0200 Content-Type: multipart/alternative; boundary="Apple-Mail=_3E0DED05-6E7D-40D1-A5AA-8ECB7D8BD8C9" Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) X-Ovh-Mailout: 178.32.228.5 (mo5.mail-out.ovh.net) Subject: Re: how can i get the column value? Need help!.. cassandra 1.28 and pig 0.11.1 From: Cyril Scetbon In-Reply-To: Date: Thu, 19 Sep 2013 22:28:06 +0200 Cc: user@pig.apache.org Message-Id: References: To: user@cassandra.apache.org X-Mailer: Apple Mail (2.1510) X-Ovh-Tracer-Id: 6812538861581748182 X-Ovh-Remote: 82.237.68.200 (cag06-2-82-237-68-200.fbx.proxad.net) X-Ovh-Local: 213.186.33.20 (ns0.ovh.net) X-OVH-SPAMSTATE: OK X-OVH-SPAMSCORE: 49 X-OVH-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeeiledrtddvucetufdoteggodetrfcurfhrohhfihhlvgemucfqggfjnecuuegrihhlohhuthemuceftddtnecuogfuuhhsphgvtghtffhomhgrihhnucdlgeelmd X-Spam-Check: DONE|U 0.5/N X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 49 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeeiledrtddvucetufdoteggodetrfcurfhrohhfihhlvgemucfqggfjnecuuegrihhlohhuthemuceftddtnecuogfuuhhsphgvtghtffhomhgrihhnucdlgeelmd X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_3E0DED05-6E7D-40D1-A5AA-8ECB7D8BD8C9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Hi, Did you try to build 1.2.10 and to use it for your tests ? I've got the = same issue and will give it a try as soon as it's released (expected at = the end of the week). Regards --=20 Cyril SCETBON On Sep 2, 2013, at 3:09 PM, Miguel Angel Martin junquera = wrote: > hi all: >=20 > More info : >=20 > https://issues.apache.org/jira/browse/CASSANDRA-5941 >=20 >=20 >=20 > I tried this (and gen. cassandra 1.2.9) but do not work for me,=20 >=20 > git clone http://git-wip-us.apache.org/repos/asf/cassandra.git > cd cassandra > git checkout cassandra-1.2 > patch -p1 < 5867-bug-fix-filter-push-down-1.2-branch.txt > ant >=20 >=20 >=20 > Miguel Angel Mart=EDn Junquera > Analyst Engineer. > miguelangel.martin@brainsins.com >=20 >=20 >=20 > 2013/9/2 Miguel Angel Martin junquera = > hi: >=20 > I test this in cassandra 1.2.9 new version and the issue still = persists . >=20 > :-( >=20 >=20 >=20 >=20 >=20 >=20 > Miguel Angel Mart=EDn Junquera > Analyst Engineer. > miguelangel.martin@brainsins.com >=20 >=20 >=20 > 2013/8/30 Miguel Angel Martin junquera = > I try this: >=20 > rows =3D LOAD = 'cql://keyspace1/test?page_size=3D1&split_size=3D4&where_clause=3Dage%3D30= ' USING CqlStorage(); > dump rows; > ILLUSTRATE rows; > describe rows; >=20 > values2=3D FOREACH rows GENERATE TOTUPLE (id) as = (mycolumn:tuple(name,value)); > dump values2; > describe values2; >=20 > But I get this results: >=20 >=20 >=20 > ------------------------------------------------------------- > | rows | id:chararray | age:int | title:chararray |=20 > ------------------------------------------------------------- > | | (id, 6) | (age, 30) | (title, QA) |=20 > ------------------------------------------------------------- >=20 > rows: {id: chararray,age: int,title: chararray} > 2013-08-30 09:54:37,831 [main] ERROR org.apache.pig.tools.grunt.Grunt = - ERROR 1031: Incompatable field schema: left is = "tuple_0:tuple(mycolumn:tuple(name:bytearray,value:bytearray))", right = is "org.apache.pig.builtin.totuple_id_1:tuple(id:chararray)" >=20 >=20 >=20 >=20 >=20 > or=20 >=20 >=20 >=20 > .... >=20 > values2=3D FOREACH rows GENERATE TOTUPLE (id) ; > dump values2; > describe values2; >=20 >=20 >=20 > and the results are: >=20 >=20 > ... > (((id,6))) > (((id,5))) > values2: {org.apache.pig.builtin.totuple_id_8: (id: chararray)} >=20 >=20 >=20 > Aggg!!!!! >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > Miguel Angel Mart=EDn Junquera > Analyst Engineer. > miguelangel.martin@brainsins.com >=20 >=20 >=20 > 2013/8/28 Miguel Angel Martin junquera = > hi: >=20 > I can not understand why the schema is define like = "id:chararray,age:int,title:chararray" and it does not define like = tuples or bag tuples, if we have pair key-values columns >=20 >=20 > I try other time to change schema but it does not work. >=20 > any ideas ... >=20 > perhaps, is the issue in the definition cql3 tables ? >=20 > regards >=20 >=20 > 2013/8/28 Miguel Angel Martin junquera = > hi all: >=20 >=20 > Regards >=20 > Still i can resolve this issue. ..... >=20 > does anybody have this issue or try to test this simple example? >=20 >=20 > i am stumped I can not find a solution working.=20 >=20 > I appreciate any comment or help >=20 >=20 > 2013/8/22 Miguel Angel Martin junquera = > hi all: >=20 >=20 >=20 >=20 > I,m testing the new CqlStorage() with cassandra 1.28 and pig 0.11.1=20 >=20 >=20 > I am using this sample data test: >=20 > = http://frommyworkshop.blogspot.com.es/2013/07/hadoop-map-reduce-with-cassa= ndra.html >=20 > And I load and dump data Righ with this script: >=20 > rows =3D LOAD = 'cql://keyspace1/test?page_size=3D1&split_size=3D4&where_clause=3Dage%3D30= ' USING CqlStorage(); >=20 > dump rows; > describe rows; >=20 > resutls: >=20 > ((id,6),(age,30),(title,QA)) > ((id,5),(age,30),(title,QA)) > rows: {id: chararray,age: int,title: chararray} >=20 >=20 > But i can not get the column values=20 >=20 > I try to define another schemas in Load like I used with = cassandraStorage() >=20 > = http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra= -and-Pig-how-to-get-column-values-td5641158.html >=20 >=20 > example: >=20 > rows =3D LOAD = 'cql://keyspace1/test?page_size=3D1&split_size=3D4&where_clause=3Dage%3D30= ' USING CqlStorage() AS (columns: bag {T: tuple(name, value)}); >=20 >=20 > and I get this error: >=20 > 2013-08-22 12:24:45,426 [main] ERROR org.apache.pig.tools.grunt.Grunt = - ERROR 1031: Incompatable schema: left is = "columns:bag{T:tuple(name:bytearray,value:bytearray)}", right is = "id:chararray,age:int,title:chararray" >=20 >=20 >=20 > I try to use, FLATTEN, SUBSTRING, SPLIT UDF`s but i have not get good = result: >=20 > Example: >=20 > when I flatten , I get a set of tuples like > (title,QA) > (title,QA) > 2013-08-22 12:42:20,673 [main] INFO = org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total = input paths to process : 1 > A: {title: chararray} >=20 >=20 > but i can get value QA=20 >=20 > Sustring only works with title >=20 >=20 >=20 > example: >=20 > B =3D FOREACH A GENERATE SUBSTRING(title,2,5); >=20 > dump B; > describe B; >=20 >=20 > results: >=20 > (tle) > (tle) > B: {chararray} >=20 >=20 >=20 > i try, this like ERIC LEE inthe other mail and have the same results: >=20 >=20 > Anyways, what I really what is the column value, not the name. Is = there a way to do that? I listed all of the failed attempts I made = below. > colnames =3D FOREACH cols GENERATE $1 and was told $1 was out of = bounds.=20 > casted =3D FOREACH cols GENERATE (tuple(chararray, chararray))$0; but = all I got back were empty tuples > values =3D FOREACH cols GENERATE $0.$1; but I got an error telling me = data byte array can't be casted to tuple >=20 > Please, I will appreciate any help=20 >=20 >=20 > Regards >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 > --=20 >=20 >=20 >=20 >=20 >=20 >=20 > Miguel Angel Mart=EDn Junquera > Analyst Engineer. > miguelangel.martin@brainsins.com > Tel. / Fax: (+34) 91 485 56 66 > http://www.brainsins.com > Smart eCommerce > Madrid: http://goo.gl/4B5kv > London: http://goo.gl/uIXdv > Barcelona: http://goo.gl/NZslW >=20 > Antes de imprimir este e-mail, piense si es necesario. > La legislaci=F3n espa=F1ola ampara el secreto de las comunicaciones. = Este correo electr=F3nico es estrictamente confidencial y va dirigido = exclusivamente a su destinatario/a. Si no es Ud., le rogamos que no = difunda ni copie la transmisi=F3n y nos lo notifique cuanto antes. >=20 >=20 >=20 > --=20 >=20 >=20 >=20 >=20 >=20 > Miguel Angel Mart=EDn Junquera > Analyst Engineer. > miguelangel.martin@brainsins.com >=20 >=20 >=20 >=20 --Apple-Mail=_3E0DED05-6E7D-40D1-A5AA-8ECB7D8BD8C9 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1
-- 
Cyril = SCETBON

On Sep 2, 2013, at 3:09 PM, Miguel Angel Martin junquera = <mianmarjun.mailinglist@gm= ail.com> wrote:

hi all:

More info = :

https://issu= es.apache.org/jira/browse/CASSANDRA-5941



I tried this (and gen. cassandra 1.2.9) =  but do not work for me, 

git =
clone http://git-w=
ip-us.apache.org/repos/asf/cassandra.git
cd cassandra
git checkout cassandra-1.2
patch -p1 < 5867-bug-fix-filter-push-down-1.2-branch.txt
ant


Miguel Angel Mart=EDn = Junquera
Analyst Engineer.



2013/9/2 Miguel Angel Martin junquera = <mianmarjun.mailinglist@gmail.com>
hi:

I test this in cassandra 1.2.9 = new  version and the issue still persists = .

:-(




Miguel Angel Mart=EDn = Junquera
Analyst Engineer.



2013/8/30 = Miguel Angel Martin junquera <mianmarjun.mailinglist@gmail.com>
I try this:

ILLUSTRATE rows;
describe = rows;

values2=3D FOREACH rows = GENERATE  TOTUPLE (id) as = (mycolumn:tuple(name,value));
dump = values2;
describe = values2;

But I get this = results:



-----------= --------------------------------------------------
| rows     | id:chararray   | age:int   | = title:chararray   = | 
--------------------------------------------------------= -----
|          | (id, 6)   =      | (age, 30) | (title, QA)       = | 
= -------------------------------------------------------------

rows: {id: chararray,age: int,title: = chararray}
2013-08-30 09:54:37,831 [main] ERROR = org.apache.pig.tools.grunt.Grunt - ERROR 1031: Incompatable field = schema: left is = "tuple_0:tuple(mycolumn:tuple(name:bytearray,value:bytearray))", right = is "org.apache.pig.builtin.totuple_id_1:tuple(id:chararray)"
=





or 



....=

values2=3D FOREACH rows GENERATE  TOTUPLE (id) = ;
dump values2;
describe = values2;



and  the results = are:


...
(((id,6)))=
(((id,5)))
values2: = {org.apache.pig.builtin.totuple_id_8: (id: chararray)}
=



Aggg!!!!!


=




Miguel Angel Mart=EDn = Junquera
Analyst Engineer.



2013/8/28 Miguel Angel = Martin junquera <mianmarjun.mailinglist@gmail.com>
hi:

I can not understand why the = schema is  define like "id:chararray,age:in= t,title:chararray"  and it does not define like tuples or bag = tuples,  if we have pair key-values  columns

<= i = style=3D"font-size:13px;font-family:arial,sans-serif">
<= i style=3D"font-size:13px;font-family:arial,sans-serif">I try other time = to change schema  but it does not work.

<= i style=3D"font-size:13px;font-family:arial,sans-serif">any ideas = ...

perhaps, is the = issue in the definition cql3 tables ?

<= i style=3D"font-size:13px;font-family:arial,sans-serif">regards


2013/8/28 Miguel Angel Martin junquera <mianmarjun.mailinglist@gmail.com>
hi = all:


Regards

St= ill i can resolve this issue. .....

does anybody have this issue or try to test this simple = example?


i am stumped I can not find a = solution working. 

I appreciate any = comment or help


2013/8/22 Miguel Angel Martin junquera <mianmarjun.mailinglist@gmail.com>
hi = all:




I,m = testing the new CqlStorage() with cassandra 1.28 and pig = 0.11.1 


I am using this sample data = test:


And I load and dump data Righ with this = script:


dump = rows;
describe = rows;

resutls:

((id,6),(age,30),(title,QA))
((id,5),(age,30),(title,QA))
rows: {id: chararray,age: int,title: = chararray}


But i can not  get  the column = values 

I try to define   another = schemas in Load like I used with = cassandraStorage()



and I get this = error:

2013-08-22 12:24:45,426 [main] = ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1031: Incompatable = schema: left is "columns:bag{T:tuple(name:bytearray,value:bytearray)}", = right is "id:chararray,age:int,title:chararray"



I try to = use, FLATTEN, SUBSTRING, SPLIT UDF`s but i have not get good = result:

Example:

    when I flatten , I get a set of tuples like
(title,QA)
(title,QA)
2013-08-22 12:42:20,673 [main] INFO =  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - = Total input paths to process : 1
A: {title: chararray}


but i can = get value QA 

Sustring only works with = title



example:

B =3D FOREACH A GENERATE = SUBSTRING(title,2,5);

dump = B;
describe B;
=


results:

(tle)
(tle)
B: = {chararray}



<= /div>
i try, this like ERIC LEE inthe other mail  and have the = same results:


 Anyways, what I really what is = the column value, not the name. Is there a way to do that? I listed all = of the failed attempts I made below.
  • colnames =3D FOREACH cols GENERATE $1 and was told $1 was out of = bounds. 
  • casted =3D FOREACH cols GENERATE (tuple(chararray, = chararray))$0; but all I got back were empty tuples
  • values = =3D FOREACH cols GENERATE $0.$1; but I got an error telling me data = byte array can't be casted to tuple

Please, I will appreciate = any help 
<= div>
=
Regards



<= div>





-- =

Miguel Angel Mart=EDn = Junquera
Analyst Engineer.
Tel. / Fax: (+34) 91 485 56 66
Smart = eCommerce
Barcelona: http://goo.gl/NZslW

Antes de imprimir este e-mail, piense si es necesario.
La legislaci=F3n espa=F1ola = ampara el secreto de las comunicaciones. Este correo electr=F3nico es = estrictamente confidencial y va dirigido exclusivamente a su = destinatario/a. Si no es Ud., le rogamos que no difunda ni copie la = transmisi=F3n y nos lo notifique cuanto antes.



--

Miguel Angel Mart=EDn = Junquera
Analyst Engineer.





= --Apple-Mail=_3E0DED05-6E7D-40D1-A5AA-8ECB7D8BD8C9--