Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of
 mianmarjun.mailinglist@gmail.com designates 209.85.217.170 as permitted
 sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAPZMN7HquhuoNVj+hjp67Ckv8c_WOWV0HaaYnnmYyS6SZw-yqA@mail.gmail.com>
References: 
 <CAPZMN7HquhuoNVj+hjp67Ckv8c_WOWV0HaaYnnmYyS6SZw-yqA@mail.gmail.com>
Date: Mon, 26 Aug 2013 10:32:33 +0200
Message-ID: 
 <CAJeG_hSO-Apii=Xt4b+w3+eu_Yq-0y35c8kiL64k1ekLJtZkdg@mail.gmail.com>
Subject: Re: CqlStorage creates wrong schema for Pig
From: Miguel Angel Martin junquera <mianmarjun.mailinglist@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=089e0158cb9206d58f04e4d5995d

--089e0158cb9206d58f04e4d5995d
Content-Type: text/plain; charset=ISO-8859-1

hi Chad .

I have this issue

I send a mail to user-pig-list and  I still i can resolve this, and I can
not  access to column values.
In this mail  I write some things that I try without results... and
information about this issue.


http://mail-archives.apache.org/mod_mbox/pig-user/201308.mbox/%3CCAJeG_hQ9S2Po3_XytZX5Xki4J1maO8q26jYdG2Wndy_KYiv9CQ@mail.gmail.com%3E


I hope  someOne reply  one comment, idea or  solution about  this issue or
bug.


I have reviewed the CqlStorage class in code cassandra 1.2.8  but i do not
have configure the environmetn to debug  and trace this issue.

Only  I find some comments like, but I do not understand at all.


/**

 * A LoadStoreFunc for retrieving data from and storing data to Cassandra

 *

 * A row from a standard CF will be returned as nested tuples:

 * (((key1, value1), (key2, value2)), ((name1, val1), (name2, val2))).
 */


I you found some idea or solution, please post it

thanks


2013/8/23 Chad Johnston <cjohnston@megatome.com>

> (I'm using Cassandra 1.2.8 and Pig 0.11.1)
>
> I'm loading some simple data from Cassandra into Pig using CqlStorage. The
> CqlStorage loader defines a Pig schema based on the Cassandra schema, but
> it seems to be wrong.
>
> If I do:
>
> data = LOAD 'cql://bookdata/books' USING CqlStorage();
> DESCRIBE data;
>
> I get this:
>
> data: {isbn: chararray,bookauthor: chararray,booktitle:
> chararray,publisher: chararray,yearofpublication: int}
>
> However, if I DUMP data, I get results like these:
>
> ((isbn,0425093387),(bookauthor,Georgette Heyer),(booktitle,Death in the
> Stocks),(publisher,Berkley Pub Group),(yearofpublication,1986))
>
> Clearly the results from Cassandra are key/value pairs, as would be
> expected. I don't know why the schema generated by CqlStorage() would be so
> different.
>
> This is really causing me problems trying to access the column values. I
> tried a naive approach of FLATTENing each tuple, then trying to access the
> values that way:
>
> flattened = FOREACH data GENERATE
>   FLATTEN(isbn),
>   FLATTEN(booktitle),
>   ...
> values = FOREACH flattened GENERATE
>   $1 AS ISBN,
>   $3 AS BookTitle,
>   ...
>
> As soon as I try to access field $5, Pig complains about the index being
> out of bounds.
>
> Is there a way to solve the schema/reality mismatch? Am I doing something
> wrong, or have I stumbled across a defect?
>
> Thanks,
> Chad
>

--089e0158cb9206d58f04e4d5995d
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">hi Chad .<div><br></div><div>I have this issue</div><div><=
br></div><div>I send a mail to user-pig-list and =A0I still i can resolve t=
his, and I can not =A0access to column values.</div><div>In this mail =A0I =
write some things that I try without results... and information about this =
issue.</div>

<div><br></div><div><br></div><div><a href=3D"http://mail-archives.apache.o=
rg/mod_mbox/pig-user/201308.mbox/%3CCAJeG_hQ9S2Po3_XytZX5Xki4J1maO8q26jYdG2=
Wndy_KYiv9CQ@mail.gmail.com%3E" target=3D"_blank">http://mail-archives.apac=
he.org/mod_mbox/pig-user/201308.mbox/%3CCAJeG_hQ9S2Po3_XytZX5Xki4J1maO8q26j=
YdG2Wndy_KYiv9CQ@mail.gmail.com%3E</a><br>

</div><div><br></div><div><br></div><div><br></div><div>I hope =A0someOne r=
eply =A0one comment, idea or =A0solution about =A0this issue or bug.</div><=
div><br></div><div><br></div><div>I have reviewed the CqlStorage class in c=
ode cassandra 1.2.8 =A0but i do not have configure the environmetn to debug=
 =A0and trace this issue.</div>
<div><br></div><div>Only =A0I find some comments like, but I do not underst=
and at all.=A0</div><div><br></div><div><br></div>


<p class=3D"">/**</p>
<p class=3D"">=A0* A LoadStoreFunc for retrieving data from and storing dat=
a to <span class=3D"">Cassandra</span></p>
<p class=3D"">=A0*</p>
<p class=3D"">=A0* A row from a standard CF will be returned as nested tupl=
es:=A0</p>
<p class=3D"">=A0* (((key1, value1), (key2, value2)), ((name1, val1), (name=
2, val2))).</p>
<div>=A0*/</div><div><br></div><div><br></div><div>I you found some idea or=
 solution, please post it</div><div><br></div><div>thanks</div><div><br></d=
iv><div><br></div><div><br></div><div>=A0</div>


<div><br></div><div><br></div><div>
<br></div></div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quot=
e">2013/8/23 Chad Johnston <span dir=3D"ltr">&lt;<a href=3D"mailto:cjohnsto=
n@megatome.com" target=3D"_blank">cjohnston@megatome.com</a>&gt;</span><br>=
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<div dir=3D"ltr"><div>(I&#39;m using Cassandra 1.2.8 and Pig 0.11.1)<br></d=
iv><div><br></div><div>I&#39;m loading some simple data from Cassandra into=
 Pig using CqlStorage. The CqlStorage loader defines a Pig schema based on =
the Cassandra schema, but it seems to be wrong.</div>

<div><br></div><div>If I do:</div><div>=A0 =A0=A0</div><div>data =3D LOAD &=
#39;cql://bookdata/books&#39; USING CqlStorage();</div><div>DESCRIBE data;<=
/div><div><br></div><div>I get this:</div><div><br></div><div>data: {isbn: =
chararray,bookauthor: chararray,booktitle: chararray,publisher: chararray,y=
earofpublication: int}</div>

<div><br></div><div>However, if I DUMP data, I get results like these:</div=
><div><br></div><div>((isbn,0425093387),(bookauthor,Georgette Heyer),(bookt=
itle,Death in the Stocks),(publisher,Berkley Pub Group),(yearofpublication,=
1986))</div>

<div><br></div><div>Clearly the results from Cassandra are key/value pairs,=
 as would be expected. I don&#39;t know why the schema generated by CqlStor=
age() would be so different.</div><div><br></div><div>This is really causin=
g me problems trying to access the column values. I tried a naive approach =
of FLATTENing each tuple, then trying to access the values that way:</div>

<div><br></div><div>flattened =3D FOREACH data GENERATE</div><div>=A0 FLATT=
EN(isbn),</div><div>=A0 FLATTEN(booktitle),</div><div>=A0 ...</div><div>val=
ues =3D FOREACH flattened GENERATE</div><div>=A0 $1 AS ISBN,</div><div>=A0 =
$3 AS BookTitle,</div>

<div>=A0 ...</div><div><br></div><div>As soon as I try to access field $5, =
Pig complains about the index being out of bounds.=A0</div><div><br></div><=
div>Is there a way to solve the schema/reality mismatch? Am I doing somethi=
ng wrong, or have I stumbled across a defect?</div>

<div><br></div><div>Thanks,</div><div>Chad</div></div>
</blockquote></div><br></div>

--089e0158cb9206d58f04e4d5995d--