Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: neutral (athena.apache.org: 209.85.214.44 is neither permitted
 nor denied by domain of oberman@civicscience.com)
MIME-Version: 1.0
From: William Oberman <oberman@civicscience.com>
Date: Wed, 15 Jun 2011 14:17:02 -0400
Message-ID: <BANLkTim01V37vHO1dnO_uVuOzo4g_PGjeg@mail.gmail.com>
Subject: prep for cassandra storage from pig
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=0016e6dbe6b7ef379a04a5c42998

--0016e6dbe6b7ef379a04a5c42998
Content-Type: text/plain; charset=ISO-8859-1

I think I'm stuck on typing issues trying to store data in cassandra.  To
verify, cassandra wants (key, {tuples})

My pig script is fairly brief:
raw = LOAD 'cassandra://test_in/test_cf' USING CassandraStorage() AS
(key:chararray, columns:bag {column:tuple (name, value)});
--colums == timeUUID -> JSON
rows = FOREACH raw GENERATE key, FLATTEN(columns);
alias_target_day = FOREACH rows {
    --I wrote a specialized parser that does exactly what I need
    observation_map = com.civicscience.pig.ParseObservation($2);
    GENERATE $0 as alias, observation_map#'_fqt' as target,
observation_map#'_day' as day;
};
grouping = GROUP alias_target_day BY ((chararray)target,(chararray)day);
X = FOREACH grouping GENERATE group.$0 as target, TOTUPLE(group.$1,
COUNT($1)) as day_count;

This gets me:
(targetA, (day1, count))
(targetA, (day2, count))
(targetB, (day1, count))
....

But, cassandra wants the 2nd item to be a bag.  So, I tried:
X = FOREACH grouping GENERATE group.$0 as target, TOBAG(TOTUPLE(group.$1,
COUNT($1))) as day_count;

But this results in:
(targetA, {((day1, count))})
(targetA, {((day2, count))})
(targetB, {((day1, count))})
It's hard to see, but the 2nd item now has a nested tuple as the first
value, which is still bad.

How to I get (key, {tuple})???  I wasn't sure where to post this (pig or
cassandra), so I'm posting to the pig list too.

will

--0016e6dbe6b7ef379a04a5c42998
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I think I&#39;m stuck on typing issues trying to store data in cassandra.=
=A0 To verify, cassandra wants (key, {tuples})<br><br>My pig script is fair=
ly brief:<br>raw =3D LOAD &#39;cassandra://test_in/test_cf&#39; USING Cassa=
ndraStorage() AS (key:chararray, columns:bag {column:tuple (name, value)});=
<br>

--colums =3D=3D timeUUID -&gt; JSON<br>
rows =3D FOREACH raw GENERATE key, FLATTEN(columns);<br>alias_target_day =
=3D FOREACH rows {<br>=A0=A0=A0 --I wrote a specialized parser that does ex=
actly what I need<br>
=A0=A0=A0 observation_map =3D com.civicscience.pig.ParseObservation($2);<br=
>
=A0=A0=A0 GENERATE $0 as alias, observation_map#&#39;_fqt&#39; as target, o=
bservation_map#&#39;_day&#39; as day;<br>
};<br>
grouping =3D GROUP alias_target_day BY ((chararray)target,(chararray)day);<=
br>
X =3D FOREACH grouping GENERATE group.$0 as target, TOTUPLE(group.$1, COUNT=
($1)) as day_count;<br><br>This gets me:<br>(targetA, (day1, count))<br>(ta=
rgetA, (day2, count))<br>(targetB, (day1, count))<br>....<br><br>But, cassa=
ndra wants the 2nd item to be a bag.=A0 So, I tried:<br>

X =3D FOREACH grouping GENERATE group.$0 as target, TOBAG(TOTUPLE(group.$1,=
 COUNT($1))) as day_count;<br clear=3D"all"><br>But this results in:<br>(ta=
rgetA, {((day1, count))})<br>
(targetA, {((day2, count))})<br>
(targetB, {((day1, count))})<br>It&#39;s hard to see, but the 2nd item now =
has a nested tuple as the first value, which is still bad.<br><br>How to I =
get (key, {tuple})???=A0 I wasn&#39;t sure where to post this (pig or cassa=
ndra), so I&#39;m posting to the pig list too.<br>

<br>will<br>

--0016e6dbe6b7ef379a04a5c42998--