cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremy Hanna (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-5867) The Pig CqlStorage/AbstractCassandraStorage classes don't handle collection types
Date Thu, 15 Aug 2013 12:32:51 GMT


Jeremy Hanna commented on CASSANDRA-5867:

We may not want to use the MapType for maps.

In Pig, you can only have text based keys for maps.  From Programming Pig: "A map in Pig is
a chararray to data element mapping, where that element can be any Pig type, including a complex

In Cassandra, you can have map keys of any time, like timestamp in the maps example on

So we may want to do a bag or tuple of tuples.
> The Pig CqlStorage/AbstractCassandraStorage classes don't handle collection types
> ---------------------------------------------------------------------------------
>                 Key: CASSANDRA-5867
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Hadoop
>            Reporter: Jeremy Hanna
>            Assignee: Alex Liu
>              Labels: pig
>         Attachments: 5867-1.2-branch.txt
> The CqlStorage class gets the Pig data type for values from the AbstractCassandraStorage
class, in the getPigType method.  If it isn't a known data type, it makes the value into a
ByteArray.  Currently there aren't any cases there for lists, maps, and sets.
> See this describe output from the grunt shell:
> {code}
> grunt> describe listdata ;                                        
> listdata: {id: (name: chararray,value: int),alist: (name: chararray,value: bytearray),amap:
(name: chararray,value: bytearray),aset: (name: chararray,value: bytearray)}
> {code}
> where the cql data structures had this schema:
> {code}
> CREATE TABLE alltypes (
>   id int PRIMARY KEY,
>   alist list<text>,
>   amap map<text, text>,
>   aset set<text>
> {code}
> It turns out that if you cast the map in grunt to a pig map, then it sort of works, but
I don't think we should probably use a pig map.  Lists don't appear to work at all, as there
is no Pig analogue.  I *think* you could probably just do a UDF to cast these things, but
we already have all of the type information, so we just need to change them to tuples or bags
or whatever.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message