Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 18F471019F for ; Tue, 27 Aug 2013 04:23:55 +0000 (UTC) Received: (qmail 98201 invoked by uid 500); 27 Aug 2013 04:23:54 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 98169 invoked by uid 500); 27 Aug 2013 04:23:53 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 98146 invoked by uid 99); 27 Aug 2013 04:23:52 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Aug 2013 04:23:52 +0000 Date: Tue, 27 Aug 2013 04:23:52 +0000 (UTC) From: "Alex Liu (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CASSANDRA-5867) The Pig CqlStorage/AbstractCassandraStorage classes don't handle collection types MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-5867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13750947#comment-13750947 ] Alex Liu commented on CASSANDRA-5867: ------------------------------------- First apply 5867-bug-fix-filter-push-down-1.2-branch.txt to fix issue with filter push down. (schema changes), then apply 5867-4-1.2-branch.txt for collection supports in CqlStorage which map SetType/ListType to tuple, MapType to tuple of tuples. > The Pig CqlStorage/AbstractCassandraStorage classes don't handle collection types > --------------------------------------------------------------------------------- > > Key: CASSANDRA-5867 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5867 > Project: Cassandra > Issue Type: Bug > Components: Hadoop > Reporter: Jeremy Hanna > Assignee: Alex Liu > Labels: pig > Attachments: 5867-1.2-branch.txt, 5867-2-1.2-branch.txt, 5867-3-1.2-branch.txt, 5867-4-1.2-branch.txt, 5867-bug-fix-filter-push-down-1.2-branch.txt > > > The CqlStorage class gets the Pig data type for values from the AbstractCassandraStorage class, in the getPigType method. If it isn't a known data type, it makes the value into a ByteArray. Currently there aren't any cases there for lists, maps, and sets. > https://github.com/apache/cassandra/blob/cassandra-1.2.8/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java#L336 > See this describe output from the grunt shell: > {code} > grunt> describe listdata ; > listdata: {id: (name: chararray,value: int),alist: (name: chararray,value: bytearray),amap: (name: chararray,value: bytearray),aset: (name: chararray,value: bytearray)} > {code} > where the cql data structures had this schema: > {code} > CREATE TABLE alltypes ( > id int PRIMARY KEY, > alist list, > amap map, > aset set > {code} > It turns out that if you cast the map in grunt to a pig map, then it sort of works, but I don't think we should probably use a pig map. Lists don't appear to work at all, as there is no Pig analogue. I *think* you could probably just do a UDF to cast these things, but we already have all of the type information, so we just need to change them to tuples or bags or whatever. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira