incubator-hcatalog-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HCATALOG-2) Support nested schema conversion between Hive an Pig
Date Thu, 19 May 2011 18:34:47 GMT

    [ https://issues.apache.org/jira/browse/HCATALOG-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036355#comment-13036355
] 

Ashutosh Chauhan commented on HCATALOG-2:
-----------------------------------------

HCatalog uses following mapping:
TUPLE    ->   STRUCT
ARRAY   ->   BAG

One caveat here is HIve's Array can nest any other type, but Pig restricts bag to contain
only tuples. So, HCatalog gets rid of this nested tuple when writing out data coming from
Pig and inserts in a tuple when reading in. So, Pig's bag { tuple : (i : int)}
gets translated into Hive's Array <i:int> and NOT into Array<Struct<i:int>>


> Support nested schema conversion between Hive an Pig
> ----------------------------------------------------
>
>                 Key: HCATALOG-2
>                 URL: https://issues.apache.org/jira/browse/HCATALOG-2
>             Project: HCatalog
>          Issue Type: Improvement
>            Reporter: Julien Le Dem
>            Assignee: Jakob Homan
>
> Hive supports STRUCT and ARRAY
> Pig supports Tuple and Bag
> In both cases the types can be arbitrarily nested.
> example:
> STRUCT< foo : ARRAY<STRUCT < bar1: STRING, bar2: STRING > > mystruct
> mytuple: ( foo: { t: ( bar1:chararray, bar2:chararray ) } )
> HCatalog should support nested schema conversion.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message