Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Thu, 17 Apr 2014 00:13:17 +0000 (UTC)
From: "Szehon Ho (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: <JIRA.12704406.1396059368439.129153.1397693597820@arcas>
In-Reply-To: <JIRA.12704406.1396059368439@arcas>
References: <JIRA.12704406.1396059368439@arcas>
Subject: [jira] [Commented] (HIVE-6783) Incompatible schema for maps between
 parquet-hive and parquet-pig
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-6783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13972122#comment-13972122 ] 

Szehon Ho commented on HIVE-6783:
---------------------------------

Hi [~rhbutani], the Parquet community is wondering if this can be also in 0.13.1, or some release of 0.13?  That would be helpful , as this fixes parquet format compatibility between hive and pig.  Thanks.

> Incompatible schema for maps between parquet-hive and parquet-pig
> -----------------------------------------------------------------
>
>                 Key: HIVE-6783
>                 URL: https://issues.apache.org/jira/browse/HIVE-6783
>             Project: Hive
>          Issue Type: Bug
>          Components: File Formats
>    Affects Versions: 0.13.0
>            Reporter: Tongjie Chen
>             Fix For: 0.14.0
>
>         Attachments: HIVE-6783.1.patch.txt, HIVE-6783.2.patch.txt, HIVE-6783.3.patch.txt, HIVE-6783.4.patch.txt
>
>
> see also in following parquet issue:
> https://github.com/Parquet/parquet-mr/issues/290
> The schema written for maps isn't compatible between hive and pig. This means any files written in one cannot be properly read in the other.
> More specifically,  for the same map column c1, parquet-pig generates schema:
> message pig_schema {
>   optional group c1 (MAP) {
>     repeated group map (MAP_KEY_VALUE) {
>       required binary key (UTF8);
>       optional binary value;
>     }   
>   }
> }
> while parquet-hive generates schema:
> message hive_schema {
>    optional group c1 (MAP_KEY_VALUE) {
>      repeated group map {
>        required binary key;
>        optional binary value;
>    }
>  }
> }


--
This message was sent by Atlassian JIRA
(v6.2#6252)