Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Mon, 23 Jul 2012 15:43:34 +0000 (UTC)
From: "Travis Crawford (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: <250751764.91367.1343058214707.JavaMail.jiratomcat@issues-vm>
In-Reply-To: <512794351.78410.1342733735023.JavaMail.jiratomcat@issues-vm>
Subject: [jira] [Commented] (HIVE-3279) Table schema not being copied to
 Partitions with no columns
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420724#comment-13420724 ] 

Travis Crawford commented on HIVE-3279:
---------------------------------------

Applying the above change and rerunning these statements causes the partition to have dynamically-reported columns:

{code:title=Proposed behavior: partition has correct columns with above change}
hive> create external table int_string
    >   partitioned by (b string)
    >   row format serde "org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer"
    >   with serdeproperties (
    >     "serialization.class"="org.apache.hadoop.hive.serde2.thrift.test.IntString",
    >     "serialization.format"="org.apache.thrift.protocol.TBinaryProtocol");
OK
Time taken: 0.085 seconds
hive> alter table int_string add partition (b='part1');
OK
Time taken: 0.128 seconds
hive> describe int_string partition (b='part1');     
OK
myint	int	from deserializer
mystring	string	from deserializer
underscore_int	int	from deserializer
b	string	
Time taken: 0.09 seconds
hive> 
{code}
                
> Table schema not being copied to Partitions with no columns
> -----------------------------------------------------------
>
>                 Key: HIVE-3279
>                 URL: https://issues.apache.org/jira/browse/HIVE-3279
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Travis Crawford
>            Assignee: Travis Crawford
>
> Hive has a feature where {{Partition}}'s without any defined columns use the {{Table}} schema. This happens in {{[Partition.initialize|https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java#L167]}}
> {code}
> // set default if columns are not set
> if (tPartition.getSd().getCols() == null) {
>   if (table.getCols() != null) {
>     tPartition.getSd().setCols(table.getCols());
>   }
> }
> {code}
> There's an issue though, because {{[Table.getEmptyTable|https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java#L121]}} initializes cols to an empty array, which of course is not null, causing the above feature to not work as expected.
> I'm not sure of the fix - is there a case where cols can indeed be null? I think the best thing to do here is:
> {code}
> -        if (tPartition.getSd().getCols() == null) {
> +        if (tPartition.getSd().getCols() == null || tPartition.getSd().getCols().size() == 0) {
> {code}
> Thoughts?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira