hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Travis Crawford (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-3279) Table schema not being copied to Partitions with no columns
Date Mon, 23 Jul 2012 19:10:35 GMT

    [ https://issues.apache.org/jira/browse/HIVE-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420880#comment-13420880
] 

Travis Crawford commented on HIVE-3279:
---------------------------------------

In both the serde-reported & non-serde cases, the table schema is copied into the partition
storage descriptor. If the schema was explicitly defined, there's no need to copy it from
the table so things work correctly.

I can't actually generate a test case where the partition storage descriptor cols are null
– its either the list of explicitly defined fields, or an empty list when serde-reported.

Any ideas how to explicitly define fields for the table, but not have them copied into the
partition storage descriptor?

To double-check - do you think the current serde-reported schema behavior is a bug? If so,
I'm very interested in helping figure this one out. It feels like a simple issue where perhaps
the table cols should be initialized to null instead of an empty list, or the empty list should
be accommodated when choosing to copy the table schema.
                
> Table schema not being copied to Partitions with no columns
> -----------------------------------------------------------
>
>                 Key: HIVE-3279
>                 URL: https://issues.apache.org/jira/browse/HIVE-3279
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Travis Crawford
>            Assignee: Travis Crawford
>
> Hive has a feature where {{Partition}}'s without any defined columns use the {{Table}}
schema. This happens in {{[Partition.initialize|https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java#L167]}}
> {code}
> // set default if columns are not set
> if (tPartition.getSd().getCols() == null) {
>   if (table.getCols() != null) {
>     tPartition.getSd().setCols(table.getCols());
>   }
> }
> {code}
> There's an issue though, because {{[Table.getEmptyTable|https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java#L121]}}
initializes cols to an empty array, which of course is not null, causing the above feature
to not work as expected.
> I'm not sure of the fix - is there a case where cols can indeed be null? I think the
best thing to do here is:
> {code}
> -        if (tPartition.getSd().getCols() == null) {
> +        if (tPartition.getSd().getCols() == null || tPartition.getSd().getCols().size()
== 0) {
> {code}
> Thoughts?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message