hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Travis Crawford (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-3279) Table schema not being copied to Partitions with no columns
Date Mon, 23 Jul 2012 17:53:35 GMT

    [ https://issues.apache.org/jira/browse/HIVE-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420819#comment-13420819
] 

Travis Crawford commented on HIVE-3279:
---------------------------------------

Looking at [Table.getEmptyTable|https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java#L121]
we see:

{code}
sd.setCols(new ArrayList<FieldSchema>());
{code}

What I believe happens is a new empty table is created, which initializes an empty list of
columns. No columns are actually set, because the serde reports them at runtime.

Later, when initializing a Partition we check if the storage descriptor has null for its columns.
It actually has the empty list copied from the empty table (not null) and we do not copy the
table schema into the partition.

Typically tables/partitions have an explicitly defined schema so maybe this use-case just
hasn't come up? If you explicitly define the schema things work as expected.
                
> Table schema not being copied to Partitions with no columns
> -----------------------------------------------------------
>
>                 Key: HIVE-3279
>                 URL: https://issues.apache.org/jira/browse/HIVE-3279
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Travis Crawford
>            Assignee: Travis Crawford
>
> Hive has a feature where {{Partition}}'s without any defined columns use the {{Table}}
schema. This happens in {{[Partition.initialize|https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java#L167]}}
> {code}
> // set default if columns are not set
> if (tPartition.getSd().getCols() == null) {
>   if (table.getCols() != null) {
>     tPartition.getSd().setCols(table.getCols());
>   }
> }
> {code}
> There's an issue though, because {{[Table.getEmptyTable|https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java#L121]}}
initializes cols to an empty array, which of course is not null, causing the above feature
to not work as expected.
> I'm not sure of the fix - is there a case where cols can indeed be null? I think the
best thing to do here is:
> {code}
> -        if (tPartition.getSd().getCols() == null) {
> +        if (tPartition.getSd().getCols() == null || tPartition.getSd().getCols().size()
== 0) {
> {code}
> Thoughts?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message