hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Yongqiang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-918) Allow partition-wise file format
Date Tue, 10 Nov 2009 00:49:32 GMT

    [ https://issues.apache.org/jira/browse/HIVE-918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12775211#action_12775211

He Yongqiang commented on HIVE-918:

Just had an offline discussion with Namit.
1) Will add metadata about file format and serde into partition metadata. 
2) And also will add in table new metadata about new partitions' fileformat and serde. So
there will be two set of metadata, one (let's say mold) about old fileformat and serde, and
a new one(let's say mold) for newly added partitions. The old one is specifies when creating
the table and will not changed. The new one can be changed  anytime.

For 1, will need to add a command " alter table T PARTITION p SET tableRowFormat tableSerde
this will change the fileformat and serde metadata about the specified patition. And if force
is present, will conver the partition's data by running an insert overwrite job.
For 2, will need to add a new command "ALTER TABLE SET tableRowFormat tableSerde". This will
change mnew, and will only influence the new paritions added. We will copy the mnew into new
partitions added. If mnew is null, we will just use mold and will not copy mold into partition

Any comments?

> Allow partition-wise  file format
> ---------------------------------
>                 Key: HIVE-918
>                 URL: https://issues.apache.org/jira/browse/HIVE-918
>             Project: Hadoop Hive
>          Issue Type: Improvement
>            Reporter: He Yongqiang
>            Assignee: He Yongqiang
> Right now all partitions in a hive table share the same file format. We should allow
partition wise file format. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message