hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ying Chen <ying.in...@gmail.com>
Subject Re: Alter table partition rename with location - moves partition back to hive warehouse
Date Wed, 22 Mar 2017 15:09:33 GMT
I think the usage scenario for external tables generally indicates that
Hive does not try to manage it - and so it makes sense not to move tables
or change its names...

See external table:
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-ExternalTables

See rename table: (which talks about managed tables - ie not external)
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-RenameTable

On Tue, Mar 21, 2017 at 1:40 PM, Pushkar.Gujar <pushkarvgujar@gmail.com>
wrote:

> An extension to below problem, I have noticed something else too in Hive
> 2.1.0.
>
> If I create a *external* table with specific location and with
> partitions, after renaming the partition, the underline folder names do not
> change.
>
>
> ​for e.g -
> insert into test_local_part  partition (col2=1) values (1),(3);
> insert into test_local_part  partition (col2=2) values (3);
>
> this will create folder structure like-
>
> /tmp/testtable/test_local_part/col2=1/
> /tmp/testtable/test_local_part/col2=2/
>
> And after running rename,
> alter table test_local_part partition (col2='1') rename to partition
> (col2='4');
>
> *​*
> The under​line folder structure remains same as previous, shouldnt it
> change to appropriate renamed column like -
>
>
> /tmp/testtable/test_local_part/col2=
> ​4​
> /
>
>
>
> Thank you,
> *Pushkar Gujar*
>
>
> On Mon, Mar 20, 2017 at 4:40 PM, Ying Chen <ying.in.sj@gmail.com> wrote:
>
>> Hello all -
>>
>> I was renaming my partition in a table that I've created using the
>> location clause, and noticed that when after rename is completed, my
>> partition is moved to the hive warehouse (hive.metastore.warehouse.dir).
>> I was wondering, what should be the correct behavior?  Should the partition
>> be renamed and maintain on the same file system, or no name change and not
>> moved (so treating it like if someone declared external table) ?  I don't
>> think it should be moved to hive.metastore.warehouse.dir
>>
>> A similar JIRA was open for renaming table:
>> https://issues.apache.org/jira/browse/HIVE-14909
>> In which, if the table is determined not belonging to
>> /apps/hive/warehouse (ie created by location clause), then table is not
>> moved.
>>
>> Thanks much!
>> Ying
>>
>> ==========
>> This is a problem for Hive 2.1 ...
>>
>> create table test_local_part (col1 int) partitioned by (col2 int)
>> location '/tmp/testtable/test_local_part';
>> insert into test_local_part  partition (col2=1) values (1),(3);
>> insert into test_local_part  partition (col2=2) values (3);
>> alter table test_local_part partition (col2='1') rename to partition
>> (col2='4');
>>
>> Running:
>>   describe formatted test_local_part partition (col2='2')
>>
>> # Detailed Partition Information
>> Partition Value:     [2]
>> Database:           default
>> Table:               test_local_part
>> CreateTime:         Mon Mar 20 13:25:28 PDT 2017
>> LastAccessTime:     UNKNOWN
>> Protect Mode:       None
>> Location:           *hdfs://my.server.com:8020/tmp/testtable/test_local_part/col2=2
>> <http://my.server.com:8020/tmp/testtable/test_local_part/col2=2>*
>>
>> Running:
>>    describe formatted test_local_part partition (col2='4')
>>
>> # Detailed Partition Information
>> Partition Value:     [4]
>> Database:           default
>> Table:               test_local_part
>> CreateTime:         Mon Mar 20 13:24:53 PDT 2017
>> LastAccessTime:     UNKNOWN
>> Protect Mode:       None
>> Location:           *hdfs://my.server.com:8020/apps/hive/warehouse/test_local_part/col2=4
>> <http://my.server.com:8020/apps/hive/warehouse/test_local_part/col2=4>  *
>> Partition Parameters:
>>
>>
>>
>>
>

Mime
View raw message