hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "anci_sun@yahoo.com" <anci_...@yahoo.com>
Subject Re: Re: Question about INSERT OVERWRITE TABLE with dynamic partition
Date Fri, 26 Oct 2018 03:25:01 GMT
Thanks, I think it's the proper explanation. For the query result in the second query is null,
there won't be a partition name generated in dynamic partition step, so the system doesn't
know which partition to overwrite.
Thanks very much!


Regards,
    孙志禹
 
From: Tanvi Thacker
Date: 2018-10-25 08:34
To: user
Subject: Re: Question about INSERT OVERWRITE TABLE with dynamic partition
A logical explanation could be:-
In the first query, you are telling hive which partition to overwrite, so a step which actually
deletes the partition data and overwrites it with the query result, knows that which partition
to delete and there is an empty result/file to move.

but for the second query, Dynamic partition step needs to deduce partition name from the query
result, but as your query is not producing any row, there is no info of the partition to take
action on.

Regards,
Tanvi Thacker

On Tue, Oct 23, 2018 at 9:38 PM anci_sun@yahoo.com <anci_sun@yahoo.com> wrote:
Dears,
    I found an interesting thing. 
    When inserting a NULL result into a partition which already contained some records, there
was a difference in the results between using static partition INSERT and using dynamic partition
INSERT.
    See the example below: 
        Partition '20180101' of table A contained 100 records.
        By using 
        we can delete the records in partition '20180101'.
        But by using 
        there would be no change to the partition '20180101'. 
        In fact, if we running 'select * from A where partition_A = '20180101'' , we will
still get 100 records from it.
        Expecting an explanation for it.
        Thanks!



孙志禹
Mime
View raw message