hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjeev Verma <sanjeev.verm...@gmail.com>
Subject Re: hive transaction strange behaviour
Date Tue, 17 Nov 2015 09:09:24 GMT
Thank Elliot, Eugene
I am able to see the Base file created in one of the partition, seems the
Compactor kicked in and created it but it has not created base files in
rest of the partition where delta files still exists.why compactor has not
picked the other partition, when and how these partition will be picked up
for compaction.

Thanks

On Sat, Nov 14, 2015 at 11:01 PM, Eugene Koifman <ekoifman@hortonworks.com>
wrote:

> When Compaction process runs, it will create base directory.
>
> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
>
>
> at a minimum you need hive.compactor.initiator.on
> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.compactor.initiator.on>=true
> and hive.compactor.worker.threads
> <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.compactor.worker.threads>
> >0
>
> Also, see
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterTable/PartitionCompact
on
> how to trigger compaction manually.
>
> *Eugene*
>
> From: Sanjeev Verma <sanjeev.verma82@gmail.com>
> Reply-To: "user@hive.apache.org" <user@hive.apache.org>
> Date: Thursday, November 12, 2015 at 11:41 PM
> To: "user@hive.apache.org" <user@hive.apache.org>, "dev@hive.apache.org" <
> dev@hive.apache.org>
> Subject: hive transaction strange behaviour
>
> I have enable the hive transaction and able to see the delta files created
> for some of the partition but i dont not see any base file created yet.it
> seems strange to me seeing so many delta files without any base file.
> Could somebody let me know when Base file created.
>
> Thanks
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message