hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Gates <alanfga...@gmail.com>
Subject Re: Compaction in hive
Date Tue, 06 Dec 2016 19:02:33 GMT
What exactly do you mean by compaction?  Hive has a compactor that runs over ACID tables to
handle the delta files[1], but I’m guessing you don’t mean that.  Are you wanting to concatenate
files in existing tables?  The usual way to do that is alter table concatenate[2].  Or do
you mean something else?

Alan.

1. see https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Compactor
2. see https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterTable/PartitionConcatenate

> On Dec 6, 2016, at 07:03, Nishant Aggarwal <nishant.agg@gmail.com> wrote:
> 
> Dear Hive Gurus,
> 
> I am looking to some practical solution on how to implement Compaction in Hive. Hiveserver2
version 1.1.0.
> 
> We have some external Hive tables on which we  need to implement Compaction. 
> 
> Merging the map files is one option which is turned down since it is very CPU intensive.
> 
> Need your help in order to implement Compaction, how to implement, what are the pros
and cons.
> 
> Also, is it mandatory to have bucketing to implement compaction?
> 
> Request you to please help.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Thanks and Regards
> Nishant Aggarwal, PMP
> Cell No:- +91 99588 94305
> http://in.linkedin.com/pub/nishant-aggarwal/53/698/11b
> 


Mime
View raw message