hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "hezhiqiang (Ransom)" <ransom.hezhiqi...@huawei.com>
Subject RE: [jira] [Created] (HIVE-2814) Can we have a feature to disable creating empty buckets on a larger number of buckets creates?
Date Wed, 22 Feb 2012 15:26:08 GMT
I think maybe datas in your partitions is skew,
Why not change anther column for partition?


Best regards
Ransom.


-----Original Message-----
From: Nitin Pawar (Created) (JIRA) [mailto:jira@apache.org] 
Sent: Wednesday, February 22, 2012 9:54 PM
To: hive-dev@hadoop.apache.org
Subject: [jira] [Created] (HIVE-2814) Can we have a feature to disable creating empty buckets
on a larger number of buckets creates?

Can we have a feature to disable creating empty buckets on a larger number of buckets creates?

-----------------------------------------------------------------------------------------------

                 Key: HIVE-2814
                 URL: https://issues.apache.org/jira/browse/HIVE-2814
             Project: Hive
          Issue Type: Bug
            Reporter: Nitin Pawar
            Priority: Minor


When we create buckets on a larger datasets, its not often that all the partitions have same
number of buckets so we choose the largest possible number to capture the buckets mostly.

It results into creating lot of empty buckets, which might be an overhead of hadoop as well
as for hive queries. 
Also it takes a lot of time to just create empty buckets. 

Is there a way where I can say do not create empty buckets? 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message