hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-13646) make hive.optimize.sort.dynamic.partition compatible with ACID tables
Date Thu, 28 Apr 2016 20:34:12 GMT
Eugene Koifman created HIVE-13646:
-------------------------------------

             Summary: make hive.optimize.sort.dynamic.partition compatible with ACID tables
                 Key: HIVE-13646
                 URL: https://issues.apache.org/jira/browse/HIVE-13646
             Project: Hive
          Issue Type: Bug
          Components: Transactions
    Affects Versions: 1.0.0
            Reporter: Eugene Koifman
            Priority: Critical


HIVE-8875 disabled hive.optimize.sort.dynamic.partition for ACID queries.
dynamic inserts are common in ACID and this leaves users with few options if they are seeing
OutOfMemory errors due to too many writers.

hive.optimize.sort.dynamic.partition sorts data by partition col/bucket col/sort col to ensure
each reducer only needs 1 writer.

Acid requires data in each bucket file to be sorted by ROW__ID and thus doesn't allow end
user to determine sorting.

So we should be able to support hive.optimize.sort.dynamic.partition with
sort on partition col/bucket col/ROW__ID 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message