hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <>
Subject [jira] [Created] (HIVE-17970) LOAD DATA with OVERWRITE doesn't use base_n directory concept
Date Thu, 02 Nov 2017 17:18:01 GMT
Eugene Koifman created HIVE-17970:

             Summary: LOAD DATA with OVERWRITE doesn't use base_n directory concept
                 Key: HIVE-17970
             Project: Hive
          Issue Type: Sub-task
          Components: Transactions
    Affects Versions: 3.0.0
            Reporter: Eugene Koifman
            Priority: Major

Judging by 

Hive.loadTable(Path loadPath, String tableName, LoadFileType loadFileType, boolean isSrcLocal,
      boolean isSkewedStoreAsSubdir, boolean isAcid, boolean hasFollowingStatsTask,
      Long txnId, int stmtId, boolean isMmTable)

LOAD DATA with OVERWRITE will delete all existing data then write new data into the table.
 This logic makes sense for non-acid tables but for Acid/MM it should work like INSERT OVERWRITE
statement and write new data to base_n/. This way the lock manager can be used to either get
an X lock for IOW and thus block all readers or let it run with SemiShared and let readers
continue and make the system more concurrent.

This message was sent by Atlassian JIRA

View raw message