hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-14988) Support INSERT OVERWRITE into a partition on transactional tables
Date Fri, 03 Nov 2017 20:54:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-14988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eugene Koifman updated HIVE-14988:
----------------------------------
    Description: 
Insert overwrite operation on transactional table will currently raise an error.
This can/should be supported

This is implemented by using the concept of base_N/ directory in Acid.
INSERT OVERWRITE starts a new transaction N and creates a new base_N directory where it writes
the data.
This has the effect that on read all files with transaction IDs < N are ignored.
The advantage is that IOW doesn't need to be a mutexed operation.  All readers that stared
before this IOW can proceed at the
snapshot (Acid tables support at Snapshot Isolation) they locked in.

If mutexed semantics are desired we could add an option where IOW will request an X lock but
the base_N/ based implementation can support both.

  was:
Insert overwrite operation on transactional table will currently raise an error.
This can/should be supported


> Support INSERT OVERWRITE into a partition on transactional tables
> -----------------------------------------------------------------
>
>                 Key: HIVE-14988
>                 URL: https://issues.apache.org/jira/browse/HIVE-14988
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 1.0.0
>            Reporter: Eugene Koifman
>            Assignee: Wei Zheng
>            Priority: Major
>             Fix For: 3.0.0
>
>         Attachments: HIVE-14988.01.patch, HIVE-14988.02.patch, HIVE-14988.03.patch, HIVE-14988.04.patch,
HIVE-14988.05.patch, HIVE-14988.06.patch
>
>
> Insert overwrite operation on transactional table will currently raise an error.
> This can/should be supported
> This is implemented by using the concept of base_N/ directory in Acid.
> INSERT OVERWRITE starts a new transaction N and creates a new base_N directory where
it writes the data.
> This has the effect that on read all files with transaction IDs < N are ignored.
> The advantage is that IOW doesn't need to be a mutexed operation.  All readers that stared
before this IOW can proceed at the
> snapshot (Acid tables support at Snapshot Isolation) they locked in.
> If mutexed semantics are desired we could add an option where IOW will request an X lock
but the base_N/ based implementation can support both.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message