hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sankar Hariappan (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-16644) Hook Change Manager to Insert Overwrite
Date Wed, 07 Jun 2017 07:49:18 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16040425#comment-16040425
] 

Sankar Hariappan edited comment on HIVE-16644 at 6/7/17 7:48 AM:
-----------------------------------------------------------------

Added 02.patch with fixes for Anishek's comments.
- Get lastReplID of INSERT INTO using REPL DUMP instead of assuming it to 3 events.
- Remove verifySetup calls which validates the operations is primary cluster.

Moving the test to TestReplicationScenariosAcrossInstances is not done now as Repl Change
Management doesn't work if cmroot and warehouse path belongs to different file systems. Also,
it doesn't work even if primary and replica configures CMDIR to different file systems. As
it is a bigger problem, it will be handled with another JIRA.

ReplChangeManager.recycle skips the temp files with prefix "_" but replaceFiles method need
to delete those files as well. Hence retaining the trashFiles call even if CM enabled.

Request [~anishek] to review the new patch! 


was (Author: sankarh):
Added 02.patch with fixes for Anishek's comments.
- Get lastReplID of INSERT INTO using REPL DUMP instead of assuming it to 3 events.
- Remove verifySetup calls which validates the operations is primary cluster.

Moving the test to TestReplicationScenariosAcrossInstances is not done now as Repl Change
Management doesn't work if cmroot and warehouse path belongs to different file systems. Also,
it doesn't work even if primary and replica configures CMDIR to different file systems. As
it is a bigger problem, it will be handled with another JIRA.
ReplChangeManager.recycle skips the temp files with prefix "_" but replaceFiles method need
to delete those files as well. Hence retaining the trashFiles call even if CM enabled.

Request [~anishek] to review the new patch! 

> Hook Change Manager to Insert Overwrite
> ---------------------------------------
>
>                 Key: HIVE-16644
>                 URL: https://issues.apache.org/jira/browse/HIVE-16644
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Hive, repl
>    Affects Versions: 2.1.0
>            Reporter: Sankar Hariappan
>            Assignee: Sankar Hariappan
>              Labels: DR, replication
>         Attachments: HIVE-16644.01.patch, HIVE-16644.02.patch
>
>
> For insert overwrite Hive.replaceFiles is called to replace contents of existing partitions/table.
This should trigger move of old files into $CMROOT.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message