hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sushanth Sowmyan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15534) Update db/table repl.last.id at the end of REPL LOAD of a batch of events
Date Fri, 13 Jan 2017 19:55:26 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822258#comment-15822258
] 

Sushanth Sowmyan commented on HIVE-15534:
-----------------------------------------

The actual semantic needing implementation is a bit more involved than the one in the description,
The actual semantic implemented is as per the following comment in code:

{code}
        // Now, we need to update repl.last.id for the various parent objects that were updated.
        // This update logic will work differently based on what "level" REPL LOAD was run
on.
        //  a) If this was a REPL LOAD at a table level, i.e. both dbNameOrPattern and
        //     tblNameOrPattern were specified, then the table is the only thing we should
        //     update the repl.last.id for.
        //  b) If this was a db-level REPL LOAD, then we should update the db, as well as
any
        //     tables affected by partition level operations. (any table level ops will
        //     automatically be updated as the table gets updated. Note - renames will need
        //     careful handling.
        //  c) If this was a wh-level REPL LOAD, then we should update every db for which
there
        //     were events occurring, as well as tables for which there were ptn-level ops
        //     happened. Again, renames must be taken care of.
        //
        // So, what we're going to do is have each event load update dbsUpdated and tablesUpdated
        // accordingly, but ignore updates to tablesUpdated & dbsUpdated in the case of
a
        // table-level REPL LOAD, using only the table itself. In the case of a db-level REPL
        // LOAD, we ignore dbsUpdated, but inject our own, and do not ignore tblsUpdated.
        // And for wh-level, we do no special processing, and use all of dbsUpdated and
        // tblsUpdated as-is.
{code}

> Update db/table repl.last.id at the end of REPL LOAD of a batch of events
> -------------------------------------------------------------------------
>
>                 Key: HIVE-15534
>                 URL: https://issues.apache.org/jira/browse/HIVE-15534
>             Project: Hive
>          Issue Type: Sub-task
>          Components: repl
>            Reporter: Sushanth Sowmyan
>            Assignee: Sushanth Sowmyan
>         Attachments: HIVE-15534.patch
>
>
> Tracking TODO task in ReplSemanticAnalyzer :
> {noformat}
>         // TODO : Over here, we need to track a Map<dbName:String,evLast:Long>
for every db updated
>         // and update repl.last.id for each, if this is a wh-level load, and if it is
a db-level load,
>         // then a single repl.last.id update, and if this is a tbl-lvl load which does
not alter the
>         // table itself, we'll need to update repl.last.id for that as well.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message