hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Rodionov (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
Date Thu, 16 Nov 2017 23:14:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16256124#comment-16256124
] 

Vladimir Rodionov edited comment on HBASE-17852 at 11/16/17 11:13 PM:
----------------------------------------------------------------------

{quote}
How do general backup ops and bulk loaded files effect each other?
{quote}

When backup fails, we restore backup system table from snapshot. If Observers write to the
same table as general backup operation, some data from Observers may be lost when we restore
table from snapshot. I thought, I explained that.

Second table keeps only bulk load related references. We do not care about consistency of
this table, because bulk load is idempotent operation and can be repeated after failure. Partially
written data in second table does not affect on BackupHFileCleaner plugin, because this data
(list of bulk loaded files) correspond to a files which *have not been loaded yet successfully*
and, hence - are not visible to the system






was (Author: vrodionov):
{quote}
How do general backup ops and bulk loaded files effect each other?
{quote}

When backup fails, we restore backup system table from snapshot. If Observers write to the
same table as general backup operation, some data from Observers may be lost when we restore
table from snapshot. I thought, I explained that.

Second table keeps only bulk load related references. We do not care about consistency of
this table, because bulk load is idempotent operation and can be repeated after failure. Partially
written data in second table does not affect on BackupHFileCleaner plugin, because this data
(list of bulk loaded files) correspond to a files which are *have not been loaded yet successfully*
and, hence - are not visible to the system





> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
> ------------------------------------------------------------------------------------
>
>                 Key: HBASE-17852
>                 URL: https://issues.apache.org/jira/browse/HBASE-17852
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>             Fix For: 2.0.0-beta-1
>
>         Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, HBASE-17852-v3.patch,
HBASE-17852-v4.patch, HBASE-17852-v5.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message