Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 16 Nov 2017 23:14:00 +0000 (UTC)
From: "Vladimir Rodionov (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13060154.1490821740000.260308.1510874040249@Atlassian.JIRA>
In-Reply-To: <JIRA.13060154.1490821740000@Atlassian.JIRA>
References: <JIRA.13060154.1490821740000@Atlassian.JIRA> <JIRA.13060154.1490821740887@jira-lw-us.apache.org>
Subject: [jira] [Comment Edited] (HBASE-17852) Add Fault tolerance to
 HBASE-14417 (Support bulk loaded files in incremental backup)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 16 Nov 2017 23:14:05 -0000


    [ https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16256124#comment-16256124 ] 

Vladimir Rodionov edited comment on HBASE-17852 at 11/16/17 11:13 PM:
----------------------------------------------------------------------

{quote}
How do general backup ops and bulk loaded files effect each other?
{quote}

When backup fails, we restore backup system table from snapshot. If Observers write to the same table as general backup operation, some data from Observers may be lost when we restore table from snapshot. I thought, I explained that.

Second table keeps only bulk load related references. We do not care about consistency of this table, because bulk load is idempotent operation and can be repeated after failure. Partially written data in second table does not affect on BackupHFileCleaner plugin, because this data (list of bulk loaded files) correspond to a files which *have not been loaded yet successfully* and, hence - are not visible to the system


was (Author: vrodionov):
{quote}
How do general backup ops and bulk loaded files effect each other?
{quote}

When backup fails, we restore backup system table from snapshot. If Observers write to the same table as general backup operation, some data from Observers may be lost when we restore table from snapshot. I thought, I explained that.

Second table keeps only bulk load related references. We do not care about consistency of this table, because bulk load is idempotent operation and can be repeated after failure. Partially written data in second table does not affect on BackupHFileCleaner plugin, because this data (list of bulk loaded files) correspond to a files which are *have not been loaded yet successfully* and, hence - are not visible to the system


> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
> ------------------------------------------------------------------------------------
>
>                 Key: HBASE-17852
>                 URL: https://issues.apache.org/jira/browse/HBASE-17852
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>             Fix For: 2.0.0-beta-1
>
>         Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, HBASE-17852-v3.patch, HBASE-17852-v4.patch, HBASE-17852-v5.patch
>
>


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)