hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
Date Tue, 30 Jan 2018 00:08:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344231#comment-16344231

Josh Elser commented on HBASE-17852:

HBASE-19568 had basically everything that was objected in the reviews here, why wasn't it
brought to the attention of people who raised objections? The title/reason of that jira reason
doesn't matter.
I see it as a really sly move - going behind community and committed changes which were heavily
objected against, by using separate jira.

[~appy], let's take a step back, please. I called this out to your attention -- I was under
the impression that, based on your earlier comment ([here|https://issues.apache.org/jira/browse/HBASE-17852?focusedCommentId=16327774&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16327774])
that you were OK of this implementation landing in master as-is.

HBASE-19568 was used to commit to master (with what I thought was your blessing) while we
continue to use this JIRA issue to flesh out design because of all of the discussion that
has happened. If I misunderstood you or poorly asked you the question, let's take that over
to HBASE-19568 and get a revert in place. There was nothing malicious intending to happen

> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
> ------------------------------------------------------------------------------------
>                 Key: HBASE-17852
>                 URL: https://issues.apache.org/jira/browse/HBASE-17852
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>            Priority: Major
>             Fix For: 3.0.0
>         Attachments: HBASE-17852-v10.patch, screenshot-1.png
> Design approach rollback-via-snapshot implemented in this ticket:
> # Before backup create/delete/merge starts we take a snapshot of the backup meta-table
(backup system table). This procedure is lightweight because meta table is small, usually
should fit a single region.
> # When operation fails on a server side, we handle this failure by cleaning up partial
data in backup destination, followed by restoring backup meta-table from a snapshot. 
> # When operation fails on a client side (abnormal termination, for example), next time
user will try create/merge/delete he(she) will see error message, that system is in inconsistent
state and repair is required, he(she) will need to run backup repair tool.
> # To avoid multiple writers to the backup system table (backup client and BackupObserver's)
we introduce small table ONLY to keep listing of bulk loaded files. All backup observers will
work only with this new tables. The reason: in case of a failure during backup create/delete/merge/restore,
when system performs automatic rollback, some data written by backup observers during failed
operation may be lost. This is what we try to avoid.
> # Second table keeps only bulk load related references. We do not care about consistency
of this table, because bulk load is idempotent operation and can be repeated after failure.
Partially written data in second table does not affect on BackupHFileCleaner plugin, because
this data (list of bulk loaded files) correspond to a files which have not been loaded yet
successfully and, hence - are not visible to the system 

This message was sent by Atlassian JIRA

View raw message