hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
Date Tue, 24 Oct 2017 02:46:02 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16216210#comment-16216210

stack commented on HBASE-17852:

Whats going on in here? Is it written up anywhere? Are we implementing our own transaction
management as the below comment would suggest?

Some notes after initial analysis of implementation of bulk-loaded files support
RegionObserver coprocessor has pre-commit and post-commit code (kind of Tx), but only on a
region level. No Tx support on a table level. This means that if (for some regions), loading
files fail, we will be left with a partial data in both: table store physical location and
in backup system table. What we need here is Tx support for the whole bulk load operation
cluster-wide (prepare, commit). And hooks exposed for custom coprocessor (Master?).
What will happen if bulk load operation runs at the same time as incremental backup? Backup
will see partial updates, but it should not.
We add files to backup system tables, but I have not found where and when we delete those

I saw above note but no more since....

On looking at the patch, it looks like we have a backup system table and then a whole new
table for bulk loaded materials. A whole system table just for bulk loaded files?

> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
> ------------------------------------------------------------------------------------
>                 Key: HBASE-17852
>                 URL: https://issues.apache.org/jira/browse/HBASE-17852
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>             Fix For: 2.0.0-alpha-4
>         Attachments: HBASE-17852-v1.patch, HBASE-17852-v2.patch, HBASE-17852-v3.patch,

This message was sent by Atlassian JIRA

View raw message