hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Rodionov (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
Date Tue, 12 Sep 2017 22:03:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16163729#comment-16163729
] 

Vladimir Rodionov edited comment on HBASE-17852 at 9/12/17 10:02 PM:
---------------------------------------------------------------------

Some notes after initial analysis of implementation of bulk-loaded files support

# RegionObserver coprocessor has pre-commit and post-commit code (kind of Tx), but only on
a region level. No Tx support on a table level. This means that if (for some regions), loading
files fail, we will be left with a partial data in both: table store physical location and
in backup system table. What we need here is Tx support for the whole bulk load operation
cluster-wide (prepare, commit). And hooks exposed for custom coprocessor (Master?). 
# What will happen if bulk load operation runs at the same time as  incremental backup? Backup
will see partial updates, but it should not.
# We add files to backup system tables, but I have not found where and when we delete those
files. 



was (Author: vrodionov):
Some notes after initial analysis of implementation of bulk-loaded files support

# RegionObserver coprocessor has pre-commit and post-commit code (kind of Tx), but only on
a region level. No Tx support on a table level. This means that if (for some regions), loading
files fail, we will be left with a partial data in both: table store physical location and
in backup system table. What we need here is Tx support for the whole bulk load operation
cluster-wide (prepare, commit). And hooks exposed for custom coprocessor (Master?). 
# What will happen if bulk load operation runs at the same time as  incremental backup? Backup
will see partial updates, but it should not.
# We add files to backup system tables, but I have not found where and when we deleted those
files. 


> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
> ------------------------------------------------------------------------------------
>
>                 Key: HBASE-17852
>                 URL: https://issues.apache.org/jira/browse/HBASE-17852
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>             Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message