hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Rodionov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14141) HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits from backup tables
Date Mon, 07 Nov 2016 19:24:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15645161#comment-15645161
] 

Vladimir Rodionov commented on HBASE-14141:
-------------------------------------------

Two approaches

h5. Direct filtering

We can filter WAL during copy operation to include only edits for a tables being backed up.

Pro:

# Flexibility ?

Contra:

# Complexity. Requires new version of DistCp or other custom MR job
# Complexity. Tracking WALs eligible for deletion becomes a nightmare.  When WAL is closed
we need to write all tables in a current incremental backup set in hbase:backup table and
create record for every table:WAL with initial state something like WAIT. Then we will have
to update state of these records after each incremental backup session and keep WAL until
all records are updated. Two issues:
## Atomicity of combination of events: WAL closing and getting list of tables from hbase:backup.
## Handling of table deletion and removing tables from backup set
# Complexity. All backup destination file structures needs to be changed

h5. Have a separate WAL group for tables in backup

We have a dedicated WAL group for tables we want to backup. Table MUST be assigned to this
group if we want to backup it. This is something that could be done by  manipulating RegionGroupingStrategy,
as [~jinghe] suggested.

Pro:

# Simplicity of implementation

Contra:

# Requires documentation how-to to explain how to optimize backup in case of subset of tables.

I personally like the latter one.



> HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits from backup
tables
> --------------------------------------------------------------------------------------------
>
>                 Key: HBASE-14141
>                 URL: https://issues.apache.org/jira/browse/HBASE-14141
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 2.0.0
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>              Labels: backup
>             Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message