hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14417) Incremental backup and bulk loading
Date Fri, 26 Aug 2016 17:41:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439447#comment-15439447
] 

Ted Yu commented on HBASE-14417:
--------------------------------

There may be more than one round of bulk load between the full backup and incremental backup(s).
For each round, we may use timestamp of completion of bulk load for the loaded hfiles (in
terms of record in hbase:backup).

When the next incremental backup takes place, we consolidate all the recorded bulk loaded
hfiles and save the list in manifest of the incremental backup.

> Incremental backup and bulk loading
> -----------------------------------
>
>                 Key: HBASE-14417
>                 URL: https://issues.apache.org/jira/browse/HBASE-14417
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 2.0.0
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>            Priority: Critical
>              Labels: backup
>             Fix For: 2.0.0
>
>
> Currently, incremental backup is based on WAL files. Bulk data loading bypasses WALs
for obvious reasons, breaking incremental backups. The only way to continue backups after
bulk loading is to create new full backup of a table. This may not be feasible for customers
who do bulk loading regularly (say, every day).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message