Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 5 Apr 2017 04:04:41 +0000 (UTC)
From: "Hudson (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12846691.1437513422000.207256.1491365081849@Atlassian.JIRA>
In-Reply-To: <JIRA.12846691.1437513422000@Atlassian.JIRA>
References: <JIRA.12846691.1437513422000@Atlassian.JIRA> <JIRA.12846691.1437513422935@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HBASE-14141) HBase Backup/Restore Phase 3:
 Filter WALs on backup to include only edits from backed up tables
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 05 Apr 2017 04:04:52 -0000


    [ https://issues.apache.org/jira/browse/HBASE-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15956269#comment-15956269 ] 

Hudson commented on HBASE-14141:
--------------------------------

SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2801 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2801/])
HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to (tedyu: rev 910b68082c8f200f0ba6395a76b7ee1c8917e401)
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/impl/RestoreTablesClient.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/mapreduce/MapReduceRestoreJob.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/impl/IncrementalTableBackupClient.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/WALInputFormat.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractFSWALProvider.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/mapreduce/HFileSplitterJob.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/util/RestoreTool.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/impl/TableBackupClient.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/backup/impl/IncrementalBackupManager.java


> HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits from backed up tables
> -----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-14141
>                 URL: https://issues.apache.org/jira/browse/HBASE-14141
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>            Priority: Blocker
>              Labels: backup
>             Fix For: 2.0.0
>
>         Attachments: HBASE-14141.HBASE-14123.v1.patch, HBASE-14141.v1.patch, HBASE-14141.v2.patch, HBASE-14141.v4.patch, HBASE-14141.v5.patch, HBASE-14141.v6.patch
>
>
> h2. High level design overview
> * When incremental backup request comes for tables {t} we select all the tables already registered in a backup system  - {T} and union them with {t}, which results in a new table set - U(t, T)
> * For every table K from U(t,T) we perform the following:
> ** Convert new WAL files into HFile applying table filter K (only edits for table T will pass the filter)
> ** Move these HFile(s) to backup destination
> During restore (incremental):
> * We run full restore first
> * Then collect all HFiles from intermediate incremental images and run them through HFileSplitterJob, which splits files into a current tables region boundaries
> * Load these files using LoadIncrementalHFiles tool
>   


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)