Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C0095200C5B for ; Tue, 4 Apr 2017 23:48:46 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id BEBD0160BA1; Tue, 4 Apr 2017 21:48:46 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id E8857160B90 for ; Tue, 4 Apr 2017 23:48:45 +0200 (CEST) Received: (qmail 82808 invoked by uid 500); 4 Apr 2017 21:48:45 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 82634 invoked by uid 99); 4 Apr 2017 21:48:45 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Apr 2017 21:48:45 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id AA0A9C0621 for ; Tue, 4 Apr 2017 21:48:44 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 9OXeNcVHzVxf for ; Tue, 4 Apr 2017 21:48:43 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 7C70A5FE07 for ; Tue, 4 Apr 2017 21:48:43 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id CE86CE0A12 for ; Tue, 4 Apr 2017 21:48:42 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id E58B924024 for ; Tue, 4 Apr 2017 21:48:41 +0000 (UTC) Date: Tue, 4 Apr 2017 21:48:41 +0000 (UTC) From: "Hadoop QA (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-14141) HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits from backup tables MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 04 Apr 2017 21:48:46 -0000 [ https://issues.apache.org/jira/browse/HBASE-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15955921#comment-15955921 ] Hadoop QA commented on HBASE-14141: ----------------------------------- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 36s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 42s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 48s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 15s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 53s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 46s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 44s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 15s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 28m 3s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 57s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 142m 40s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m 5s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 185m 29s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hbase.client.TestBlockEvictionFromClient | | | hadoop.hbase.regionserver.wal.TestAsyncLogRollPeriod | | | hadoop.hbase.filter.TestColumnRangeFilter | | | hadoop.hbase.quotas.TestQuotaThrottle | | Timed out junit tests | org.apache.hadoop.hbase.client.TestAsyncTableScanAll | | | org.apache.hadoop.hbase.client.TestMultiRespectsLimits | | | org.apache.hadoop.hbase.util.TestMiniClusterLoadEncoded | | | org.apache.hadoop.hbase.regionserver.wal.TestLogRolling | | | org.apache.hadoop.hbase.TestAcidGuarantees | | | org.apache.hadoop.hbase.regionserver.TestCompoundBloomFilter | | | org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreadedWithEagerCompaction | | | org.apache.hadoop.hbase.client.TestAsyncTable | | | org.apache.hadoop.hbase.filter.TestMultiRowRangeFilter | | | org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel | | | org.apache.hadoop.hbase.client.TestHCM | | | org.apache.hadoop.hbase.client.TestSizeFailures | | | org.apache.hadoop.hbase.client.TestCloneSnapshotFromClientWithRegionReplicas | | | org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient | | | org.apache.hadoop.hbase.regionserver.TestPriorityRpc | | | org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover | | | org.apache.hadoop.hbase.util.TestIdReadWriteLock | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.10.1 Server=1.10.1 Image:yetus/hbase:8d52d23 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12861939/HBASE-14141.v6.patch | | JIRA Issue | HBASE-14141 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 812eb760e4d7 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / e916b79 | | Default Java | 1.8.0_121 | | findbugs | v3.0.0 | | unit | https://builds.apache.org/job/PreCommit-HBASE-Build/6313/artifact/patchprocess/patch-unit-hbase-server.txt | | unit test logs | https://builds.apache.org/job/PreCommit-HBASE-Build/6313/artifact/patchprocess/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/6313/testReport/ | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/6313/console | | Powered by | Apache Yetus 0.3.0 http://yetus.apache.org | This message was automatically generated. > HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits from backup tables > -------------------------------------------------------------------------------------------- > > Key: HBASE-14141 > URL: https://issues.apache.org/jira/browse/HBASE-14141 > Project: HBase > Issue Type: New Feature > Reporter: Vladimir Rodionov > Assignee: Vladimir Rodionov > Priority: Blocker > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-14141.HBASE-14123.v1.patch, HBASE-14141.v1.patch, HBASE-14141.v2.patch, HBASE-14141.v4.patch, HBASE-14141.v5.patch, HBASE-14141.v6.patch > > > h2. High level design overview > * When incremental backup request comes for tables {t} we select all the tables already registered in a backup system - {T} and union them with {t}, which results in a new table set - U(t, T) > * For every table K from U(t,T) we perform the following: > ** Convert new WAL files into HFile applying table filter K (only edits for table T will pass the filter) > ** Move these HFile(s) to backup destination > During restore (incremental): > * We run full restore first > * Then collect all HFiles from intermediate incremental images and run them through HFileSplitterJob, which splits files into a current tables region boundaries > * Load these files using LoadIncrementalHFiles tool > -- This message was sent by Atlassian JIRA (v6.3.15#6346)