Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 7CECF200B70 for ; Fri, 12 Aug 2016 19:44:22 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 79BAC160A85; Fri, 12 Aug 2016 17:44:22 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id C8C13160AB0 for ; Fri, 12 Aug 2016 19:44:21 +0200 (CEST) Received: (qmail 82841 invoked by uid 500); 12 Aug 2016 17:44:20 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 82620 invoked by uid 99); 12 Aug 2016 17:44:20 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 12 Aug 2016 17:44:20 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id B062C2C02A1 for ; Fri, 12 Aug 2016 17:44:20 +0000 (UTC) Date: Fri, 12 Aug 2016 17:44:20 +0000 (UTC) From: "Ted Yu (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-14450) HBase Backup/Restore Phase 3: Multiwal support MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Fri, 12 Aug 2016 17:44:22 -0000 [ https://issues.apache.org/jira/browse/HBASE-14450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419192#comment-15419192 ] Ted Yu commented on HBASE-14450: -------------------------------- With hadoop 2.7.3 patched with HADOOP-13496, I got: {code} 2016-08-12 10:37:25,691 ERROR [LocalJobRunner Map Task Executor #0] util.RetriableCommand(89): Failure in Retriable command: Copying hdfs://localhost:54465/user/tyu/test-data/4dba2cf9-4c45-4eaa-86cb- a84be3a21712/WALs/10.22.9.171,54473,1471023349505/10.22.9.171%2C54473%2C1471023349505.regiongroup-1.1471023351523 to hdfs://localhost:54465/backupUT/backup_1471023430951/WALs/10.22.9. 171%2C54473%2C1471023349505.regiongroup-1.1471023351523 java.io.IOException: Mismatch in length of source:hdfs://localhost:54465/user/tyu/test-data/4dba2cf9-4c45-4eaa-86cb-a84be3a21712/WALs/10.22.9.171,54473,1471023349505/10.22.9.171%2C54473%2C1471023349505.regiongroup-1.1471023351523 and target:hdfs://localhost:54465/backupUT/backup_1471023430951/WALs/.distcp.tmp.attempt_local1883551993_0006_m_000000_0 - 83 : 973 at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.compareFileLengths(RetriableFileCopyCommand.java:194) at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:126) at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:99) at org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87) at org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:281) at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:253) at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) {code} There was no change in length of either source or dest file across the two distcp runs. There was no log indicating that the source WAL file was written in between the two distcp runs either. > HBase Backup/Restore Phase 3: Multiwal support > ---------------------------------------------- > > Key: HBASE-14450 > URL: https://issues.apache.org/jira/browse/HBASE-14450 > Project: HBase > Issue Type: Task > Affects Versions: 2.0.0 > Reporter: Vladimir Rodionov > Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: 14450.v2.txt, org.apache.hadoop.hbase.backup.TestIncrementalBackup-output.txt > > > We need to support multiwal configurations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)