hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14450) HBase Backup/Restore Phase 3: Multiwal support
Date Fri, 12 Aug 2016 17:44:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419192#comment-15419192
] 

Ted Yu commented on HBASE-14450:
--------------------------------

With hadoop 2.7.3 patched with HADOOP-13496, I got:
{code}
2016-08-12 10:37:25,691 ERROR [LocalJobRunner Map Task Executor #0] util.RetriableCommand(89):
Failure in Retriable command: Copying hdfs://localhost:54465/user/tyu/test-data/4dba2cf9-4c45-4eaa-86cb-
  a84be3a21712/WALs/10.22.9.171,54473,1471023349505/10.22.9.171%2C54473%2C1471023349505.regiongroup-1.1471023351523
to hdfs://localhost:54465/backupUT/backup_1471023430951/WALs/10.22.9.                   171%2C54473%2C1471023349505.regiongroup-1.1471023351523
java.io.IOException: Mismatch in length of source:hdfs://localhost:54465/user/tyu/test-data/4dba2cf9-4c45-4eaa-86cb-a84be3a21712/WALs/10.22.9.171,54473,1471023349505/10.22.9.171%2C54473%2C1471023349505.regiongroup-1.1471023351523
and target:hdfs://localhost:54465/backupUT/backup_1471023430951/WALs/.distcp.tmp.attempt_local1883551993_0006_m_000000_0
- 83 : 973
  at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.compareFileLengths(RetriableFileCopyCommand.java:194)
  at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:126)
  at org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:99)
  at org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87)
  at org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:281)
  at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:253)
  at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50)
  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
  at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
{code}
There was no change in length of either source or dest file across the two distcp runs.
There was no log indicating that the source WAL file was written in between the two distcp
runs either.

> HBase Backup/Restore Phase 3: Multiwal support
> ----------------------------------------------
>
>                 Key: HBASE-14450
>                 URL: https://issues.apache.org/jira/browse/HBASE-14450
>             Project: HBase
>          Issue Type: Task
>    Affects Versions: 2.0.0
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>              Labels: backup
>             Fix For: 2.0.0
>
>         Attachments: 14450.v2.txt, org.apache.hadoop.hbase.backup.TestIncrementalBackup-output.txt
>
>
> We need to support multiwal configurations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message