hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16006) FileSystem should be obtained from specified path in WALInputFormat#getSplits()
Date Fri, 10 Jun 2016 18:13:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324997#comment-15324997
] 

Hadoop QA commented on HBASE-16006:
-----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s {color} | {color:blue}
The patch file was not named according to hbase's naming conventions. Please see https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for instructions. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color} | {color:red}
HBASE-16006 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12809515/16006.v1.txt
|
| JIRA Issue | HBASE-16006 |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/2177/console |
| Powered by | Apache Yetus 0.2.1   http://yetus.apache.org |


This message was automatically generated.



> FileSystem should be obtained from specified path in WALInputFormat#getSplits()
> -------------------------------------------------------------------------------
>
>                 Key: HBASE-16006
>                 URL: https://issues.apache.org/jira/browse/HBASE-16006
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>              Labels: backup
>         Attachments: 16006.v1.txt
>
>
> I was trying out restore feature and encountered the following exception:
> {code}
> 2016-06-10 16:56:57,533 ERROR [main] impl.RestoreClientImpl: ERROR: restore failed with
error: java.io.IOException: Can not restore from backup directory hdfs://hbase-test-rc-rerun-6:8020/user/hbase/backup_1465575766499/WALs
(check Hadoop and HBase logs)
> java.io.IOException: java.io.IOException: Can not restore from backup directory hdfs://hbase-test-rc-rerun-6:8020/user/hbase/backup_1465575766499/WALs
(check Hadoop and HBase logs)
> 	at org.apache.hadoop.hbase.backup.impl.RestoreClientImpl.restoreStage(RestoreClientImpl.java:257)
> 	at org.apache.hadoop.hbase.backup.impl.RestoreClientImpl.restore(RestoreClientImpl.java:112)
> 	at org.apache.hadoop.hbase.backup.RestoreDriver.parseAndRun(RestoreDriver.java:169)
> 	at org.apache.hadoop.hbase.backup.RestoreDriver.doWork(RestoreDriver.java:198)
> 	at org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> 	at org.apache.hadoop.hbase.backup.RestoreDriver.main(RestoreDriver.java:203)
> Caused by: java.io.IOException: Can not restore from backup directory hdfs://hbase-test-rc-rerun-6:8020/user/hbase/backup_1465575766499/WALs
(check Hadoop and HBase logs)
> 	at org.apache.hadoop.hbase.backup.mapreduce.MapReduceRestoreService.run(MapReduceRestoreService.java:92)
> 	at org.apache.hadoop.hbase.backup.util.RestoreServerUtil.incrementalRestoreTable(RestoreServerUtil.java:165)
> 	at org.apache.hadoop.hbase.backup.impl.RestoreClientImpl.restoreImage(RestoreClientImpl.java:293)
> 	at org.apache.hadoop.hbase.backup.impl.RestoreClientImpl.restoreStage(RestoreClientImpl.java:238)
> 	... 6 more
> Caused by: java.lang.IllegalArgumentException: Wrong FS: hdfs://hbase-test-rc-rerun-6:8020/user/hbase/backup_1465575766499/WALs,
expected: hdfs://hbase-test-rc-rerun-6.openstacklocal:8020
> 	at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:658)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:212)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:882)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:112)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:951)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:947)
> 	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:947)
> 	at org.apache.hadoop.hbase.mapreduce.WALInputFormat.getFiles(WALInputFormat.java:266)
> 	at org.apache.hadoop.hbase.mapreduce.WALInputFormat.getSplits(WALInputFormat.java:246)
> 	at org.apache.hadoop.hbase.mapreduce.WALInputFormat.getSplits(WALInputFormat.java:227)
> 	at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
> 	at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
> 	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
> 	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
> 	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> 	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
> 	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
> 	at org.apache.hadoop.hbase.mapreduce.WALPlayer.run(WALPlayer.java:380)
> 	at org.apache.hadoop.hbase.backup.mapreduce.MapReduceRestoreService.run(MapReduceRestoreService.java:73)
> 	... 9 more
> {code}
> It turned out that the refactoring from HBASE-14140 changed the code:
> {code}
> -    FileSystem fs = inputDir.getFileSystem(conf);
> -    List<FileStatus> files = getFiles(fs, inputDir, startTime, endTime);
> -
> -    List<InputSplit> splits = new ArrayList<InputSplit>(files.size());
> -    for (FileStatus file : files) {
> +    FileSystem fs = FileSystem.get(conf);
> {code}
> We shouldn't be using the default FileSystem.
> Instead, FileSystem should be obtained from specified path.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message