Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 22 Sep 2016 00:38:20 +0000 (UTC)
From: "Jerry He (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13006663.1474494100000.632794.1474504700620@Atlassian.JIRA>
In-Reply-To: <JIRA.13006663.1474494100000@Atlassian.JIRA>
References: <JIRA.13006663.1474494100000@Atlassian.JIRA> <JIRA.13006663.1474494100178@arcas>
Subject: [jira] [Commented] (HBASE-16672) Add option for bulk load to copy
 hfile(s) instead of renaming
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 22 Sep 2016 00:38:22 -0000


    [ https://issues.apache.org/jira/browse/HBASE-16672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15511645#comment-15511645 ] 

Jerry He commented on HBASE-16672:
----------------------------------

Hi, Ted

The stack trace looks confusing.
1. Why incremental restore was bulk loading the file paths that are in an existing table?  The incremental restore should be restoring files from the backup image. The files in the backup image may be coming from the existing table originally. But the files should be physically in the backup image.  This is different from the snapshot restore, where it is a in-place restore.
2. What the snapshot failed with the missing file?

An bulk load option to keep the original files (not move/rename them to target) seems a fine option. We can simplify the JIRA description to describe just that, without the confusing stack trace.

> Add option for bulk load to copy hfile(s) instead of renaming
> -------------------------------------------------------------
>
>                 Key: HBASE-16672
>                 URL: https://issues.apache.org/jira/browse/HBASE-16672
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>         Attachments: 16672.v1.txt, 16672.v2.txt, 16672.v3.txt, 16672.v4.txt
>
>
> While working on HBASE-14417, I found that taking full backup of table which received bulk loaded hfiles and later gone through incremental restore failed with the following exception:
> {code}
> 2016-09-21 11:33:06,340 WARN  [member: '10.22.9.171,53915,1474482617015' subprocedure-pool7-thread-1] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(347): Got Exception in SnapshotSubprocedurePool
> java.util.concurrent.ExecutionException: java.io.FileNotFoundException: File does not exist: hdfs://localhost:53901/user/tyu/test-data/48928d44-9757-4923-af29-60288fb8a553/data/ns1/test-1474482646700/                        d1a93d05a75c596533ba4331a378fb3a/f/3dfb3a37cf6b4b519769b3116d8ab35a_SeqId_205_
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>   at org.apache.hadoop.hbase.regionserver.snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool.waitForOutstandingTasks(RegionServerSnapshotManager.java:323)
>   at org.apache.hadoop.hbase.regionserver.snapshot.FlushSnapshotSubprocedure.flushSnapshot(FlushSnapshotSubprocedure.java:139)
>   at org.apache.hadoop.hbase.regionserver.snapshot.FlushSnapshotSubprocedure.insideBarrier(FlushSnapshotSubprocedure.java:159)
> {code}
> The cause was that the bulk loaded hfiles in source table were renamed during bulk load step of incremental restore.
> To support incrementally restoring to multiple destinations, this issue adds option which would always copy hfile(s) during bulk load.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)