hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matteo Bertozzi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14302) TableSnapshotInputFormat should not create back references when restoring snapshot
Date Mon, 24 Aug 2015 22:54:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14710220#comment-14710220

Matteo Bertozzi commented on HBASE-14302:

I'm not talking about RestoreSnapshotHelper constructor but the method used by the input format
+    // we send createBackRefs=false so that restored hfiles do not create back reference
+    // in the base hbase root dir.
     RestoreSnapshotHelper helper = new RestoreSnapshotHelper(conf, fs,
-      manifest, manifest.getTableDescriptor(), restoreDir, monitor, status);
+      manifest, manifest.getTableDescriptor(), restoreDir, monitor, status, false);

there are two cases of MR over snapshot. when your snapshot is hosted outside of the hbase
cluster where you should have createBackRefs=False and when you are on the hbase cluster reading
the cluster files, there should be true otherwise we don't have anything to prevent the user
to remove the snapshot and crash the MR job

> TableSnapshotInputFormat should not create back references when restoring snapshot
> ----------------------------------------------------------------------------------
>                 Key: HBASE-14302
>                 URL: https://issues.apache.org/jira/browse/HBASE-14302
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Enis Soztutar
>            Assignee: Enis Soztutar
>             Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3
>         Attachments: hbase-14302_v1.patch
> TableSnapshotInputFormat restores the snapshot to a temporary directory that is outside
the HBase's root directory so that it can read from the restored snapshot and also refer to
hfiles in the actual root directory. 
> In restoring the snapshot, we create new hfilelinks in the ephemeral location. Creating
new hfile links also creates "back references" for GC. These back references are dangling
since the restore location is outside the root dir, and requires a WRITE permission to the
root dir (as opposed to just READ permission) to be able to run the MR-over-snapshot job.

This message was sent by Atlassian JIRA

View raw message