hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jan Van Besien (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-16953) ExportSnapshot -overwrite is not atomic (enough)
Date Thu, 27 Oct 2016 10:06:58 GMT

     [ https://issues.apache.org/jira/browse/HBASE-16953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jan Van Besien updated HBASE-16953:
    Attachment: HBASE-16953-avoid-window-without-snapshot.patch

Attached is a very rough patch to better explain what I mean.

In the patch, I use a UUID to temporary move the existing snapshot out of the way, without
loosing it. This works but I don't like it because it is difficult to implement an automated
snapshot-restore procedure on top of possible random snapshot names. Ideally I would want
a predictable suffix such that while restoring a snapshot I always know that I can either
find it with the name I was using or the name + a fixed suffix. However, choosing a fixed
suffix is of course problematic because people might already use that as part of their own
snapshot names. Ideas welcome.

> ExportSnapshot -overwrite is not atomic (enough)
> ------------------------------------------------
>                 Key: HBASE-16953
>                 URL: https://issues.apache.org/jira/browse/HBASE-16953
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Jan Van Besien
>         Attachments: HBASE-16953-avoid-window-without-snapshot.patch
> Use case: we use hbase snapshots and ExportSnapshot to regularly export snapshots to
another cluster. We want to have a snapshot ready on the destination cluster at all times.
> Currently this requires either to use new unique snapshot names every time (which is
cumbersome and also requires regular cleanup) or to use the '-overwrite' option of the ExportSnapshot
tool. However the -overwrite option is implemented such that the snapshot dir (in /hbase/.hbase-snapshot)
is deleted before copying the actual data files and only recreated (atomically renamed from
inside /hbase/.hbase-snapshot/.tmp) afterwards. This leaves a (large) window in which there
is no snapshot at all on the destination cluster.
> Wouldn't it be better to leave the old snapshot in place (perhaps renamed or moved in
a separate directory) all the way until after the atomic rename of the new snapshot. That
would ensure there is always a working snapshot when using ExportSnapshot for this use case.

This message was sent by Atlassian JIRA

View raw message