hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesse Yates (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96
Date Wed, 26 Sep 2012 23:27:10 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464283#comment-13464283

Jesse Yates commented on HBASE-6055:

Another semi-unrelated note... currently we keep full logs files, and the restore needs to
split them (see the restore code SnapshotLogSplitter, https://github.com/matteobertozzi/hbase/blob/snapshot-dev/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/restore/RestoreSnapshotHelper.java#L398)
Can we move this logic at the end of the take snapshot operation and split the logs in .snapshot/region/recover.edits?

If we move it into the snapshot operation, then that will slow down the overall operation
and make it more difficult to reason about how long a snapshot 'should' take. In particular,
this becomes difficult because we want to give the client firm time bounds, but the log splitting
is not time bounded (AFAIK).  

An alternative would be to have a background snapshot-log-splitter task that just goes through
and splits logs for snapshots. It would basically comb though the snapshot directory, looking
for snapshots. If it finds one it hasn't seen, it starts doing the current log splitting on
that snapshot (which looks basically like the root directory of hbase - less the ROOT and
META tables - so it should be almost, if not entirely, drop-in useable). When the logs are
split, we would have to do a little extra checking to make sure that we don't restore a snapshot
mid-split, or that if we do that it handles it properly. 
> Snapshots in HBase 0.96
> -----------------------
>                 Key: HBASE-6055
>                 URL: https://issues.apache.org/jira/browse/HBASE-6055
>             Project: HBase
>          Issue Type: New Feature
>          Components: Client, master, regionserver, snapshots, Zookeeper
>            Reporter: Jesse Yates
>            Assignee: Jesse Yates
>             Fix For: hbase-6055, 0.96.0
>         Attachments: Snapshots in HBase.docx
> Continuation of HBASE-50 for the current trunk. Since the implementation has drastically
changed, opening as a new ticket.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message