hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5547) Don't delete HFiles when in "backup mode"
Date Thu, 19 Apr 2012 00:36:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257114#comment-13257114
] 

Lars Hofhansl commented on HBASE-5547:
--------------------------------------

What I had in mind when I filed this was something quite simple: A single znode that all RegionServers
would check when deleting any HFile and then based on existence or value of that znode either
delete the HFile or rename it. Of things are never quite as simple...

To get a guaranteed consistent snapshot the RegionServers need to check for the znode's value
synchronously in the delete path (or at least I see no other way). Otherwise there are times
when the RegionServers do not agree and some files will be deleted and some will be backed
up with no possibility for the client to know exactly as of when the backup would be consistent.

Since HFiles are deleted as result of a compaction in an asynchronous thread, synchronously
checking the znode should not cause performance issues, unless we fear we'll overload ZK.

This simple solution would add special code for just this scenario, which is bad. At the same
time it would be relatively simple (famous last words), so that's something to weigh.
                
> Don't delete HFiles when in "backup mode"
> -----------------------------------------
>
>                 Key: HBASE-5547
>                 URL: https://issues.apache.org/jira/browse/HBASE-5547
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Lars Hofhansl
>            Assignee: Jesse Yates
>
> This came up in a discussion I had with Stack.
> It would be nice if HBase could be notified that a backup is in progress (via a znode
for example) and in that case either:
> 1. rename HFiles to be delete to <file>.bck
> 2. rename the HFiles into a special directory
> 3. rename them to a general trash directory (which would not need to be tied to backup
mode).
> That way it should be able to get a consistent backup based on HFiles (HDFS snapshots
or hard links would be better options here, but we do not have those).
> #1 makes cleanup a bit harder.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message