hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesse Yates (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5547) Don't delete HFiles when in "backup mode"
Date Wed, 18 Apr 2012 20:44:43 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256939#comment-13256939

Jesse Yates commented on HBASE-5547:

@Ted, stack: there is a race condition where the client thinks that backups are active, when
ZK hasn't in fact sent watch notifications to all the RS. Granted, this is a small time window
(hopefully), but is definitely a concern and something that I've seen take up to a few seconds
to propagate. It's probably not a big deal, but that is the concern and if you are doing tables
backups based on when archiving is enabled, you want to make sure archiving is in fact enabled
on all nodes.

If we encode the backup mode in the data associated with the table node, the region servers
would be able to register watcher for the table nodes 
Are you basically talking about doing per-table configuration storage in the table znode?
I think this would be completely viable with the transaction stuff - essentially check the
version and update the data if matches expected, if not propose new value and try again (see
below for paxos concerns, though this isn't actually all that rough from the client side).

If we had a dynamic Configuration system, one that didn't require roll of table to set the
table 'read-only' or 'in-back-up mode', would that help here?
That's essentially what I was building here, but just for state changes. Unfortunately, its
pretty close to Paxos (propsing new value, getting acceptance, confirming value update to
client), and has a lot of complexity to ensure changes propagate - the latter being the biggest
concern from a 'consistent view of the cluster by the client' standpoint.

For sure could poll zk the first time but should then cache the setting and only drop it later
if a callback says it changed.
If they are disabled, they need to check everytime to see if it has been enabled. Once it
has been enabled, it could then wait for the disable to propagate via notifications because
the excess backup files aren't a big deal, but the loss of backup files is an issue.
> Don't delete HFiles when in "backup mode"
> -----------------------------------------
>                 Key: HBASE-5547
>                 URL: https://issues.apache.org/jira/browse/HBASE-5547
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Lars Hofhansl
>            Assignee: Jesse Yates
> This came up in a discussion I had with Stack.
> It would be nice if HBase could be notified that a backup is in progress (via a znode
for example) and in that case either:
> 1. rename HFiles to be delete to <file>.bck
> 2. rename the HFiles into a special directory
> 3. rename them to a general trash directory (which would not need to be tied to backup
> That way it should be able to get a consistent backup based on HFiles (HDFS snapshots
or hard links would be better options here, but we do not have those).
> #1 makes cleanup a bit harder.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message