hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5547) Don't delete HFiles when in "backup mode"
Date Thu, 12 Apr 2012 23:25:19 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252963#comment-13252963
] 

jiraposter@reviews.apache.org commented on HBASE-5547:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4633/#review6902
-----------------------------------------------------------


Patch looks good.  Some concerns in the below.


src/main/java/org/apache/hadoop/hbase/HConstants.java
<https://reviews.apache.org/r/4633/#comment15341>

    Do these defines need to be global?  Could they be package scoped?



src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
<https://reviews.apache.org/r/4633/#comment15342>

    What is one of these?  Might say so in here.



src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
<https://reviews.apache.org/r/4633/#comment15343>

    Good doc.



src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
<https://reviews.apache.org/r/4633/#comment15344>

    Are you missing a version that takes a String?



src/main/java/org/apache/hadoop/hbase/client/HFileArchiveManager.java
<https://reviews.apache.org/r/4633/#comment15345>

    Does it have to be public?  Can it be only of this package?



src/main/java/org/apache/hadoop/hbase/client/HFileArchiveManager.java
<https://reviews.apache.org/r/4633/#comment15346>

    Is there a close on this class so we can cleanup the zkw?



src/main/java/org/apache/hadoop/hbase/client/HFileArchiveManager.java
<https://reviews.apache.org/r/4633/#comment15347>

    Should this stuff be generally accessible?



src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
<https://reviews.apache.org/r/4633/#comment15348>

    We are in master package.  Why reach over into regionserver package?



src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
<https://reviews.apache.org/r/4633/#comment15349>

    Can hfilemanager be an interface so that when standing up a region to test, we don't have
to know much about this type?'
    
    hmm... I see it is an Interface.  Good.  Thanks.



src/main/java/org/apache/hadoop/hbase/master/HMaster.java
<https://reviews.apache.org/r/4633/#comment15350>

    This thing is watching a per table config?  Other per table configs are whether a table
is enabled or not.  Should we coalesce?  This is also a configuration maintained via another
system, one that persists to zk rather than to an hbase-site.xml or into an HTableDescriptor.
 Why not go into an htabledescriptor?  Nicolas just made it dynamically configurable though
the table is rolled



src/main/java/org/apache/hadoop/hbase/regionserver/HFileArchiveMonitor.java
<https://reviews.apache.org/r/4633/#comment15351>

    Does it need to be public?  I see master reaches over to get this class so for that its
needed but maybe it coudl be packaged elsewhere?



src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
<https://reviews.apache.org/r/4633/#comment15352>

    Do you have to stop this on way out?



src/main/java/org/apache/hadoop/hbase/zookeeper/HFileArchiveTracker.java
<https://reviews.apache.org/r/4633/#comment15354>

    Why a ref into regionserver package?



src/main/java/org/apache/hadoop/hbase/zookeeper/HFileArchiveTracker.java
<https://reviews.apache.org/r/4633/#comment15353>

    Whats this class do?
    
    Should it be abstract since others subclass it?
    
    Does it need to be public?



src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerHFileTracker.java
<https://reviews.apache.org/r/4633/#comment15355>

    Why these references?  Should this tracker be in regionserver package?  (Though seems
like we are collecting trackers here)



src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java
<https://reviews.apache.org/r/4633/#comment15356>

    I think its bad that we do this.
    
    Not your fault.


- Michael


On 2012-04-07 19:51:11, Jesse Yates wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4633/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-04-07 19:51:11)
bq.  
bq.  
bq.  Review request for hbase, Michael Stack and Lars Hofhansl.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Essentially, whenever an hfile would be deleted, it is instead moved to the archive directory.
In this impl, the archive directory is on a per table basis, but defaults to '.archive'. Removing
hfiles occurs in three places - compaction, merge and catalog janitor. The former and two
latter are distinctly different code paths, but but did pull out some similarities. The latter
two end up calling the same method, so there should be a reasonable amount of overlap.
bq.  
bq.  Implementation wise: 
bq.      Updated the HMasterInterface to pass the calls onto the zookeeper.
bq.      Added a zk listener to handle updates from the master to the RS to backup.
bq.      Added a utility for removing files and finding archive directories
bq.      Added tests for the regionserver and catalogjanitor approaches.
bq.      Added creation of manager in regionserver.
bq.  
bq.  
bq.  This addresses bug HBASE-5547.
bq.      https://issues.apache.org/jira/browse/HBASE-5547
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    src/main/java/org/apache/hadoop/hbase/HConstants.java a4b989e 
bq.    src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 
bq.    src/main/java/org/apache/hadoop/hbase/client/HFileArchiveManager.java PRE-CREATION

bq.    src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java 79d5fdd 
bq.    src/main/java/org/apache/hadoop/hbase/master/HMaster.java fb21bdd 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/HFileArchiveMonitor.java PRE-CREATION

bq.    src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c3df319 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6884d53

bq.    src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 509a467 
bq.    src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/HFileArchiveTracker.java PRE-CREATION

bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerHFileTracker.java PRE-CREATION

bq.    src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 4fc105f 
bq.    src/main/resources/hbase-default.xml 44ee689 
bq.    src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 
bq.    src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionHFileArchiving.java PRE-CREATION

bq.    src/test/java/org/apache/hadoop/hbase/util/HFileArchiveTestingUtil.java PRE-CREATION

bq.    src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java 7d02759 
bq.  
bq.  Diff: https://reviews.apache.org/r/4633/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Added two tests for the separate cases - archiving via the regionserver and for the catalog
tracker. Former runs in a mini cluster and also touches the changes to HMasterInterface and
zookeeper.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Jesse
bq.  
bq.


                
> Don't delete HFiles when in "backup mode"
> -----------------------------------------
>
>                 Key: HBASE-5547
>                 URL: https://issues.apache.org/jira/browse/HBASE-5547
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Lars Hofhansl
>            Assignee: Jesse Yates
>
> This came up in a discussion I had with Stack.
> It would be nice if HBase could be notified that a backup is in progress (via a znode
for example) and in that case either:
> 1. rename HFiles to be delete to <file>.bck
> 2. rename the HFiles into a special directory
> 3. rename them to a general trash directory (which would not need to be tied to backup
mode).
> That way it should be able to get a consistent backup based on HFiles (HDFS snapshots
or hard links would be better options here, but we do not have those).
> #1 makes cleanup a bit harder.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message