hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "HBase Review Board (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-50) Snapshot of table
Date Thu, 19 Aug 2010 15:49:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900321#action_12900321
] 

HBase Review Board commented on HBASE-50:
-----------------------------------------

Message from: "Chongxin Li" <lichongxin@zju.edu.cn>

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/467/
-----------------------------------------------------------

(Updated 2010-08-19 08:35:37.043957)


Review request for hbase.


Summary
-------

This patch includes the first three sub-tasks of HBASE-50:
1. Start and monitor the creation of snapshot via ZooKeeper
2. Create snapshot of an HBase table
3. Some existing functions of HBase are modified to support snapshot

Currently snapshots can be created as expected, but can not be restored or deleted yet


This addresses bug HBASE-50.
    http://issues.apache.org/jira/browse/HBASE-50


Diffs (updated)
-----

  bin/add_snapshot_family.rb PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/HConstants.java c77ebf5 
  src/main/java/org/apache/hadoop/hbase/HRegionInfo.java ee94690 
  src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 0d57270 
  src/main/java/org/apache/hadoop/hbase/SnapshotDescriptor.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/SnapshotExistsException.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/TablePartiallyOpenException.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 8b01aa0 
  src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java ed12e7a 
  src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java 85fde3a 
  src/main/java/org/apache/hadoop/hbase/io/Reference.java 219203c 
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java b2de7e4 
  src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPCProtocolVersion.java d4bcbed 
  src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java bd48a4b 
  src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 1183584 
  src/main/java/org/apache/hadoop/hbase/master/BaseScanner.java 69eab39 
  src/main/java/org/apache/hadoop/hbase/master/DeleteSnapshot.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/master/HMaster.java e4bd30d 
  src/main/java/org/apache/hadoop/hbase/master/LogsCleaner.java 9d1a8b8 
  src/main/java/org/apache/hadoop/hbase/master/RestoreSnapshot.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/master/SnapshotLogCleaner.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/master/SnapshotMonitor.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/master/SnapshotOperation.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/master/SnapshotSentinel.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/master/TableDelete.java 1153e62 
  src/main/java/org/apache/hadoop/hbase/master/TableSnapshot.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 6dc41a4 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 6a54736 
  src/main/java/org/apache/hadoop/hbase/regionserver/Snapshotter.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java ae9e190 
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 757a50c 
  src/main/java/org/apache/hadoop/hbase/regionserver/ZKSnapshotWatcher.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 9593286 
  src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 4d4b00a

  src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 5cf3481 
  src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWrapper.java 3827fa5 
  src/main/resources/hbase-default.xml b73f0ff 
  src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 4d09fe9 
  src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java c9b78b9 
  src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 8b7f60f 
  src/test/java/org/apache/hadoop/hbase/master/TestSnapshot.java PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/master/TestSnapshotFailure.java PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java 34b8044 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java 98bd3e5 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionSnapshot.java PRE-CREATION

  src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java 38ef520 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestZKSnapshotWatcher.java PRE-CREATION


Diff: http://review.cloudera.org/r/467/diff


Testing
-------

Unit tests and integration tests with mini cluster passed.


Thanks,

Chongxin




> Snapshot of table
> -----------------
>
>                 Key: HBASE-50
>                 URL: https://issues.apache.org/jira/browse/HBASE-50
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Billy Pearson
>            Assignee: Li Chongxin
>            Priority: Minor
>         Attachments: HBase Snapshot Design Report V2.pdf, HBase Snapshot Design Report
V3.pdf, HBase Snapshot Implementation Plan.pdf, Snapshot Class Diagram.png
>
>
> Havening an option to take a snapshot of a table would be vary useful in production.
> What I would like to see this option do is do a merge of all the data into one or more
files stored in the same folder on the dfs. This way we could save data in case of a software
bug in hadoop or user code. 
> The other advantage would be to be able to export a table to multi locations. Say I had
a read_only table that must be online. I could take a snapshot of it when needed and export
it to a separate data center and have it loaded there and then i would have it online at multi
data centers for load balancing and failover.
> I understand that hadoop takes the need out of havening backup to protect from failed
servers, but this does not protect use from software bugs that might delete or alter data
in ways we did not plan. We should have a way we can roll back a dataset.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message