hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Chongxin (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-50) Snapshot of table
Date Sun, 09 May 2010 06:32:55 GMT

    [ https://issues.apache.org/jira/browse/HBASE-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12865552#action_12865552

Li Chongxin commented on HBASE-50:

I'm working on this issue as my gsoc project right now. For the last few days, I've developed
a test for starting a snapshot via ZooKeeper. Attached are the source code and a flow chart
which depicts the whole process on master and region servers. Any feedbacks are welcome.

I develop and test this program under package src/test. I didn't extend the HBaseClusterTestCase
or HBaseTestCase because these classes are more like simulators for region server and master.
To execute the program, just put the files under right package, that is org.apache.hadoop.hbase.snapshot
under src/test. The main method is in class Master. 

I was trying to start/end the snapshot using a method like Double Barriers (http://hadoop.apache.org/zookeeper/docs/r3.0.0/recipes.html#sc_doubleBarriers).
But there are a lot of situations when run several region servers concurrently, even with
a small number of them. When some exceptions are taken into consideration, things become complex.
Here I just start three threads to simulate three region servers. Two test cases are provided
in the source code, one is to start the snapshot normally and the other is to start the snapshot
when one region server is down. It'll be great if you can test the program under some other
cases. I also have thinked about the situation where region server goes down during the processing
of snapshot, but no good solution yet.

> Snapshot of table
> -----------------
>                 Key: HBASE-50
>                 URL: https://issues.apache.org/jira/browse/HBASE-50
>             Project: Hadoop HBase
>          Issue Type: New Feature
>            Reporter: Billy Pearson
>            Assignee: Li Chongxin
>            Priority: Minor
> Havening an option to take a snapshot of a table would be vary useful in production.
> What I would like to see this option do is do a merge of all the data into one or more
files stored in the same folder on the dfs. This way we could save data in case of a software
bug in hadoop or user code. 
> The other advantage would be to be able to export a table to multi locations. Say I had
a read_only table that must be online. I could take a snapshot of it when needed and export
it to a separate data center and have it loaded there and then i would have it online at multi
data centers for load balancing and failover.
> I understand that hadoop takes the need out of havening backup to protect from failed
servers, but this does not protect use from software bugs that might delete or alter data
in ways we did not plan. We should have a way we can roll back a dataset.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message