hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10900) FULL table backup and restore
Date Sun, 27 Apr 2014 00:41:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13982145#comment-13982145

Hadoop QA commented on HBASE-10900:

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  against trunk revision .
  ATTACHMENT ID: 12642095

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 34 warning messages.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 findbugs{color}.  The patch appears to introduce 18 new Findbugs (version
1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
    +      "Usage: hbase backup create <type> <backup_root_path> [tables] [-s
name] [-convert] [-silent]\n"
+          + "   tables      If no tables (\"\") are specified, all tables are backed up.
Otherwise it is a\n"
+      LOG.debug("Backup znode " + backupContext.getBackupId() + " has been removed from /ongoing/.");
+          + " does not exists, should have already been deleted from /ongoing/ during cleanup
for cancel.");
+    LOG.debug("Backup znode " + backupContext.getBackupId() + " has been created under /complete/.");
+      LOG.debug("Backup znode " + backupContext.getBackupId() + " has been removed from /ongoing/.");
+          + " does not exists, should have already been deleted from /ongoing/ during cleanup
for cancel.");
+          LOG.error("Failed to cancel the backup " + backupContext.getBackupId() + " due
to " + msg);
+    LOG.debug("Manifest file has been persisted to " + this.tableBackupDir != null ? this.tableBackupDir
+  protected static ZooKeeperWatcher getZKWatcher() throws ZooKeeperConnectionException, IOException

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9409//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9409//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9409//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9409//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9409//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9409//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9409//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9409//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9409//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9409//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9409//console

This message is automatically generated.

> FULL table backup and restore
> -----------------------------
>                 Key: HBASE-10900
>                 URL: https://issues.apache.org/jira/browse/HBASE-10900
>             Project: HBase
>          Issue Type: Task
>            Reporter: Demai Ni
>            Assignee: Demai Ni
>             Fix For: 1.0.0
>         Attachments: HBASE-10900-fullbackup-trunk-v1.patch, HBASE-10900-trunk-v2.patch
> h2. Feature Description
> This is a subtask of [HBase-7912|https://issues.apache.org/jira/browse/HBASE-7912] to
support FULL backup/restore, and will complete the following function:
> {code:title=Backup Restore example|borderStyle=solid}
> /* backup from sourcecluster to targetcluster                                  */
> /* if no table name specified, all tables from source cluster will be backuped */
> [sourcecluster]$ hbase backup create full hdfs://hostname.targetcluster.org:9000/userid/backupdir
> /* restore on targetcluser, this is a local restore                                 
> /* backup_1396650096738 - backup image name                                         
> /* t1_dn,etc are the original table names. All tables will be restored if not specified
> /* t1_dn_restore, etc. are the restored table. if not specified, orginal table name will
be used*/
> [targetcluster]$ hbase restore /userid/backupdir backup_1396650096738 t1_dn,t2_dn,t3_dn
> /* restore from targetcluster back to source cluster, this is a remote restore
> [sourcecluster]$ hbase restore hdfs://hostname.targetcluster.org:9000/userid/backupdir
backup_1396650096738 t1_dn,t2_dn,t3_dn t1_dn_restore,t2_dn_restore,t3_dn_restore
> {code}
> h2. Detail layout and frame work for the next jiras
> The patch is a wrapper of the existing snapshot and exportSnapshot, and will use as the
base framework for the over-all solution of  [HBase-7912|https://issues.apache.org/jira/browse/HBASE-7912]
as described below:
> * *bin/hbase*          : end-user command line interface to invoke BackupClient and RestoreClient
> * *BackupClient.java*  : 'main' entry for backup operations. This patch will only support
'full' backup. In future jiras, will support:
> ** *create* incremental backup
> ** *cancel* an ongoing backup
> ** *delete* an exisitng backup image
> ** *describe* the detailed informaiton of backup image
> ** show *history* of all successful backups 
> ** show the *status* of the latest backup request
> ** *convert* incremental backup WAL files into HFiles.  either on-the-fly during create
or after create
> ** *merge* backup image
> ** *stop* backup a table of existing backup image
> ** *show* tables of a backup image 
> * *BackupCommands.java* : a place to keep all the command usages and options
> * *BackupManager.java*  : handle backup requests on server-side, create BACKUP ZOOKEEPER
nodes to keep track backup. The timestamps kept in zookeeper will be used for future incremental
backup (not included in this jira). Create BackupContext and DispatchRequest. 
> * *BackupHandler.java*  : in this patch, it is a wrapper of snapshot and exportsnapshot.
In future jiras, 
> ** *timestamps* info will be recorded in ZK
> ** carry on *incremental* backup.  
> ** update backup *progress*
> ** set flags of *status*
> ** build up *backupManifest* file(in this jira only limited info for fullback. later
on, timestamps and dependency of multipl backup images are also recorded here)
> ** clean up after *failed* backup 
> ** clean up after *cancelled* backup
> ** allow on-the-fly *convert* during incremental backup 
> * *BackupContext.java* : encapsulate backup information like backup ID, table names,
directory info, phase, TimeStamps of backup progress, size of data, ancestor info, etc. 
> * *BackupCopier.java*  : the copying operation.  Later on, to support progress report
and mapper estimation; and extends DisCp for progress updating to ZK during backup. 
> * *BackupExcpetion.java*: to handle exception from backup/restore
> * *BackupManifest.java* : encapsulate all the backup image information. The manifest
info will be bundled as manifest file together with data. So that each backup image will contain
all the info needed for restore. 
> * *BackupStatus.java*   : encapsulate backup status at table level during backup progress
> * *BackupUtil.java*     : utility methods during backup process
> * *RestoreClient.java*  : 'main' entry for restore operations. This patch will only support
'full' backup. 
> * *RestoreUtil.java*    : utility methods during restore process
> * *ExportSnapshot.java* : remove 'final' so that another class SnapshotCopy.java can
extends from it
> * *SnapshotCopy.java*   : only a wrapper at this moment. But will be extended to keep
track progress(maybe should implemented in ExportSnapshot directly?)
> * *BackupRestoreConstants.java*     : add the constants used by backup/restore code.
> * *HBackupFilesystem.java*     :   the filesystem related api used by BackupClient and
> h2. Questions for your inputs
> * currently, the code is under *hbase-sever* because it already contain a package name
called 'backup'. Another place is *hbase-client*. Any prefereence?
> * Is bin/hbase the right place as CLI interface? another place would be hbase shell.

This message was sent by Atlassian JIRA

View raw message