zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "maoling (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ZOOKEEPER-3318) Add a complete backup mechanism for zookeeper internal
Date Mon, 22 Apr 2019 12:19:00 GMT

     [ https://issues.apache.org/jira/browse/ZOOKEEPER-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

maoling updated ZOOKEEPER-3318:
-------------------------------
    Description: 
We already had some workaround ways for the backup, e.g:
scenario 1: just write a cron shell to copy the snapshots periodically. 
scenario 2: use the observer as the role of backup, then write the snapshots to distributed
file system. (e.g HDFS)

this issue is aiming to implement a complete backup mechanism for zookeeper internal:
the init propose:
1. for realtime backup.
write a new CLI:snapshot
1.1
[zk: 127.0.0.1:2180(CONNECTED) 0] snapshot backupDataDir
[zk: 127.0.0.1:2180(CONNECTED) 1] snapshot
 ***************************************************************************************************************
1.2 
if no parameter, the default backupDataDir is the dataDir. the format of the backup-snapshot
is just like: snapshot.f9f800002834 which is the same as the original one.
when recovering,moving the snapshot.f9f800002834 to the dataDir, then restart the ensemble.
1.3
don't worry about exposing the takeSnap() api to the client.Look at this two references:
https://github.com/etcd-io/etcd/blob/master/clientv3/snapshot/v3_snapshot.go
https://github.com/xetorthio/jedis/blob/master/src/main/java/redis/clients/jedis/commands/BasicCommands.java#L68
2. for no-realtime backup.
2.1 
write a new tool/shell: zkBackup.sh which is the reverse proces of the zkCleanup.sh for no-realtime
backup.

  was:
We already had some workaround ways for the backup, e.g
*scenario 1:* just write a cron shell to copy the snapshots periodically. 
*scenario 2:* use the observer as the role of backup, then write the snapshots to file system.
(e.g HDFS)

this issue is aiming to implement a complete backup mechanism for zookeeper internal:
the init propose:
*1*. write a new CLI:snapshot
 *1.1* 
 because this CLI may be time-consuming.A confirmation is needed. e.g.
 [zk: 127.0.0.1:2180(CONNECTED) 0] snapshot backupDataDir
 Are you sure to exec:snapshot [yes/no]
 *1.2* 
 if no parameter, the default backupDataDir is the dataDir. the format of the backup-snapshot
is just like: backup_snapshot.f9f800002834 with the "backup_" prefix,when recovering,rename
backup_snapshot.f9f800002834 to snapshot.f9f800002834 and move it to the dataDir, then restart
the ensemble.
 *1.3* 
 don't worry about exposing the takeSnap() api to the client.Look at this two references:
 https://github.com/etcd-io/etcd/blob/master/clientv3/snapshot/v3_snapshot.go
 https://github.com/xetorthio/jedis/blob/master/src/main/java/redis/clients/jedis/commands/BasicCommands.java#L68
*2*. 
 *2.1* 
 write a new tool/shell: zkBackup.sh which is the reverse proces of the zkCleanup.sh for no-realtime
backup
 *2.2* 
 write a new tool/shell: zkBackup_v2.sh which calls the api of the takeSnap() for realtime
backup.


> Add a complete backup mechanism for zookeeper internal
> ------------------------------------------------------
>
>                 Key: ZOOKEEPER-3318
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3318
>             Project: ZooKeeper
>          Issue Type: New Feature
>          Components: other
>            Reporter: maoling
>            Assignee: maoling
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> We already had some workaround ways for the backup, e.g:
> scenario 1: just write a cron shell to copy the snapshots periodically. 
> scenario 2: use the observer as the role of backup, then write the snapshots to distributed
file system. (e.g HDFS)
> this issue is aiming to implement a complete backup mechanism for zookeeper internal:
> the init propose:
> 1. for realtime backup.
> write a new CLI:snapshot
> 1.1
> [zk: 127.0.0.1:2180(CONNECTED) 0] snapshot backupDataDir
> [zk: 127.0.0.1:2180(CONNECTED) 1] snapshot
>  ***************************************************************************************************************
> 1.2 
> if no parameter, the default backupDataDir is the dataDir. the format of the backup-snapshot
is just like: snapshot.f9f800002834 which is the same as the original one.
> when recovering,moving the snapshot.f9f800002834 to the dataDir, then restart the ensemble.
> 1.3
> don't worry about exposing the takeSnap() api to the client.Look at this two references:
> https://github.com/etcd-io/etcd/blob/master/clientv3/snapshot/v3_snapshot.go
> https://github.com/xetorthio/jedis/blob/master/src/main/java/redis/clients/jedis/commands/BasicCommands.java#L68
> 2. for no-realtime backup.
> 2.1 
> write a new tool/shell: zkBackup.sh which is the reverse proces of the zkCleanup.sh for
no-realtime backup.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message