Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <CADcMMgGbeanFsmQsCE6MvdEbX0Y=544m1YcQK=cJSd9xZChDWw@mail.gmail.com>
References: <CAMUu0w9mX-=zb9exu5J_hoBBei8Hwb3vAKBa_DD+KNp+X7V7sw@mail.gmail.com>
 <CADcMMgGbeanFsmQsCE6MvdEbX0Y=544m1YcQK=cJSd9xZChDWw@mail.gmail.com>
From: Vladimir Rodionov <vladrodionov@gmail.com>
Date: Wed, 20 Jul 2016 09:43:46 -0700
Message-ID: <CAAg3a2oW8t=TaZzeBWVbzcRFaBB=NCE_bRBQYwoO4p+qWx0Z6g@mail.gmail.com>
Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a113f900cc90375053813e7b1
archived-at: Wed, 20 Jul 2016 16:43:59 -0000

--001a113f900cc90375053813e7b1
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Stack,

This is brief description of phases:

Phase 1 and Phase 2 :

1. Support for full backup of a table, list of tables
2. Support for incremental backup of a table, list of tables
3. Restore table(s) from backup (full or incremental)
4. HBase Backup API - via BackupAdmin
5. Command - line interface to backup / restore
6. Set of command supported: hbase backup create, restore, history, delete,
describe, progress
7. Backup sets (one can create backup set - named collection of tables)
support.

This gives user fully functional backup/restore tool set.

Phase 3 (major ones):
Umbrella HBASE-14414

1. Support for bulk loaded data in incremental backups
2. Security
3. Various performance optimizations during backup/restore
4. Merge of backup images
5. Filter WALs on incremental backup
6. Fault tolerance
7. Multiple backup destinations
8. MultiWAL support

-Vlad


On Wed, Jul 20, 2016 at 9:14 AM, Stack <stack@duboce.net> wrote:

> On Mon, Jul 11, 2016 at 12:21 PM, Enis S=C3=B6ztutar <enis@apache.org> wr=
ote:
>
> > As you guys may already be familiar, Vladimir, Ted, Jerry and others ha=
ve
> > been developing the backup / restore functionality in a series of issue=
s
> > committed in the separate branch HBASE-7912[1].
> >
> > Backup / Restore functionality is tracked as a 4-phase project, and the
> > first two phases are complete and useable. We are now working on Phase =
3
> > items, which are mostly improvements. We think that the current code in
> the
> > branch containing all Phase 1 and Phase 2 items, and some Phase 3 items
> is
> > useable on it's own, and we do not have to wait for all the subtickets =
to
> > be finished to make it completely useable (as follow up tickets are
> mostly
> > improvements or optimizations). The improvements in the works are all
> > backwards compatible with the existing stuff. Thus, we would like to
> > propose that the branch HBASE-7912 be merged into master.  The parent
> jira
> > has a design doc that goes into details about the implementation and
> design
> > choices in case you are interested[2].
> >
> >
> Thanks Enis.
>
> I could read the design doc but it would be nice to have a succinct summa=
ry
> of what we get with phase 1 and phase 2 of this feature, especially here =
on
> this thread, and then what is coming in phase 3.
>
> Thanks,
> St.Ack
>
>
> > Most of the changes are largely non-intrusive and confined to the
> > backup subsystem.
> > The unit tests have been passing on manual runs and we (hortonworks) ha=
ve
> > been running the integration tests as well as some other shell-based
> system
> > tests on a forked version of the code. Most of the work has been review=
ed
> > by 1, 2 or 3 committers already (mostly Ted, myself and Jerry).
> >
> > What do you guys think? Is it time to call a vote? Any concerns or
> feedback
> > appreciated.
> >
> > [1] https://issues.apache.org/jira/browse/HBASE-7912
> > [2]
> >
> >
> https://issues.apache.org/jira/secure/attachment/12816339/HBaseBackupAndR=
estore%20-0.91.pdf
> >
> > Enis
> >
>

--001a113f900cc90375053813e7b1--