hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
Date Fri, 10 Mar 2017 23:30:10 GMT
Thanks for the feedback, Andrew.

How about the following plan:

create branch HBASE-14123 off of master with mega patch v61 as the first
commit (reviewed by Stack and Enis)
Vlad and I continue development (the 3 blockers) based on HBASE-14123 branch
when all of the blockers get +1 and merged into HBASE-14123 branch, we
propose to community for merging into branch-2 (master branch, if branch-2
doesn't materialize for whatever reason) again

Cheers


On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell <apurtell@apache.org> wrote:

> Thanks for the offer but I like that you were honest about compiling a list
> of issues that you thought were blockers for release. Since this proposal
> is a merge into 2.0, and we are trying to release 2.0, I am -1 on this
> merge until those blockers are addressed.
>
> I had a look at the list.
>
> I think the documentation issue is important but not actually a blocker.
> That may be a controversial opinion, but documentation can be back-filled
> worst case. So take HBASE-17133 off the list.
>
> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. They
> all have patches attached to the respective JIRAs so completing this work
> won't be onerous. Get these committed and I will lift my -1. The others who
> voted +1 on this thread surely can help with that.
>
> Thanks.
>
>
> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov <vladrodionov@gmail.com
> >
> wrote:
>
> > No problem I will downgrade Blockers to Majors if it scares you, Andrew
> πŸ™‚
> >
> > Sent from my iPhone
> >
> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell <apurtell@apache.org>
> wrote:
> > >
> > > ​I know the merge of this feature has lagged substantially. I think
> that
> > is
> > > regrettable but on another thread we are lamenting that 2.0 is already
> > > late. Unless I misunderstand, this is a proposal to merge something
> with
> > > known blockers into trunk before we branch it for 2.0 which will
> > > effectively prevent that release because these blockers will be there.
> I
> > am
> > > inclined to veto. Probably we should not propose branch merges into
> code
> > we
> > > are trying to get out the door with known blockers. Why not do that
> work
> > > first? It seems an obvious question. Perhaps I am missing something.
> > >
> > > If we can branch for 2.0 now and then merge this, and not into the 2.0
> > > branch, I would vote +1 for branch merge even with known blockers
> > pending.
> > > ​
> > >
> > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov <
> > vladrodionov@gmail.com>
> > > wrote:
> > >
> > >> They are not blockers for merge - only for 2.0. GA
> > >> As I said already the feature is usable right now
> > >> We would like to continue working on master and we would like to see a
> > >> commitment from community
> > >>
> > >> Sent from my iPhone
> > >>
> > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell <apurtell@apache.org>
> > wrote:
> > >>
> > >>>> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
> > >>>
> > >>> If we have identified blockers, why merge this before they are in?
> > >>> Otherwise we can't release 2.0, and it is overdue.
> > >>>
> > >>>
> > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov <
> > >> vladrodionov@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> Hello, HBase folks
> > >>>>
> > >>>> For your consideration today is Backup/Restore feature for Apache
> > HBAse
> > >>>> 2.0.
> > >>>> Backup code is available as a mega patch in HBASE-14123 (v61),
> applies
> > >>>> cleanly to the current master, all test PASS, patch has no other
> > issues.
> > >>>>
> > >>>> The patch has gone through numerous rounds of code reviews and
has
> > >> probably
> > >>>> the most lengthy discussion thread on Apache JIRA (HBASE-14123)
:)
> > >>>>
> > >>>> The work has been split into 3 phases (HBASE-14030, 14123, 14414)
> Two
> > >> first
> > >>>> are complete, third one is still in progress.
> > >>>>
> > >>>>
> > >>>> *** Summary of work HBASE-14123
> > >>>>
> > >>>> The new feature introduces new command-line extensions to the hbase
> > >> command
> > >>>> and, from the client side, is accessible through command-line only
> > >>>> Operations:
> > >>>> * Create full backup on a list of tables or backup set
> > >>>> * Create incremental backup image for table list or backup set
> > >>>> * Restore list of tables from a given backup image
> > >>>> * Show current backup progress
> > >>>> * Delete backup image and all related images
> > >>>> * Show history of backups
> > >>>> * Backup set operations: create backup set, add/remove table to/from
> > >> backup
> > >>>> set, etc
> > >>>>
> > >>>> In the current implementation, the feature is already usable,
> meaning
> > >> that
> > >>>> users can backup tables and restore them using provided command-line
> > >> tools.
> > >>>> Both: full and incremental backups are supported.
> > >>>> This work is based on original work of IBM team (HBASE-7912). The
> full
> > >> list
> > >>>> of JIRAs included in this mega patch can be found in three umbrella
> > >> JIRAs:
> > >>>> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase
> 3
> > -
> > >> all
> > >>>> resolved ones made it into the patch)
> > >>>>
> > >>>> *** What are the remaining work items
> > >>>>
> > >>>> All remaining items can be found in Phase 3 umbrella JIRA:
> > HBASE-14414.
> > >>>> They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
> > >>>> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
> > >>>>
> > >>>> ***** BLOCKER
> > >>>>
> > >>>> * HBASE-14417 Incremental backup and bulk loading ( Patch available)
> > >>>> * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
> > >>>> * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup
to
> > >>>> include only edits from backup tables (Patch available)
> > >>>> * HBASE-17133 Backup documentation
> > >>>> * HBASE-15227 Fault tolerance support
> > >>>>
> > >>>> ***** CRITICAL
> > >>>>
> > >>>> * HBASE-16465 Disable split/merges during backup
> > >>>>
> > >>>> We have umbrella JIRA (HBASE-14414) to track all the remaining
work
> > >>>> All the BLOCKER and CRITICAL JIRAs currently in open state will
be
> > >>>> implemented by 2.0 release time. Some MAJOR too, but it depends
on
> > >> resource
> > >>>> availability
> > >>>> The former development branch (HBASE-7912) is obsolete and will
be
> > >>>> closed/deleted after the merge.
> > >>>> We want backup to be a GA feature in 2.0
> > >>>> We are going to support full backward compatibility for backup
tool
> in
> > >> 2.0
> > >>>> and onwards.
> > >>>>
> > >>>> **** Configuration
> > >>>>
> > >>>> Backup is disabled, by default. To enable it, the following
> > >> configuration
> > >>>> properties must be added to hbase-site.xml:
> > >>>>
> > >>>> hbase.backup.enable=true
> > >>>> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
> > >>>> apache.hadoop.hbase.backup.master.BackupLogCleaner
> > >>>> hbase.procedure.master.classes=YOUR_CLASSES,org.
> > >>>> apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
> > >>>> hbase.procedure.regionserver.classes=YOUR_CLASSES,org.
> > >>>> apache.hadoop.hbase.backup.regionserver.
> > LogRollRegionServerProcedureMa
> > >>>> nager
> > >>>>
> > >>>>
> > >>>> I would like to thank IBM team and Jerry He for original work,
> > >>>>
> > >>>> Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews
> > >>>>
> > >>>> Special thanks to Ted Yu for his co-development work.
> > >>>>
> > >>>> References:
> > >>>>
> > >>>> https://issues.apache.org/jira/browse/HBASE-7912 (original IBM,
> > >> contains
> > >>>> design doc)
> > >>>> https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1)
> > >>>> https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2)
> > >>>> https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3)
> > >>>>
> > >>>> Please  vote +1/-1 by midnight Pacific Time (00:00
> > >>>> -0800 GMT) on March 11th  ​on whether or not we should merge
this
> into
> > >> the
> > >>>> current master.
> > >>>>
> > >>>> -Vladimir Rodionov
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Best regards,
> > >>>
> > >>>  - Andy
> > >>>
> > >>> If you are given a choice, you believe you have acted freely. -
> Raymond
> > >>> Teller (via Peter Watts)
> > >>
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >   - Andy
> > >
> > > If you are given a choice, you believe you have acted freely. - Raymond
> > > Teller (via Peter Watts)
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> If you are given a choice, you believe you have acted freely. - Raymond
> Teller (via Peter Watts)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message