hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vladrodio...@gmail.com>
Subject Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
Date Mon, 13 Mar 2017 22:09:39 GMT
>>  It ignores the feedback

If I "ignore" feedback, I put my comment - why? I am always  open for
further discussions. If reviewer does not insist on a particular request -
it will be dropped. I think it is fair.

>> he list is incomplete because a bunch of
>> follow-ons came of the review cycle (including moving backup/restore out
of
>> core to live in its own module).

For those who were not following our lengthy conversation on a review
board, separation of a backup code into a separate module has been
done last year, but has been reverted back by request of a reviewer.


-Vladimir

On Mon, Mar 13, 2017 at 2:23 PM, Stack <stack@duboce.net> wrote:

> On Fri, Mar 10, 2017 at 9:09 PM, Stack <stack@duboce.net> wrote:
>
> > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> >> HBASE-14123 branch has been created, with Vlad's mega patch v61.
> >>
> >
> > The patch put up for VOTE here was done on a branch. The call to merge
> > seems to have been premature given the many cycles of review and test
> that
> > happened subsequent (The cycles burned a bunch of dev resource).
> >
> > The patch as is is now in a state where it is too big for our infra; rb
> > and JIRA are creaking under the size and # of iterations.
> >
> > Adding finish of new JIRAs to this merge implies a new round of review
> and
> > test of an already massive patch. Who is going to do this work?
> >
> > Going back to a new branch seems wrong route to take.
> >
> > St.Ack
> >
> >
> >
> To be more explicit, this patch was developed on a branch and then a bunch
> of dev resources were burned getting it into a state where it could be
> merged to master. Going back to a branch to bulk up the merge so it
> includes more JIRAs than the many it already incorporates is the wrong
> direction for us to be headed in. It ignores the feedback given and the
> work done by Vladimir slimming down an already over-broad scope. It is also
> predicated on abundant review and testing resource being on tap to cycle on
> a feature that is useful, but non-core.
>
> The patch is ready for merge IMO. Geoffrey makes a nice list of what is
> still to do though IIRC, the list is incomplete because a bunch of
> follow-ons came of the review cycle (including moving backup/restore out of
> core to live in its own module).
>
> The patch needs three votes to merge. I am not against merge but I am not
> voting for the patch because I do have any more time to spend on this
> non-core feature and feel that a vote will have me assume a responsibility
> I will not shirk.
>
> S
>
>
>
> >
> >
> >
> >> FYI
> >>
> >> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >>
> >> > Thanks for the feedback, Andrew.
> >> >
> >> > How about the following plan:
> >> >
> >> > create branch HBASE-14123 off of master with mega patch v61 as the
> first
> >> > commit (reviewed by Stack and Enis)
> >> > Vlad and I continue development (the 3 blockers) based on HBASE-14123
> >> >  branch
> >> > when all of the blockers get +1 and merged into HBASE-14123 branch, we
> >> > propose to community for merging into branch-2 (master branch, if
> >> branch-2
> >> > doesn't materialize for whatever reason) again
> >> >
> >> > Cheers
> >> >
> >> >
> >> > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell <apurtell@apache.org>
> >> > wrote:
> >> >
> >> >> Thanks for the offer but I like that you were honest about compiling
> a
> >> >> list
> >> >> of issues that you thought were blockers for release. Since this
> >> proposal
> >> >> is a merge into 2.0, and we are trying to release 2.0, I am -1 on
> this
> >> >> merge until those blockers are addressed.
> >> >>
> >> >> I had a look at the list.
> >> >>
> >> >> I think the documentation issue is important but not actually a
> >> blocker.
> >> >> That may be a controversial opinion, but documentation can be
> >> back-filled
> >> >> worst case. So take HBASE-17133 off the list.
> >> >>
> >> >> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227.
> >> They
> >> >> all have patches attached to the respective JIRAs so completing this
> >> work
> >> >> won't be onerous. Get these committed and I will lift my -1. The
> others
> >> >> who
> >> >> voted +1 on this thread surely can help with that.
> >> >>
> >> >> Thanks.
> >> >>
> >> >>
> >> >> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov <
> >> >> vladrodionov@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > No problem I will downgrade Blockers to Majors if it scares you,
> >> Andrew
> >> >> πŸ™‚
> >> >> >
> >> >> > Sent from my iPhone
> >> >> >
> >> >> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell <apurtell@apache.org
> >
> >> >> wrote:
> >> >> > >
> >> >> > > ​I know the merge of this feature has lagged substantially.
I
> think
> >> >> that
> >> >> > is
> >> >> > > regrettable but on another thread we are lamenting that 2.0
is
> >> already
> >> >> > > late. Unless I misunderstand, this is a proposal to merge
> something
> >> >> with
> >> >> > > known blockers into trunk before we branch it for 2.0 which
will
> >> >> > > effectively prevent that release because these blockers will
be
> >> >> there. I
> >> >> > am
> >> >> > > inclined to veto. Probably we should not propose branch merges
> into
> >> >> code
> >> >> > we
> >> >> > > are trying to get out the door with known blockers. Why not
do
> that
> >> >> work
> >> >> > > first? It seems an obvious question. Perhaps I am missing
> >> something.
> >> >> > >
> >> >> > > If we can branch for 2.0 now and then merge this, and not
into
> the
> >> 2.0
> >> >> > > branch, I would vote +1 for branch merge even with known
blockers
> >> >> > pending.
> >> >> > > ​
> >> >> > >
> >> >> > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov <
> >> >> > vladrodionov@gmail.com>
> >> >> > > wrote:
> >> >> > >
> >> >> > >> They are not blockers for merge - only for 2.0. GA
> >> >> > >> As I said already the feature is usable right now
> >> >> > >> We would like to continue working on master and we would
like to
> >> see
> >> >> a
> >> >> > >> commitment from community
> >> >> > >>
> >> >> > >> Sent from my iPhone
> >> >> > >>
> >> >> > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell <
> apurtell@apache.org
> >> >
> >> >> > wrote:
> >> >> > >>
> >> >> > >>>> Only BLOCKERs and CRITICALs are guaranteed for
HBase 2.0
> >> release.
> >> >> > >>>
> >> >> > >>> If we have identified blockers, why merge this before
they are
> >> in?
> >> >> > >>> Otherwise we can't release 2.0, and it is overdue.
> >> >> > >>>
> >> >> > >>>
> >> >> > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov
<
> >> >> > >> vladrodionov@gmail.com>
> >> >> > >>> wrote:
> >> >> > >>>
> >> >> > >>>> Hello, HBase folks
> >> >> > >>>>
> >> >> > >>>> For your consideration today is Backup/Restore
feature for
> >> Apache
> >> >> > HBAse
> >> >> > >>>> 2.0.
> >> >> > >>>> Backup code is available as a mega patch in HBASE-14123
(v61),
> >> >> applies
> >> >> > >>>> cleanly to the current master, all test PASS,
patch has no
> other
> >> >> > issues.
> >> >> > >>>>
> >> >> > >>>> The patch has gone through numerous rounds of
code reviews and
> >> has
> >> >> > >> probably
> >> >> > >>>> the most lengthy discussion thread on Apache
JIRA
> (HBASE-14123)
> >> :)
> >> >> > >>>>
> >> >> > >>>> The work has been split into 3 phases (HBASE-14030,
14123,
> >> 14414)
> >> >> Two
> >> >> > >> first
> >> >> > >>>> are complete, third one is still in progress.
> >> >> > >>>>
> >> >> > >>>>
> >> >> > >>>> *** Summary of work HBASE-14123
> >> >> > >>>>
> >> >> > >>>> The new feature introduces new command-line extensions
to the
> >> hbase
> >> >> > >> command
> >> >> > >>>> and, from the client side, is accessible through
command-line
> >> only
> >> >> > >>>> Operations:
> >> >> > >>>> * Create full backup on a list of tables or backup
set
> >> >> > >>>> * Create incremental backup image for table list
or backup set
> >> >> > >>>> * Restore list of tables from a given backup
image
> >> >> > >>>> * Show current backup progress
> >> >> > >>>> * Delete backup image and all related images
> >> >> > >>>> * Show history of backups
> >> >> > >>>> * Backup set operations: create backup set, add/remove
table
> >> >> to/from
> >> >> > >> backup
> >> >> > >>>> set, etc
> >> >> > >>>>
> >> >> > >>>> In the current implementation, the feature is
already usable,
> >> >> meaning
> >> >> > >> that
> >> >> > >>>> users can backup tables and restore them using
provided
> >> >> command-line
> >> >> > >> tools.
> >> >> > >>>> Both: full and incremental backups are supported.
> >> >> > >>>> This work is based on original work of IBM team
(HBASE-7912).
> >> The
> >> >> full
> >> >> > >> list
> >> >> > >>>> of JIRAs included in this mega patch can be found
in three
> >> umbrella
> >> >> > >> JIRAs:
> >> >> > >>>> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2)
and HBASE-14414
> >> >> (Phase 3
> >> >> > -
> >> >> > >> all
> >> >> > >>>> resolved ones made it into the patch)
> >> >> > >>>>
> >> >> > >>>> *** What are the remaining work items
> >> >> > >>>>
> >> >> > >>>> All remaining items can be found in Phase 3 umbrella
JIRA:
> >> >> > HBASE-14414.
> >> >> > >>>> They are split into 3 groups: BLOCKER, CRITICAL,
MAJOR
> >> >> > >>>> Only BLOCKERs and CRITICALs are guaranteed for
HBase 2.0
> >> release.
> >> >> > >>>>
> >> >> > >>>> ***** BLOCKER
> >> >> > >>>>
> >> >> > >>>> * HBASE-14417 Incremental backup and bulk loading
( Patch
> >> >> available)
> >> >> > >>>> * HBASE-14135 HBase Backup/Restore Phase 3: Merge
backup
> images
> >> >> > >>>> * HBASE-14141 HBase Backup/Restore Phase 3: Filter
WALs on
> >> backup
> >> >> to
> >> >> > >>>> include only edits from backup tables (Patch
available)
> >> >> > >>>> * HBASE-17133 Backup documentation
> >> >> > >>>> * HBASE-15227 Fault tolerance support
> >> >> > >>>>
> >> >> > >>>> ***** CRITICAL
> >> >> > >>>>
> >> >> > >>>> * HBASE-16465 Disable split/merges during backup
> >> >> > >>>>
> >> >> > >>>> We have umbrella JIRA (HBASE-14414) to track
all the remaining
> >> work
> >> >> > >>>> All the BLOCKER and CRITICAL JIRAs currently
in open state
> will
> >> be
> >> >> > >>>> implemented by 2.0 release time. Some MAJOR too,
but it
> depends
> >> on
> >> >> > >> resource
> >> >> > >>>> availability
> >> >> > >>>> The former development branch (HBASE-7912) is
obsolete and
> will
> >> be
> >> >> > >>>> closed/deleted after the merge.
> >> >> > >>>> We want backup to be a GA feature in 2.0
> >> >> > >>>> We are going to support full backward compatibility
for backup
> >> >> tool in
> >> >> > >> 2.0
> >> >> > >>>> and onwards.
> >> >> > >>>>
> >> >> > >>>> **** Configuration
> >> >> > >>>>
> >> >> > >>>> Backup is disabled, by default. To enable it,
the following
> >> >> > >> configuration
> >> >> > >>>> properties must be added to hbase-site.xml:
> >> >> > >>>>
> >> >> > >>>> hbase.backup.enable=true
> >> >> > >>>> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
> >> >> > >>>> apache.hadoop.hbase.backup.master.BackupLogCleaner
> >> >> > >>>> hbase.procedure.master.classes=YOUR_CLASSES,org.
> >> >> > >>>> apache.hadoop.hbase.backup.master.
> LogRollMasterProcedureManager
> >> >> > >>>> hbase.procedure.regionserver.classes=YOUR_CLASSES,org.
> >> >> > >>>> apache.hadoop.hbase.backup.regionserver.
> >> >> > LogRollRegionServerProcedureMa
> >> >> > >>>> nager
> >> >> > >>>>
> >> >> > >>>>
> >> >> > >>>> I would like to thank IBM team and Jerry He for
original work,
> >> >> > >>>>
> >> >> > >>>> Enis, Ted, Stack, Matteo, Jerry for time spent
on code reviews
> >> >> > >>>>
> >> >> > >>>> Special thanks to Ted Yu for his co-development
work.
> >> >> > >>>>
> >> >> > >>>> References:
> >> >> > >>>>
> >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-7912
(original
> IBM,
> >> >> > >> contains
> >> >> > >>>> design doc)
> >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-14030
(Phase 1)
> >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-14123
(Phase 2)
> >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-14414
(Phase 3)
> >> >> > >>>>
> >> >> > >>>> Please  vote +1/-1 by midnight Pacific Time (00:00
> >> >> > >>>> -0800 GMT) on March 11th  ​on whether or not
we should merge
> >> this
> >> >> into
> >> >> > >> the
> >> >> > >>>> current master.
> >> >> > >>>>
> >> >> > >>>> -Vladimir Rodionov
> >> >> > >>>>
> >> >> > >>>
> >> >> > >>>
> >> >> > >>>
> >> >> > >>> --
> >> >> > >>> Best regards,
> >> >> > >>>
> >> >> > >>>  - Andy
> >> >> > >>>
> >> >> > >>> If you are given a choice, you believe you have acted
freely. -
> >> >> Raymond
> >> >> > >>> Teller (via Peter Watts)
> >> >> > >>
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > > --
> >> >> > > Best regards,
> >> >> > >
> >> >> > >   - Andy
> >> >> > >
> >> >> > > If you are given a choice, you believe you have acted freely.
-
> >> >> Raymond
> >> >> > > Teller (via Peter Watts)
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best regards,
> >> >>
> >> >>    - Andy
> >> >>
> >> >> If you are given a choice, you believe you have acted freely. -
> Raymond
> >> >> Teller (via Peter Watts)
> >> >>
> >> >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message