hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017
Date Tue, 14 Mar 2017 01:07:38 GMT
As of 6:06pm PST, I still saw these branches on
https://github.com/apache/hbase/

Maybe there is some delay.

Thanks Andrew.

On Mon, Mar 13, 2017 at 5:59 PM, Andrew Purtell <apurtell@apache.org> wrote:

> That's because this was pushed with a different name. It's gone now.
>
> apurtell@onyx:~/src/hbase$ git branch -a | grep 14123
>   remotes/origin/14123
> apurtell@onyx:~/src/hbase$ git push origin :14123
> Username for 'https://git-wip-us.apache.org': apurtell
> Password for 'https://apurtell@git-wip-us.apache.org':
> To https://git-wip-us.apache.org/repos/asf/hbase.git
>  - [deleted]         14123
>
>
> On Mon, Mar 13, 2017 at 4:13 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > I tried to delete the HBASE-14123 branch using commands I found on
> > http://stackoverflow.com/questions/2003505/how-to-
> > delete-a-git-branch-both-locally-and-remotely
> >
> > Not sure if there is lag on github side:
> >
> > $ git push origin :origin/HBASE-14123
> > error: unable to delete 'origin/HBASE-14123': remote ref does not exist
> > error: failed to push some refs to '
> > https://git-wip-us.apache.org/repos/asf/hbase.git'
> >
> > FYI
> >
> > On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar <enis@apache.org> wrote:
> >
> > > I think there is some misconception of using the term "blockers" for
> > > referring to those jiras. My understanding is that those three jiras
> are
> > > blockers for the backup functionality to be more mature and more
> usable.
> > > But they are not release blockers. Let's say we merged the code in, and
> > for
> > > some reason those did not get addressed in time. We can still do the
> 2.0
> > > release without having to wait for the commits. We can instead mark the
> > > "backup" feature as experimental with known issues and go on with the
> > > release. In that sense they are not real release blockers.
> > >
> > > We are proposing the merge at this time because of the above that
> > > maintaining this in a branch is becoming extremely costly and not
> > > productive for the HBase community. Realistically, we cannot have the
> > > luxury of having to wait another couple of months and doing yet another
> > > giant round of reviews because the code base is a moving target.
> > >
> > > Enis
> > >
> > > On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das <ddas@hortonworks.com>
> > wrote:
> > >
> > > > Vlad, on the first point, I think what Stack is saying is that
> creating
> > > > the new branch (as Ted did) ignores the feedback incorporated thus
> far
> > in
> > > > the iterations of the mega-patch. That's a wrong way to go.
> > > > On the separation into a backup module, again, that was reverted to
> > ease
> > > > reviews of the mega-patch, and was noted as work to be done later. I
> > > think
> > > > Stack just wants to make the list of remaining work more complete by
> > > citing
> > > > that as pending work.
> > > > ________________________________________
> > > > From: Vladimir Rodionov <vladrodionov@gmail.com>
> > > > Sent: Monday, March 13, 2017 3:09 PM
> > > > To: dev@hbase.apache.org
> > > > Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote
> closing
> > > > 3/11/2017
> > > >
> > > > >>  It ignores the feedback
> > > >
> > > > If I "ignore" feedback, I put my comment - why? I am always  open for
> > > > further discussions. If reviewer does not insist on a particular
> > request
> > > -
> > > > it will be dropped. I think it is fair.
> > > >
> > > > >> he list is incomplete because a bunch of
> > > > >> follow-ons came of the review cycle (including moving
> backup/restore
> > > out
> > > > of
> > > > >> core to live in its own module).
> > > >
> > > > For those who were not following our lengthy conversation on a review
> > > > board, separation of a backup code into a separate module has been
> > > > done last year, but has been reverted back by request of a reviewer.
> > > >
> > > >
> > > > -Vladimir
> > > >
> > > > On Mon, Mar 13, 2017 at 2:23 PM, Stack <stack@duboce.net> wrote:
> > > >
> > > > > On Fri, Mar 10, 2017 at 9:09 PM, Stack <stack@duboce.net> wrote:
> > > > >
> > > > > > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu <yuzhihong@gmail.com>
> > wrote:
> > > > > >
> > > > > >> HBASE-14123 branch has been created, with Vlad's mega patch
v61.
> > > > > >>
> > > > > >
> > > > > > The patch put up for VOTE here was done on a branch. The call
to
> > > merge
> > > > > > seems to have been premature given the many cycles of review
and
> > test
> > > > > that
> > > > > > happened subsequent (The cycles burned a bunch of dev resource).
> > > > > >
> > > > > > The patch as is is now in a state where it is too big for our
> > infra;
> > > rb
> > > > > > and JIRA are creaking under the size and # of iterations.
> > > > > >
> > > > > > Adding finish of new JIRAs to this merge implies a new round
of
> > > review
> > > > > and
> > > > > > test of an already massive patch. Who is going to do this work?
> > > > > >
> > > > > > Going back to a new branch seems wrong route to take.
> > > > > >
> > > > > > St.Ack
> > > > > >
> > > > > >
> > > > > >
> > > > > To be more explicit, this patch was developed on a branch and then
> a
> > > > bunch
> > > > > of dev resources were burned getting it into a state where it could
> > be
> > > > > merged to master. Going back to a branch to bulk up the merge so
it
> > > > > includes more JIRAs than the many it already incorporates is the
> > wrong
> > > > > direction for us to be headed in. It ignores the feedback given and
> > the
> > > > > work done by Vladimir slimming down an already over-broad scope.
It
> > is
> > > > also
> > > > > predicated on abundant review and testing resource being on tap to
> > > cycle
> > > > on
> > > > > a feature that is useful, but non-core.
> > > > >
> > > > > The patch is ready for merge IMO. Geoffrey makes a nice list of
> what
> > is
> > > > > still to do though IIRC, the list is incomplete because a bunch of
> > > > > follow-ons came of the review cycle (including moving
> backup/restore
> > > out
> > > > of
> > > > > core to live in its own module).
> > > > >
> > > > > The patch needs three votes to merge. I am not against merge but
I
> am
> > > not
> > > > > voting for the patch because I do have any more time to spend on
> this
> > > > > non-core feature and feel that a vote will have me assume a
> > > > responsibility
> > > > > I will not shirk.
> > > > >
> > > > > S
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >> FYI
> > > > > >>
> > > > > >> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu <yuzhihong@gmail.com>
> > > wrote:
> > > > > >>
> > > > > >> > Thanks for the feedback, Andrew.
> > > > > >> >
> > > > > >> > How about the following plan:
> > > > > >> >
> > > > > >> > create branch HBASE-14123 off of master with mega patch
v61 as
> > the
> > > > > first
> > > > > >> > commit (reviewed by Stack and Enis)
> > > > > >> > Vlad and I continue development (the 3 blockers) based
on
> > > > HBASE-14123
> > > > > >> >  branch
> > > > > >> > when all of the blockers get +1 and merged into HBASE-14123
> > > branch,
> > > > we
> > > > > >> > propose to community for merging into branch-2 (master
branch,
> > if
> > > > > >> branch-2
> > > > > >> > doesn't materialize for whatever reason) again
> > > > > >> >
> > > > > >> > Cheers
> > > > > >> >
> > > > > >> >
> > > > > >> > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell <
> > > > apurtell@apache.org>
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> >> Thanks for the offer but I like that you were honest
about
> > > > compiling
> > > > > a
> > > > > >> >> list
> > > > > >> >> of issues that you thought were blockers for release.
Since
> > this
> > > > > >> proposal
> > > > > >> >> is a merge into 2.0, and we are trying to release
2.0, I am
> -1
> > on
> > > > > this
> > > > > >> >> merge until those blockers are addressed.
> > > > > >> >>
> > > > > >> >> I had a look at the list.
> > > > > >> >>
> > > > > >> >> I think the documentation issue is important but
not
> actually a
> > > > > >> blocker.
> > > > > >> >> That may be a controversial opinion, but documentation
can be
> > > > > >> back-filled
> > > > > >> >> worst case. So take HBASE-17133 off the list.
> > > > > >> >>
> > > > > >> >> Remaining are effectively HBASE-14417, HBASE-14141,
and
> > > > HBASE-15227.
> > > > > >> They
> > > > > >> >> all have patches attached to the respective JIRAs
so
> completing
> > > > this
> > > > > >> work
> > > > > >> >> won't be onerous. Get these committed and I will
lift my -1.
> > The
> > > > > others
> > > > > >> >> who
> > > > > >> >> voted +1 on this thread surely can help with that.
> > > > > >> >>
> > > > > >> >> Thanks.
> > > > > >> >>
> > > > > >> >>
> > > > > >> >> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov
<
> > > > > >> >> vladrodionov@gmail.com>
> > > > > >> >> wrote:
> > > > > >> >>
> > > > > >> >> > No problem I will downgrade Blockers to Majors
if it scares
> > > you,
> > > > > >> Andrew
> > > > > >> >> 🙂
> > > > > >> >> >
> > > > > >> >> > Sent from my iPhone
> > > > > >> >> >
> > > > > >> >> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell
<
> > > > apurtell@apache.org
> > > > > >
> > > > > >> >> wrote:
> > > > > >> >> > >
> > > > > >> >> > > ​I know the merge of this feature has
lagged
> > substantially. I
> > > > > think
> > > > > >> >> that
> > > > > >> >> > is
> > > > > >> >> > > regrettable but on another thread we
are lamenting that
> 2.0
> > > is
> > > > > >> already
> > > > > >> >> > > late. Unless I misunderstand, this is
a proposal to merge
> > > > > something
> > > > > >> >> with
> > > > > >> >> > > known blockers into trunk before we branch
it for 2.0
> which
> > > > will
> > > > > >> >> > > effectively prevent that release because
these blockers
> > will
> > > be
> > > > > >> >> there. I
> > > > > >> >> > am
> > > > > >> >> > > inclined to veto. Probably we should
not propose branch
> > > merges
> > > > > into
> > > > > >> >> code
> > > > > >> >> > we
> > > > > >> >> > > are trying to get out the door with known
blockers. Why
> not
> > > do
> > > > > that
> > > > > >> >> work
> > > > > >> >> > > first? It seems an obvious question.
Perhaps I am missing
> > > > > >> something.
> > > > > >> >> > >
> > > > > >> >> > > If we can branch for 2.0 now and then
merge this, and not
> > > into
> > > > > the
> > > > > >> 2.0
> > > > > >> >> > > branch, I would vote +1 for branch merge
even with known
> > > > blockers
> > > > > >> >> > pending.
> > > > > >> >> > > ​
> > > > > >> >> > >
> > > > > >> >> > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir
Rodionov <
> > > > > >> >> > vladrodionov@gmail.com>
> > > > > >> >> > > wrote:
> > > > > >> >> > >
> > > > > >> >> > >> They are not blockers for merge -
only for 2.0. GA
> > > > > >> >> > >> As I said already the feature is
usable right now
> > > > > >> >> > >> We would like to continue working
on master and we would
> > > like
> > > > to
> > > > > >> see
> > > > > >> >> a
> > > > > >> >> > >> commitment from community
> > > > > >> >> > >>
> > > > > >> >> > >> Sent from my iPhone
> > > > > >> >> > >>
> > > > > >> >> > >> On Mar 10, 2017, at 11:16 AM, Andrew
Purtell <
> > > > > apurtell@apache.org
> > > > > >> >
> > > > > >> >> > wrote:
> > > > > >> >> > >>
> > > > > >> >> > >>>> Only BLOCKERs and CRITICALs
are guaranteed for HBase
> 2.0
> > > > > >> release.
> > > > > >> >> > >>>
> > > > > >> >> > >>> If we have identified blockers,
why merge this before
> > they
> > > > are
> > > > > >> in?
> > > > > >> >> > >>> Otherwise we can't release 2.0,
and it is overdue.
> > > > > >> >> > >>>
> > > > > >> >> > >>>
> > > > > >> >> > >>> On Wed, Mar 8, 2017 at 1:32 PM,
Vladimir Rodionov <
> > > > > >> >> > >> vladrodionov@gmail.com>
> > > > > >> >> > >>> wrote:
> > > > > >> >> > >>>
> > > > > >> >> > >>>> Hello, HBase folks
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> For your consideration today
is Backup/Restore feature
> > for
> > > > > >> Apache
> > > > > >> >> > HBAse
> > > > > >> >> > >>>> 2.0.
> > > > > >> >> > >>>> Backup code is available
as a mega patch in
> HBASE-14123
> > > > (v61),
> > > > > >> >> applies
> > > > > >> >> > >>>> cleanly to the current master,
all test PASS, patch
> has
> > no
> > > > > other
> > > > > >> >> > issues.
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> The patch has gone through
numerous rounds of code
> > reviews
> > > > and
> > > > > >> has
> > > > > >> >> > >> probably
> > > > > >> >> > >>>> the most lengthy discussion
thread on Apache JIRA
> > > > > (HBASE-14123)
> > > > > >> :)
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> The work has been split into
3 phases (HBASE-14030,
> > 14123,
> > > > > >> 14414)
> > > > > >> >> Two
> > > > > >> >> > >> first
> > > > > >> >> > >>>> are complete, third one is
still in progress.
> > > > > >> >> > >>>>
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> *** Summary of work HBASE-14123
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> The new feature introduces
new command-line extensions
> > to
> > > > the
> > > > > >> hbase
> > > > > >> >> > >> command
> > > > > >> >> > >>>> and, from the client side,
is accessible through
> > > > command-line
> > > > > >> only
> > > > > >> >> > >>>> Operations:
> > > > > >> >> > >>>> * Create full backup on a
list of tables or backup set
> > > > > >> >> > >>>> * Create incremental backup
image for table list or
> > backup
> > > > set
> > > > > >> >> > >>>> * Restore list of tables
from a given backup image
> > > > > >> >> > >>>> * Show current backup progress
> > > > > >> >> > >>>> * Delete backup image and
all related images
> > > > > >> >> > >>>> * Show history of backups
> > > > > >> >> > >>>> * Backup set operations:
create backup set, add/remove
> > > table
> > > > > >> >> to/from
> > > > > >> >> > >> backup
> > > > > >> >> > >>>> set, etc
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> In the current implementation,
the feature is already
> > > > usable,
> > > > > >> >> meaning
> > > > > >> >> > >> that
> > > > > >> >> > >>>> users can backup tables and
restore them using
> provided
> > > > > >> >> command-line
> > > > > >> >> > >> tools.
> > > > > >> >> > >>>> Both: full and incremental
backups are supported.
> > > > > >> >> > >>>> This work is based on original
work of IBM team
> > > > (HBASE-7912).
> > > > > >> The
> > > > > >> >> full
> > > > > >> >> > >> list
> > > > > >> >> > >>>> of JIRAs included in this
mega patch can be found in
> > three
> > > > > >> umbrella
> > > > > >> >> > >> JIRAs:
> > > > > >> >> > >>>> HBASE-14030 (Phase 1), HBASE-14123
(Phase 2) and
> > > HBASE-14414
> > > > > >> >> (Phase 3
> > > > > >> >> > -
> > > > > >> >> > >> all
> > > > > >> >> > >>>> resolved ones made it into
the patch)
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> *** What are the remaining
work items
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> All remaining items can be
found in Phase 3 umbrella
> > JIRA:
> > > > > >> >> > HBASE-14414.
> > > > > >> >> > >>>> They are split into 3 groups:
BLOCKER, CRITICAL, MAJOR
> > > > > >> >> > >>>> Only BLOCKERs and CRITICALs
are guaranteed for HBase
> 2.0
> > > > > >> release.
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> ***** BLOCKER
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> * HBASE-14417 Incremental
backup and bulk loading (
> > Patch
> > > > > >> >> available)
> > > > > >> >> > >>>> * HBASE-14135 HBase Backup/Restore
Phase 3: Merge
> backup
> > > > > images
> > > > > >> >> > >>>> * HBASE-14141 HBase Backup/Restore
Phase 3: Filter
> WALs
> > on
> > > > > >> backup
> > > > > >> >> to
> > > > > >> >> > >>>> include only edits from backup
tables (Patch
> available)
> > > > > >> >> > >>>> * HBASE-17133 Backup documentation
> > > > > >> >> > >>>> * HBASE-15227 Fault tolerance
support
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> ***** CRITICAL
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> * HBASE-16465 Disable split/merges
during backup
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> We have umbrella JIRA (HBASE-14414)
to track all the
> > > > remaining
> > > > > >> work
> > > > > >> >> > >>>> All the BLOCKER and CRITICAL
JIRAs currently in open
> > state
> > > > > will
> > > > > >> be
> > > > > >> >> > >>>> implemented by 2.0 release
time. Some MAJOR too, but
> it
> > > > > depends
> > > > > >> on
> > > > > >> >> > >> resource
> > > > > >> >> > >>>> availability
> > > > > >> >> > >>>> The former development branch
(HBASE-7912) is obsolete
> > and
> > > > > will
> > > > > >> be
> > > > > >> >> > >>>> closed/deleted after the
merge.
> > > > > >> >> > >>>> We want backup to be a GA
feature in 2.0
> > > > > >> >> > >>>> We are going to support full
backward compatibility
> for
> > > > backup
> > > > > >> >> tool in
> > > > > >> >> > >> 2.0
> > > > > >> >> > >>>> and onwards.
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> **** Configuration
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> Backup is disabled, by default.
To enable it, the
> > > following
> > > > > >> >> > >> configuration
> > > > > >> >> > >>>> properties must be added
to hbase-site.xml:
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> hbase.backup.enable=true
> > > > > >> >> > >>>> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
> > > > > >> >> > >>>> apache.hadoop.hbase.backup.master.BackupLogCleaner
> > > > > >> >> > >>>> hbase.procedure.master.classes=YOUR_CLASSES,org.
> > > > > >> >> > >>>> apache.hadoop.hbase.backup.master.
> > > > > LogRollMasterProcedureManager
> > > > > >> >> > >>>> hbase.procedure.regionserver.
> classes=YOUR_CLASSES,org.
> > > > > >> >> > >>>> apache.hadoop.hbase.backup.regionserver.
> > > > > >> >> > LogRollRegionServerProcedureMa
> > > > > >> >> > >>>> nager
> > > > > >> >> > >>>>
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> I would like to thank IBM
team and Jerry He for
> original
> > > > work,
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> Enis, Ted, Stack, Matteo,
Jerry for time spent on code
> > > > reviews
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> Special thanks to Ted Yu
for his co-development work.
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> References:
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-7912
> > > (original
> > > > > IBM,
> > > > > >> >> > >> contains
> > > > > >> >> > >>>> design doc)
> > > > > >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-14030
> > (Phase
> > > 1)
> > > > > >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-14123
> > (Phase
> > > 2)
> > > > > >> >> > >>>> https://issues.apache.org/jira/browse/HBASE-14414
> > (Phase
> > > 3)
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> Please  vote +1/-1 by midnight
Pacific Time (00:00
> > > > > >> >> > >>>> -0800 GMT) on March 11th
 ​on whether or not we should
> > > merge
> > > > > >> this
> > > > > >> >> into
> > > > > >> >> > >> the
> > > > > >> >> > >>>> current master.
> > > > > >> >> > >>>>
> > > > > >> >> > >>>> -Vladimir Rodionov
> > > > > >> >> > >>>>
> > > > > >> >> > >>>
> > > > > >> >> > >>>
> > > > > >> >> > >>>
> > > > > >> >> > >>> --
> > > > > >> >> > >>> Best regards,
> > > > > >> >> > >>>
> > > > > >> >> > >>>  - Andy
> > > > > >> >> > >>>
> > > > > >> >> > >>> If you are given a choice, you
believe you have acted
> > > > freely. -
> > > > > >> >> Raymond
> > > > > >> >> > >>> Teller (via Peter Watts)
> > > > > >> >> > >>
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > > --
> > > > > >> >> > > Best regards,
> > > > > >> >> > >
> > > > > >> >> > >   - Andy
> > > > > >> >> > >
> > > > > >> >> > > If you are given a choice, you believe
you have acted
> > > freely. -
> > > > > >> >> Raymond
> > > > > >> >> > > Teller (via Peter Watts)
> > > > > >> >> >
> > > > > >> >>
> > > > > >> >>
> > > > > >> >>
> > > > > >> >> --
> > > > > >> >> Best regards,
> > > > > >> >>
> > > > > >> >>    - Andy
> > > > > >> >>
> > > > > >> >> If you are given a choice, you believe you have
acted
> freely. -
> > > > > Raymond
> > > > > >> >> Teller (via Peter Watts)
> > > > > >> >>
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> If you are given a choice, you believe you have acted freely. - Raymond
> Teller (via Peter Watts)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message