hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vladrodio...@gmail.com>
Subject Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912
Date Sun, 10 Sep 2017 00:25:30 GMT
>> but the impression we have is it is unfinished and untested.
To make a conclusion that "feature is not finished and tested"  you have
had to test it at least.
Andrew, If you have discovered issues, why wouldn't you open bug JIRAs?

-Vlad

On Sat, Sep 9, 2017 at 4:40 PM, Andrew Purtell <andrew.purtell@gmail.com>
wrote:

> For what it's worth, I think AMv2 is the main reason to have a 2.0 in the
> first place, so I would both agree it needs a lot more testing and yet I
> would want us to have a 2.0 release as the vehicle for getting that to
> happen. For other features without testing from a number of parties or at
> scale the value proposition is less clear and it's fine by me for the RM to
> set them aside for future releases.
>
> Also, I can relay that there is some interest where I work in utilizing
> HBASE-7912 but the impression we have is it is unfinished and untested. So
> for now we are ignoring it and continuing with home grown solutions. Part
> of the problem is fault tolerance was left to the last phase(s) and yet it
> is an essential property for adoption for serious work. The best way to
> resolve this IMHO is for the developers of this feature to complete those
> unfinished JIRAs, especially concerning resilience to failures.
>
>
> > On Sep 9, 2017, at 4:11 PM, Vladimir Rodionov <vladrodionov@gmail.com>
> wrote:
> >
> > Hmm, the next on your list (of kicked out from branch v2) should be AM
> v2 I
> > presume?
> >
> > -Vlad
> >
> >> On Sat, Sep 9, 2017 at 4:04 PM, stack <saint.ack@gmail.com> wrote:
> >>
> >> In spite of repeated requests for eng summary of state of this feature
> --
> >> summary of what is in 2.0, what is not, what the capabilities are, how
> well
> >> it has been tested and at what scale -- all I get, when the requests are
> >> not ignored, are pointers to lists of ill-describing jiras and some
> pending
> >> user facing doc update.
> >>
> >> For other features, mob or region server groups, I know that they have
> been
> >> running at scale in production for as much as a year and more. I have
> some
> >> confidence these items basically work.  For backup/restore I have no
> such
> >> sense even after spending time in review and trying to use the feature.
> >>
> >> As release manager, I have say over what makes it into a release.
> Unless
> >> the work is done to convince me that backup/restore is more than a lump
> of
> >> code and a few unit tests that can pass on some fellows laptop, I am
> going
> >> to kick it out of branch-2.  Let the feature harden more in master
> branch
> >> before it ships in a release.
> >>
> >> S
> >>
> >> On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <vladrodionov@gmail.com>
> >> wrote:
> >>
> >>>>> Have I grasped the state of things correctly, Vlad?
> >>>
> >>> Josh, the only thing which is still pending is doc update. All other
> >>> features are good to have but not a blockers for 2.0 release.
> >>>
> >>> -Vlad
> >>>
> >>> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov <
> >> vladrodionov@gmail.com
> >>>>
> >>> wrote:
> >>>
> >>>>>> What testing and at what
> >>>>>> scale has testing been done?
> >>>>
> >>>> Do we have have that for other features?
> >>>>
> >>>>
> >>>> On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov <
> >>> vladrodionov@gmail.com
> >>>>> wrote:
> >>>>
> >>>>>>> It asks: "How do I figure what of backup/restore feature
is going
> >> to
> >>>>> be in
> >>>>>>> hbase-2.0.0?
> >>>>>
> >>>>> Hmm, wait for doc update.
> >>>>>
> >>>>>
> >>>>>> On Fri, Sep 8, 2017 at 2:39 PM, Stack <stack@duboce.net>
wrote:
> >>>>>>
> >>>>>> HBASE-14414 is a JIRA with a list of random seeming issues w/
> >>>>>> non-descript
> >>>>>> summaries: "Add nonce support to TableBackupProcedure, BackupID
must
> >>>>>> include backup set name, ...". The last comment in that issue
is
> from
> >>>>>> July.
> >>>>>> It asks: "How do I figure what of backup/restore feature is
going to
> >> be
> >>>>>> in
> >>>>>> hbase-2.0.0? Thanks Vladimir Rodionov
> >>>>>> <https://issues.apache.org/jira/secure/ViewProfile.jspa?
> >> name=vrodionov
> >>>>>>> ."
> >>>>>> to which there is no answer.  Doc update is TODO.
> >>>>>>
> >>>>>> Where is the summary of the capability in hbase-2? What testing
and
> >> at
> >>>>>> what
> >>>>>> scale has testing been done? Is this 'stable or experimental'?
If I
> >>> can't
> >>>>>> get basic info on this feature though I ask repeatedly, what
hope
> >> does
> >>>>>> the
> >>>>>> poor old operator have?
> >>>>>>
> >>>>>> St.Ack
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov <
> >>>>>> vladrodionov@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> HBASE-14414
> >>>>>>>
> >>>>>>>> On Fri, Sep 8, 2017 at 1:14 PM, Stack <stack@duboce.net>
wrote:
> >>>>>>>>
> >>>>>>>> Where do I go to get the current status of this feature?
Looking
> >> in
> >>>>>> JIRA
> >>>>>>> I
> >>>>>>>> see loads of issues open against backup including some
against
> >>>>>>> hbase-2.0.0
> >>>>>>>> and no progress being made that I can discern.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> S
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On Wed, Nov 23, 2016 at 8:52 AM, Stack <stack@duboce.net>
wrote:
> >>>>>>>>>
> >>>>>>>>> On Tue, Nov 22, 2016 at 6:48 PM, Stack <stack@duboce.net>
> >> wrote:
> >>>>>>>>>
> >>>>>>>>>> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov
<
> >>>>>>>>>> vladrodionov@gmail.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>>>> and/or he answered most of the review
feedback
> >>>>>>>>>>>
> >>>>>>>>>>> No, questions are still open, but I do not
see any blockers
> >> and
> >>>>>> we
> >>>>>>> have
> >>>>>>>>>>> HBASE-16940 to address these questions.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> Agree. No blockers but stuff that should be
dealt with (No one
> >>>>>> will
> >>>>>>> pay
> >>>>>>>>>> me any attention once merge goes in -- smile).
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>> Let me clarify the above. I want review addressed
before merge
> >>>>>> happens.
> >>>>>>>>> Sorry if any confusion.
> >>>>>>>>> St.Ack
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> St.Ack
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj
Das <
> >>>>>> ddas@hortonworks.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi Stack, hats off to you for spending
so much time on
> >> this!
> >>>>>>> Thanks!
> >>>>>>>>>>> From
> >>>>>>>>>>>> my understanding, Vlad has raised follow-up
jiras for the
> >>>>>> issues
> >>>>>>> you
> >>>>>>>>>>>> raised, and/or he answered most of the
review feedback. So,
> >>> do
> >>>>>> you
> >>>>>>>>>>> think we
> >>>>>>>>>>>> could do a merge vote now?
> >>>>>>>>>>>> Devaraj.
> >>>>>>>>>>>> ________________________________________
> >>>>>>>>>>>> From: Vladimir Rodionov <vladrodionov@gmail.com>
> >>>>>>>>>>>> Sent: Monday, November 21, 2016 8:34
PM
> >>>>>>>>>>>> To: dev@hbase.apache.org
> >>>>>>>>>>>> Subject: Re: [DISCUSSION] Merge Backup
/ Restore - Branch
> >>>>>>> HBASE-7912
> >>>>>>>>>>>>
> >>>>>>>>>>>>>> I have spent a good bit of time
reviewing and testing
> >> this
> >>>>>>>> feature.
> >>>>>>>>>>> I
> >>>>>>>>>>>> would
> >>>>>>>>>>>>>> like my review and concerns
addressed and I'd like it to
> >>> be
> >>>>>>> clear
> >>>>>>>>>>> how;
> >>>>>>>>>>>>>> either explicit follow-on issues,
pointers to where in
> >> the
> >>>>>> patch
> >>>>>>>> or
> >>>>>>>>>>> doc
> >>>>>>>>>>>> my
> >>>>>>>>>>>>>> remarks have been catered to,
etc. Until then, I am
> >>> against
> >>>>>>>> commit.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Stack, mega patch review comments will
be addressed in the
> >>>>>>> dedicated
> >>>>>>>>>>> JIRA:
> >>>>>>>>>>>> HBASE-16940
> >>>>>>>>>>>> I have open several other JIRAs to address
your other
> >>> comments
> >>>>>> (not
> >>>>>>>> on
> >>>>>>>>>>>> review board).
> >>>>>>>>>>>>
> >>>>>>>>>>>> Details are here (end of the thread):
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14123
> >>>>>>>>>>>>
> >>>>>>>>>>>> Let me know what else should we do to
move merge forward.
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Vlad
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Nov 18, 2016 at 4:54 PM, Stack
<stack@duboce.net>
> >>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:53 PM,
Ted Yu <
> >>> yuzhihong@gmail.com
> >>>>>>>
> >>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks, Matteo.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> bq. restore is not clear if
given an incremental id it
> >>>>>> will do
> >>>>>>>> the
> >>>>>>>>>>> full
> >>>>>>>>>>>>>> restore from full up to that
point or if i need to
> >> apply
> >>>>>>> manually
> >>>>>>>>>>>>>> everything
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The restore takes into consideration
of the dependent
> >>>>>>> backup(s).
> >>>>>>>>>>>>>> So there is no need to apply
preceding backup(s)
> >>> manually.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> I ask this question on the issue.
It is not clear from
> >> the
> >>>>>> usage
> >>>>>>> or
> >>>>>>>>>>> doc
> >>>>>>>>>>>> how
> >>>>>>>>>>>>> to run a restore from incremental.
Can you fix in doc and
> >>>>>> usage
> >>>>>>> how
> >>>>>>>>>>> so I
> >>>>>>>>>>>>> can be clear and try it. Currently
I am stuck verifying a
> >>>>>> round
> >>>>>>>> trip
> >>>>>>>>>>>> backup
> >>>>>>>>>>>>> restore made of incrementals.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> S
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:48
PM, Matteo Bertozzi <
> >>>>>>>>>>>>> theo.bertozzi@gmail.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I did one last pass to the
mega patch. I don't see
> >>>>>> anything
> >>>>>>>> major
> >>>>>>>>>>>> that
> >>>>>>>>>>>>>>> should block the merge.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> - most of the code is isolated
in the backup package
> >>>>>>>>>>>>>>> - all the backup code is
client side
> >>>>>>>>>>>>>>> - there are few changes
to the server side, mainly
> >> for
> >>>>>>>> cleaners,
> >>>>>>>>>>> wal
> >>>>>>>>>>>>>>> rolling and similar (which
is ok)
> >>>>>>>>>>>>>>> - there is a good number
of tests, and an integration
> >>>>>> test
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> the code seems to have still
some left overs from the
> >>> old
> >>>>>>>>>>>>> implementation,
> >>>>>>>>>>>>>>> and some stuff needs a cleanup.
but I don't think
> >> this
> >>>>>> should
> >>>>>>>> be
> >>>>>>>>>>> used
> >>>>>>>>>>>>> as
> >>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>> argument to block the merge.
I think the guys will
> >> keep
> >>>>>>> working
> >>>>>>>>>>> on
> >>>>>>>>>>>> this
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>> they may also get help of
others once the patch is in
> >>>>>> master.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I still have my concerns
about the current
> >> limitations,
> >>>>>> but
> >>>>>>>>>>> these are
> >>>>>>>>>>>>>>> things already planned for
phase 3, so some of this
> >>>>>> stuff may
> >>>>>>>>>>> even be
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>> the final 2.0.
> >>>>>>>>>>>>>>> but as long as we have a
"current limitations"
> >> section
> >>>>>> in the
> >>>>>>>>>>> user
> >>>>>>>>>>>>> guide
> >>>>>>>>>>>>>>> mentioning important stuff
like the ones below, I'm
> >> ok
> >>>>>> with
> >>>>>>> it.
> >>>>>>>>>>>>>>> - if you write to the table
with
> >> Durability.SKIP_WALS
> >>>>>> your
> >>>>>>>> data
> >>>>>>>>>>> will
> >>>>>>>>>>>>> not
> >>>>>>>>>>>>>>> be in the incremental-backup
> >>>>>>>>>>>>>>> - if you bulkload files
that data will not be in the
> >>>>>>>> incremental
> >>>>>>>>>>>>> backup
> >>>>>>>>>>>>>>> (HBASE-14417)
> >>>>>>>>>>>>>>> - the incremental backup
will not only contains the
> >>>>>> data of
> >>>>>>>> the
> >>>>>>>>>>>> table
> >>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>> specified but also the regions
from other tables that
> >>>>>> are on
> >>>>>>>> the
> >>>>>>>>>>> same
> >>>>>>>>>>>>> set
> >>>>>>>>>>>>>>> of RSs (HBASE-14141) ...maybe
a note about security
> >>>>>> around
> >>>>>>> this
> >>>>>>>>>>> topic
> >>>>>>>>>>>>>>> - the incremental backup
will not contains just the
> >>>>>> "latest
> >>>>>>>> row"
> >>>>>>>>>>>>> between
> >>>>>>>>>>>>>>> backup A and B, but it will
also contains all the
> >>> updates
> >>>>>>>>>>> occurred in
> >>>>>>>>>>>>>>> between. but the restore
does not allow you to
> >> restore
> >>>>>> up to
> >>>>>>> a
> >>>>>>>>>>>> certain
> >>>>>>>>>>>>>>> point in time, the restore
will always be up to the
> >>>>>> "latest
> >>>>>>>>>>> backup
> >>>>>>>>>>>>>> point".
> >>>>>>>>>>>>>>> - you should limit the number
of "incremental" up
> >> to N
> >>>>>> (or
> >>>>>>>> maybe
> >>>>>>>>>>>>> SIZE),
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> avoid replay time becoming
the bottleneck.
> >>> (HBASE-14135)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I'll be ok even with the
above not being in the final
> >>>>>> 2.0,
> >>>>>>>>>>>>>>> but i'd like to see as blocker
for the final 2.0 (not
> >>> the
> >>>>>>>> merge)
> >>>>>>>>>>>>>>> - the backup code moved
in an hbase-backup module
> >>>>>>>>>>>>>>> - and some more work around
tools, especially to try
> >>> to
> >>>>>>> unify
> >>>>>>>>>>> and
> >>>>>>>>>>>> make
> >>>>>>>>>>>>>>> simple the backup experience
(simple example: in some
> >>>>>> case
> >>>>>>>> there
> >>>>>>>>>>> is a
> >>>>>>>>>>>>>>> backup_id argument in others
a backupId argument. or
> >>>>>> things
> >>>>>>>>>>> like..
> >>>>>>>>>>>>>> restore
> >>>>>>>>>>>>>>> is not clear if given an
incremental id it will do
> >> the
> >>>>>> full
> >>>>>>>>>>> restore
> >>>>>>>>>>>>> from
> >>>>>>>>>>>>>>> full up to that point or
if i need to apply manually
> >>>>>>>> everything).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> in conclusion, I think we
can open a merge vote. I'll
> >>> be
> >>>>>> +1
> >>>>>>> on
> >>>>>>>>>>> it,
> >>>>>>>>>>>> and
> >>>>>>>>>>>>> I
> >>>>>>>>>>>>>>> think we should try to reject
-1 with just a "code
> >>>>>> cleanup"
> >>>>>>>>>>>> motivation,
> >>>>>>>>>>>>>>> since there will still be
work going on on the code
> >>>>>> after the
> >>>>>>>>>>> merge.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Matteo
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Sun, Nov 6, 2016 at 10:54
PM, Devaraj Das <
> >>>>>>>>>>> ddas@hortonworks.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Stack and others, anything
else on the patch? Merge
> >>> to
> >>>>>>> master
> >>>>>>>>>>> now?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message